Links2Linux Wiki

This is an old revision of the document!

---

SecureCopy (scp) is often used for copying files between unix machines nowadays. What if you have a really huge file and such a transfer got interrupted?

ftp and http both can do partial downloads of files so that you can continue downloading a file where it got interupted to save bandwidth and time.

But with SSH? Hm. SSH/SCP do not have someting like that.

The follwoing little shellscript can do it. It supports downloading, uploading and even transferring a file from one remote host to another!

Call it like “scp+ user@example.com:/home/foo/mybigfile /my/local/file” to download a file using ssh.

If the file exists on the destination and is smaller than the source file file, the remaining bytes will be appended to the destination file.

  - !/bin/bash
  -  scp+.sh
  -  get/put files via ssh and
  -  continue partial downloads/uploads or host-to-host transfers
  - 
  -  Copyright (c) 2005-2008 by Marc Schiffbauer
  - 
  -  Version 2.0
  - 

SSH_OPTS="-o ServerAliveInterval=5 -o ServerAliveCountMax=3"
BLOCK_SIZE="1500" # Bytes

  -  URL like "user@host:path/to/file"
SRC_URL="$1"
SRC_URL_HOST="${SRC_URL%%:*}"
SRC_URL_FILE="${SRC_URL##*:}"

DST_URL="${2:-$(basename $SRC_URL_FILE)}"
DST_URL_HOST="${DST_URL%%:*}"
DST_URL_FILE="${DST_URL##*:}"

if [[ -z "$SRC_URL" || -z "$DST_URL" ]]; then
<code>
  echo ""
  echo "  Syntax: $(basename $0) <src> [[<dst>]]"
  echo ""
  echo "    src/dst: [[[user@]]host:]/path/to/file"
  exit 1

fi

if $SRC_URL_HOST == $SRC_URL_FILE; then

  SRC_CMD="sh -c"

else

  SRC_CMD="ssh $SSH_OPTS $SRC_URL_HOST"

fi if $DST_URL_HOST == $DST_URL_FILE; then

  DST_CMD="sh -c"

else

  DST_CMD="ssh $SSH_OPTS $DST_URL_HOST"

fi

DST_SIZE=$($DST_CMD “if -f '$DST_URL_FILE' ; then find '$DST_URL_FILE' -printf '%s'; else echo 0; fi”) PADDING=$¹⁾

$SRC_CMD “if $PADDING -ne $BLOCK_SIZE ; then dd if='$SRC_URL_FILE' ibs=1 skip=$DST_SIZE status=noxfer count=$PADDING 2>/dev/null; PADDING=$PADDING; \

  else PADDING=0; fi; dd if='$SRC_URL_FILE' bs=$BLOCK_SIZE skip=\$((($DST_SIZE+PADDING)/$BLOCK_SIZE))" | $DST_CMD "cat >> $DST_URL_FILE"

exit $? </code>