• Answers
• SSH
• Data

The benefits of scp/sftp

The primary benefit of using scp and sftp is that they work in all situations where ssh works; this is especially important given that ssh tends to be available by default on most UNIX (and UNIX-like) machines.

Unlike ftp-based transfers, scp and sftp are fully encrypted; while there is a high danger of someone else listening in on your ftp session (including the unwanted acquisition of your password), the danger is almost completely mitigated by using scp as an alternative. This encryption does come at a cost-- processing power. Slow computers, especially ones made before the turn of the century, will slow noticeably when attempting to transfer a file via scp or sftp at full speed.

While most machines running ssh servers tend to be UNIX hosts, non-UNIX hosts have a wide variety of scp and sftp implementations available to them. This makes scp and sftp extremely versatile tools for cross-platform file transfers.

scp vs. sftp

These two mechanisms provide the same essential functionality. Many GUI clients for Windows and Mac OSX provide the exact same interface, but can use either scp or sftp under the hood; the only time you will notice a difference is if you are using command-line implementations, such as the OpenSSH clients. In that case, scp acts like a drop-in replacement for the UNIX cp program, which means it tends to be used to copy a specific file (or files) to a specific directory; sftp behaves more similarly to the UNIX ftp program, which is used to browse through directory trees on the remote host, and transfer files to the current working directory on the local host.

They are two sides to the same coin; other than personal preference, there is no compelling reason to choose one over the other.

Caveats about ciphers

Data transfers are one usage of ssh where special attention to the default list of ciphers can pay big dividends, performance-wise, especially on older hardware. On average, ciphers that are cryptographically weaker (less secure) will have higher performance than stronger ciphers. If your computer is fast enough (most modern computers are fast enough), the performance difference is negligible.

The weakest cipher, arcfour, tends to offer the best performance. In processor-limited situations, the use of arcfour could improve your transfer rate by 20% or more. Due to the weak nature of the arcfour cipher, it is not recommended that you use this option for any transfers where security is a strong concern.

A good middle-ground alternative is the blowfish cipher. While still a strong cipher, blowfish tends to offer substantial performance improvements over the aes cipher suite, which is the default in modern ssh clients.

Here are some sample transfer rates between a modern (circa 2006) Linux workstation and an older Solaris UltraSPARC II workstation, via a 100BT network connection:

Caveats about compression

If your data is already compressed (such as a .gz, .zip, .jpeg, or .mpeg file), there is no merit in enabling compression for your data transfer. If your data is not compressed (megabytes of flat text, or sparsely populated FITS files), you could see a performance improvement up to an order of magnitude by enabling compression for your scp or sftp session.

Transferring data from remote hosts

Since scp and sftp piggy-back on the ssh protocol for their normal operation, are subject to the same restrictions (or benefits) as ssh itself. In particular, this means that if you need to transfer data using scp or sftp from a remote host to filesystems on the UCO/Lick network, you need to do so via the ssh gateways at ssh.ucolick.org. Please see this page for additional details on restrictions to remote access.

The ssh gateways do not have a lot of disk space. In fact, your user account on these hosts is restricted to a one megabyte disk quota. If you want to scp or sftp data onto our network, you need to specify an alternate location, generally a shared NFS partition, for your incoming data. Here are two examples, using syntax appropriate to the OpenSSH command-line scp client:

scp ./myfile.tar.gz username@ssh.ucolick.org:/net/myhostname/data2/mybigdata/
scp ./myfile.tar.gz username@ssh.ucolick.org:/net/myhostname/u/myusername/ 

A variety of automounted NFS partitions are available from the ssh gateways; essentially, any /net, /home, /san, or /usr/local path is valid. For example:

scp -c arcfour ./myfile.tar.gz username@ssh.ucolick.org:/usr/local/ftp/outgoing/username/
scp -c arcfour ./myfile.tar.gz username@ssh.ucolick.org:/home/public/username/

The optional -c arcfour in these last two examples demonstrates the use of the arcfour cipher.