Friday, August 19, 2011

SSH cipher speed

When setting up backups over SSH (e.g. rsnapshot with rsync over SSH), it's important to know that the default SSH cipher isn't necessarily the fastest one. In this case, the CPU-based encryption is the performance bottleneck, and making it faster means getting faster backups.
A test (copying a 440 MB file between a fast Xeon CPU (fast=no bottleneck there) and an Atom based NAS) shows that the arcfour family of ciphers are clearly the fastest in this setup:
cipherreal timeuser timebandwidth
arcfour0m9.639s0m7.423s45.7 MB/s
arcfour1280m9.751s0m7.483s45.1 MB/s
arcfour2560m9.856s0m7.764s44.7 MB/s
blowfish-cbc0m13.093s0m10.909s33.6 MB/s
aes128-cbc0m22.565s0m20.129s19.5 MB/s
aes128-ctr0m25.400s0m22.951s17.3 MB/s
aes192-ctr0m28.047s0m25.771s15.7 MB/s
3des-cbc0m51.067s0m48.018s8.6 MB/s

The default configuration of openssh uses aes128-ctr, so changing the cipher to arcfour gets me a 2.5-fold increase in bandwidth here ! Use the "Ciphers" keyword in .ssh/config or the "-c" command line parameter to change the order of preference of the available ciphers. YMMV.

As a reference (cfr. deinoscloud's comment), I ran "nc -l -p 3333" on the Atom side, and ran "cat file | nc atom 3333" on the Xeon:
cipherreal timeuser timebandwidth
cleartext0m4.135s0m0.311s106.5 MB/s
. This shows that in the cleartext case, the CPU (user) time is not the bottleneck, and we're very close to using the full 1Gbps bandwidth.








2 comments:

deinoscloud said...

And what would be the speed without any encryption?

bert said...

@deinoscloud near full Gbps. I'm testing a cached file over the network to /dev/null, so that makes sense. Once the CPU bottleneck is gone, the network speed is the next bottleneck.