Recently, while testing the customized firmware for the RK3588 (From Bricked to Rebirth: My RK3588 Development Board Embarks on the Journey of Modifying Ubuntu), I frequently encountered file transfer issues. We previously set up an Nginx download server (Creating a Download Page for OpenVPN Configuration Files Using Nginx) and tested downloads at full speed; we also set up an NFS server (Configuring an NFS Server Using PVE 8.0’s Debian 12 System) and tested file transfers at full speed as well.
However, for the testing environment, setting up a file transfer server might introduce configuration remnants, so I have been using SCP for transfers (Implementing NFS Network Sharing by Mounting exFAT File System to Linux), but the downside is that it is slow.
If you say SCP performance is lacking, in 2023, during an SD-WAN project, China Mobile’s SCP remote data transfer rate could reach 800 Mbps. I don’t know which vendor is so impressive, but at least it shows that the SCP protocol is not the issue.
After research, I found that there are some tunable options in the SCP configuration. For detailed options, please refer to (Say Goodbye to FTP! A Comprehensive Guide to SCP Command for Secure SSH Transfers), today we will only introduce those that affect transfer efficiency.
First, there is bandwidth limiting, which can be done using the -l option to specify the bandwidth used by SCP in Kbit/s.
scp -l 200000 [email protected]:/root/15.0.0_64only-latest.tar .As we can see, without bandwidth limiting, it can reach 100 MB/s, while limiting to 200000 Kbps results in a transfer rate of 24.3 MB/s, and limiting to 400000 Kbps results in a transfer rate of 48.4 MB/s, which is quite precise.
We previously tested the encryption algorithms of IPsec (Testing the Performance Differences of Various IPsec VPN Encryption Algorithms Using vSRX) and also tested the encryption algorithms of OpenVPN (Comparative Analysis of Different Encryption Algorithms in OpenVPN), using the -v parameter allows SCP to print debugging messages about its progress. We can see that when no encryption algorithm is specified, the default encryption algorithm is [email protected].
For comparison, we can use the -c option to select the algorithm used for encrypting data transfers. The list of available algorithms can be obtained using “ssh -Q cipher”.
The OpenSSH version on Ubuntu 22.04 is 8.9p1, and on Ubuntu 24.04 it is 9.6p1. By checking the commands, the supported encryption algorithms for both versions are the same.
Let’s first try the most complex 3des-cbc algorithm.
scp -c 3des-cbc [email protected]:/root/15.0.0_64only-latest.tar .
Right off the bat, it failed, indicating that the other end only supports aes128-ctr, aes192-ctr, aes256-ctr, [email protected], [email protected], and [email protected]. Quite a few options, but inconsistent. So let’s test the supported algorithms.
scp -c aes128-ctr [email protected]:/root/15.0.0_64only-latest.tar .scp -c aes192-ctr [email protected]:/root/15.0.0_64only-latest.tar .scp -c aes256-ctr [email protected]:/root/15.0.0_64only-latest.tar .scp -c [email protected] [email protected]:/root/15.0.0_64only-latest.tar .scp -c [email protected] [email protected]:/root/15.0.0_64only-latest.tar .scp -c [email protected] [email protected]:/root/15.0.0_64only-latest.tar .
As we can see, the lowest transfer rate is actually the default [email protected] algorithm, while the highest transfer rate is the aes192-ctr algorithm, which can increase the transfer bandwidth by about 21.4%.
Meanwhile, we observe the host’s performance and find that due to the involvement of encryption and decryption, the CPU usage remains high, with SCP consuming no less than 125%. I wonder if increasing the number of CPU cores can improve transfer bandwidth, so let’s try that.
In addition, the documentation mentions that by default, a single SFTP read/write buffer used during download or upload is 32KB. We can use the -X option to control the maximum buffer size. For comparison, let’s test the effects from 99 to 999999.
scp -X buffer=99 [email protected]:/root/15.0.0_64only-latest.tar .scp -X buffer=999 [email protected]:/root/15.0.0_64only-latest.tar .scp -X buffer=9999 [email protected]:/root/15.0.0_64only-latest.tar .scp -X buffer=99999 [email protected]:/root/15.0.0_64only-latest.tar .scp -X buffer=999999 [email protected]:/root/15.0.0_64only-latest.tar .
As we can see, as the buffer size increases, the transfer bandwidth continues to improve, reaching a maximum of 100.8 MB/s, which is about a 7% increase compared to the default situation.
Of course, there is also the -C option to enable compression, but I wonder if it can improve performance.
In actual testing, the results were quite the opposite. After enabling data compression, due to consuming more CPU resources, the transfer bandwidth was lower, and there was no significant change in file size and transfer time. This option is ineffective.
The manual also mentions that since OpenSSH 9.0, SCP defaults to using the SFTP protocol for transfers. If you want to use the traditional SCP protocol for file transfers, you can specify the -O option.
However, in actual testing, the transfer bandwidth did not change significantly. By checking the details, we found that the default encryption algorithm used is also [email protected], which is likely a CPU performance bottleneck.
Now, testing shows that the real performance improvements come from modifying the encryption algorithm and buffer size. Let’s combine these two options.
scp -c aes128-ctr -X buffer=99999 [email protected]:/root/15.0.0_64only-latest.tar .scp -c aes192-ctr -X buffer=99999 [email protected]:/root/15.0.0_64only-latest.tar .scp -c aes256-ctr -X buffer=99999 [email protected]:/root/15.0.0_64only-latest.tar .
Well, there is some effect; at least the gigabit network can be fully utilized. Finally, let’s try it on ESXi.
The effect is not obvious; the CPU usage did not increase, and there are still only two tasks: sshd and sftp-server, unable to share the load. It seems the CPU frequency is again a bottleneck.
Let’s try the parameters that previously yielded the highest data.
scp -c aes192-ctr -X buffer=99999 [email protected]:/root/15.0.0_64only-latest.tar .
Successfully breaking through the gigabit barrier, reaching 1400 Mbps, which can be paired with a 2.5G wired network port and WiFi 6 for use.
**Recommended Reading***
From Bricked to Rebirth: My RK3588 Development Board Embarks on the Journey of Modifying Ubuntu16-Core CPU Running at 100%! Testing Six Vulnerability Scanning Modes with 21 Nodes, This Mode Makes the Server Go WildRunning Docker on Windows, 8GB of Memory Can’t Satisfy the Vulnerability Scanning Tool?Tencent App Treasure Conquers Windows! Major Changes in Installing Android Apps on Windows by 2025, 5 Minutes to Set Up Xiaohongshu/King of GloryTwo Commands to Solve It! Kali’s Native Integration of GVM Vulnerability Scanner, Saving 5GB of Space Compared to DockerDocker on Windows! A Step-by-Step Guide to Deploying Redroid Cloud PhoneHigh-Performance Game Cloud Phone Revolution! GPU Acceleration Makes Redroid Performance Comparable to Xiaomi 15Automatically Install the System by Plugging in a USB Drive? Master the Creation of Automatic Installation Images for Ubuntu Server Edition in One ArticleSwitch IP Just by Connecting to WiFi! Unveiling the Black Technology of Multi-VPN Exit in Enterprise Routers!Global Deployment of Cloud Phones: Achieving Network Freedom for Docker Containers with Policy RoutingVMware Edge 620 Magic Operation: Flashing iStoreOS to Become a Versatile Enterprise Gateway, Easily Handling Multiple SSIDsFrom CentOS to Ubuntu: Zero-Cost Migration of L2TP VPN, Practical Experience in Enterprise Intranet Penetration!WireGuard Too Complicated? Teach You to Easily Achieve Global Networking with Netmaker in Ten MinutesCloud Phone Technology Unveiled! Low-Cost Implementation of Turning One Computer into Hundreds of “Virtual Phones” in PracticeiWAN Tunnel Testing: One Handshake Achieves Full 2.3Gbps, Can Free SD-WAN Really Compete with Dedicated Lines?