SFTP file transfers slow

Help & Support
1

I am having trouble with SFTP transfers in Directory Opus (v12.3, Win10 64) being very slow.

Copying the same 25GB file to/from the same destination drives across a gigabit LAN, from Windows host to Ubuntu server gave me these results

Copying a 24.4GB file to/from Windows/Ubuntu

From     To       DOpus      FileZilla (sftp) v3.23.0.2
--------------------------------------
Windows  Ubuntu   6.8MB/sec  110MB/sec (dropping to 90MB/sec)
Ubuntu   Windows  33MB/sec   110MB/sec

PSCP (beta 0.67) running on Windows
From     To       Speed
-----------------------
Windows  Ubuntu   45MB/sec average
Ubuntu   Windows  52MB/sec average

PSFTP (beta 0.67) running on Windows
From     To       Time   Speed
------------------------------
Windows  Ubuntu   9m30s  ~44MB/sec average
Ubuntu   Windows  7m54s  ~53MB/sec average

DOpus SFTP vs other tools summary
Tool                MB/sec  DOpus MB/sec  X times slower
--------------------------------------------------------
PSCP Win->Lin       45      7             ~6.5
PSCP Lin->Win       52      33            ~1.5
PSFTP Win->Lin      44      7             ~6
PSFTP Lin->Win      53      33            ~1.6
FileZilla Win->Lin  110     7             ~15.5
FileZilla Win->Lin  90      7             ~12.8
FileZilla Lin->Win  110     33            ~3.3

Speeds are in megabytes per second.

DOpus is 12-15 times slower when copying files from Windows to an SFTP host, and 2-3 times slower the other way around, compared to SFTP speeds in FileZilla. The destination and source drives were the same each time in both directions. When dealing with remote hosts there is obviously less actual throughput but still pretty damn slow and FileZilla wins by a large margin. It also seems as though DOpus is caching file listings - despite opening a new window and a new SFTP connection the file copied via FileZilla was missing until I hit refresh.
PSFTP doesn't show progress so I timed how long each copy took and worked it out from there, so apologies if I screwed up the maths (it is not my strong point). Admittedly, FileZilla seems to be performing some voodoo here, but it is what I have been using for years.

This may not be the perfect methodology for testing (I should have run each test at least three times and averaged the results, for example) but it has already taken up more time than I want, and these are real-world numbers from real-world transfers, not some random benchmark.

Having paid extra for SFTP support in DOpus specifically so I don't have to use FileZilla, this is disappointing - it is essentially unusable when dealing with large files, and very slow with smaller ones. A quick look on this forum indicates that this is a perennial issue dating back to at least 2007, implying that the developers don't care enough about it to fix it as it's still an issue a decade later.

Please, please, please fix SFTP in DOpus!

2

SFTP download speed was sped up quite a lot since initial reports. I think upload may still be a little slow, although it has not been looked at in detail yet.

What you're seeing seems different, though. That amount of speed difference is much more than we observe in our own testing. (Saying that, I have not personally tested SFTP over a gigabit LAN; only over the internet or WiFi, so I can't speak to that issue myself. I don't know which tests the rest of the team have done; SFTP isn't something I've personally worked on.)

Make sure you're comparing apples to apples. i.e. FileZilla and Opus should be using the same protocol (sometimes SCP is faster for a server than SFTP for example). If FileZilla is doing split transfers (downloading multiple parts of the same file in parallel then joining them together) then that could speed it up, and Opus doesn't do that, so the speed difference could come down to that, and something making individual transfers slow while parallel ones can still go faster. Firewall software may also treat different programs differently.

3

I can confirm similar slow SFTP speeds as citlee on the current version: 12.10, x64

I configured a Samba server on the remote Linux system after I encountered slow SFTP speeds.

DO download speeds from the Linux server running Samba: 80MB/s
DO download speeds from the same Linux server over SFTP: 20MB/s
DO upload speeds to the same Linux server over SFTP: 4MB/s

Using FileZilla or Cygwin (sftp/scp/rsync -e ssh) gives synchronous 90MB/s transfer speeds.

The base Directory Opus Pro program is useful and I'm already glad I upgraded from Light, but SFTP performance is abysmal. I would not have purchased the add-on if I had know speeds were this bad.

4

I experience this with a fresh install of dopus towards a linux server.
Uploads seems to stop at just above 4MB/s
Has there been any development in this matter?
WinSCP achives 30MB/s across the same link.

5

Just tried sftp over gbit lan and noticed a big change from major 12 to 13 to the worse!

filezilla gives me this
image

and dop gives me this
image

smaller file less performance dops
image

same files with filezilla
image

6

We'll be replacing the SFTP library with a new one as the first big piece of work after we finish working through the initial feedback on Opus 13.

1 Like