SFTP: cannot transfer a 1Gb file

Help & Support
1

Hi,

Today I tried to transfer a 1Gb file using SFTP in DOpus 12 but after some time it always fails, with different errors:

===
Started a shell/command
SSH: CTS CONNECTED
SSH: Receiving resource.tar.gz
Timeout on Command 1018.
SSH: Error while reading: did not receive a valid SFTP packet

Or simply:

===
SSH: Receiving resource.tar.gz
SSH: Error while reading: did not receive a valid SFTP packet

I tried to transfer the same file using Filezilla and it worked the first time, without any error.

So I tried again with DOpus and always stopped with one of the above errors. The max I could download is 160Mb out of 1Gb.

2

Any update on this topic?

I paid for a feature I cannot use :frowning:

Transfers stop before the end. Here is a log from this morning:

====
SSH: Receiving neo4j.tgz
Timeout on Command 1018.
SSH: Error while reading: did not receive a valid SFTP packet
Connection closed
Opening Connection production.onmap-visual.com:22
60489728 bytes received in 217.406 seconds (271.496 Kbytes/sec)
SSH: ssh_init: error during SSH connection setup 0

I don't have any problems with Filezilla transferring the same files with the same options.

3

Is it the size or the length of time it takes? e.g. Does it stop at different sizes but always after the same amount of time?

Have you tried with any antivirus/firewall software disabled, in case it is treating connections made by Opus differently to ones made by FileZilla?

Does it happens with all servers or just a particular server (machine) or version of server (server software)?

Additional:

I've just tested sending and re-downloading a 2.5GB file over SFTP without issues, so it should work in general.

I noticed from the FTP log that it appears to re-initialise the connection during the transfer (due to "too much data received" errors, which may be specific to the server I'm trying with; the same server also regularly dropped transfers from linux clients, which didn't recover as well as Opus did if it's the same issue). If your SFTP server is set up to only allow one connection at a time, it might be significant, but may not be as well, if it's specific to the server I'm testing with.

4

Here are somes tests attempting to transfer the same file that's 120Mb long:

1st attempt:

Transfer window closes after only 11.5Mb transfered out of 120Mb. Here is the log:

SSH: Receiving neo4j.tgz 
Timeout on Command 1018.
SSH: Error while reading: did not receive a valid SFTP packet
Connection closed
Opening Connection production.onmap-visual.com:22
12156928 bytes received in 99.437 seconds (119.144 Kbytes/sec)
SSH: ssh_init: error during SSH connection setup 0

2nd attempt:

Transferred worked and filesize is correct.

3rd attempt:

Transfer window closes after only 10Mb transferred out of 120Mb. Log:

SSH: Receiving neo4j.tgz 
Timeout on Command 1018.
SSH: Error while reading: did not receive a valid SFTP packet
Connection closed
Opening Connection production.onmap-visual.com:22
10518528 bytes received in 93.828 seconds (109.384 Kbytes/sec)
SSH: ssh_init: error during SSH connection setup 0
Connection closed

Note that my connection asks for my ssh key password each time I try to connect for the first time (I chose not to save it for now), but when the transfer stops, nothing happens: the window simply closes without any error.

Each time, the transfer slows down immediately, then stalls for a few seconds, and then the transfer window simply disapears (without any error).

I am not using any other server but this one: I'll do more tests on other servers. I had not antivirus other than the build-in windows 10 defender which I de-activated before running these tests.

One more problem.

Sometimes, when I log into my SFTP server, change to a directory, then drag a file to download it, I immediately get this error:

"An error occurred copying 'file.tgz': The directory or file cannot be created. (82)"

I can "Retry" "Skip", or "Abort".

If I choose "Retry", the connection is restarted (I need to re-enter the password of my ssh key) and the transfer starts.

5

Instead of drag & drop, please try using a dual-display, selecting files and using the Copy Files button on the toolbar to copy them to the other side.

Drag & drop adds some extra complications, so it'd be good to rule them out.

6

The drag & drop I was doing was between two sides of a dual-display but I'll do more tests using the copy button.

7

I did the same tests using the Copy Files button and am having the same problems, even with a much smaller file (like 150Mb).

Here is the log:

pening Connection devv6.onmap-visual.com:22
Server version: SSH-2.0-OpenSSH_6.7p1 Debian-5+deb8u3
Using SSH protocol version 2
We claim version: SSH-2.0-PuTTY_Directory_Opus
Server supports delayed compression; will try this later
Doing Diffie-Hellman group exchange
Doing Diffie-Hellman key exchange with hash SHA-256
Host key fingerprint is:
ssh-rsa 2048 3f:2b:6a:64:f8:0b:18:18:2f:18:25:b5:a6:22:37:1a
Initialised AES-256 SDCTR client->server encryption
Initialised HMAC-SHA-256 client->server MAC algorithm
Initialised AES-256 SDCTR server->client encryption
Initialised HMAC-SHA-256 server->client MAC algorithm
Reading private key file "***"
Pageant is running. Requesting keys.
Pageant has 1 SSH-2 keys
Configured key file not in Pageant
Offered public key
Offer of public key accepted
Sent public key signature
Access granted
Initiating key re-exchange (enabling delayed compression)
Opening session as main channel
Doing Diffie-Hellman group exchange
Doing Diffie-Hellman key exchange with hash SHA-256
Initialised AES-256 SDCTR client->server encryption
Initialised HMAC-SHA-256 client->server MAC algorithm
Initialised zlib (RFC1950) compression
Initialised AES-256 SDCTR server->client encryption
Initialised HMAC-SHA-256 server->client MAC algorithm
Initialised zlib (RFC1950) decompression
Opened main channel
Started a shell/command
SSH: CTS CONNECTED
SSH: Receiving neo4j-db-dev.tar.gz 
Timeout on Command 1018.
SSH: Error while reading: did not receive a valid SFTP packet
Connection closed
Opening Connection devv6.onmap-visual.com:22
104660992 bytes received in 312.922 seconds (326.176 Kbytes/sec)
SSH: ssh_init: error during SSH connection setup 0

I have no problems at all transferring the same file with WinScp or Filezilla.

8

Is there a way we could access the same server?