DO sometimes gets stuck on "counting files" when starting a transfer from SFTP

Not easily reproducable, unfortunately. It usually happens after a few days of DO being open, and many file transfer sessions from SFTP to the local machine. All of a sudden, at the beginning of a new transfer, the "Copying" window will get stuck on the "counting files" step.

The more serious problem is that when it gets stuck, the Copying window also does NOT allow you to cancel the stuck operation. The "Abort" button does not work. You can pause the transfer (and then resume it), but it remains stuck. The only way to interrupt the stuck transfer is to kill the dopus.exe process. This is a bug- DO should always allow you to abort a stuck transfer.

Looking at the FTP log, it's pretty normal until it just ends:

Opening Connection XXX:22
Server version: SSH-2.0-OpenSSH_9.0p1-hpn15v2
Using SSH protocol version 2
We claim version: SSH-2.0-PuTTY_Directory_Opus
Server supports delayed compression; will try this later
Doing ECDH key exchange with curve Curve25519 and hash SHA-256
Server also has ecdsa-sha2-nistp521 host key, but we don't know it
Host key fingerprint is:
XXXXX
Initialised AES-256 SDCTR client->server encryption
Initialised HMAC-SHA-256 client->server MAC algorithm
Initialised AES-256 SDCTR server->client encryption
Initialised HMAC-SHA-256 server->client MAC algorithm
Using username "XXXX".
Sent password
Access granted
Initiating key re-exchange (enabling delayed compression)
Opening session as main channel
Server supports delayed compression; will try this later
Doing ECDH key exchange with curve Curve25519 and hash SHA-256
Initialised AES-256 SDCTR client->server encryption
Initialised HMAC-SHA-256 client->server MAC algorithm
Initialised zlib (RFC1950) compression
Initialised AES-256 SDCTR server->client encryption
Initialised HMAC-SHA-256 server->client MAC algorithm
Initialised zlib (RFC1950) decompression
Opened main channel
Started a shell/command
SSH: CTS CONNECTED

After about 5 minutes of being stuck, "Connection closed" appears at the end of the FTP log. Great. Except... even after the connection is closed, the Copying window remains stuck open at "counting", and Abort still does not close the window.

I can confirm that the server is garden variety modern OpenSSH, and I never had any problems with it when I connected to it through a mapped drive via SSHFS. I only experience these timeouts/stuck connections when using DO's native SFTP functionality. Pretty evident that's where the bug is.

If it is possible to enable a more verbose debug level in DO, I can turn that on and give you more detailed logs for the next time it happens.

Next time it happens, please make some process snapshots and send them to us:

Thanks, I will do that.

Many thanks for sending the snapshots.

The problem is in our code. I have a few theories about the cause but I need to find a way to reproduce the problem to test them. I'll try copying sub-dirs from SFTP sites until I can get it to happen here. (If you have any ideas about possible details that might help reproduce it, those could speed things up.)

If I can't reproduce the problem, I'll add some extra logging code in the places I think things could be going wrong and send you a test version, if that's OK. That should help us work out if any of my theories is correct. But I'll keep trying to reproduce it for a while first, as that'll be easier for both of us if I can.

Thanks again!

1 Like

I'm not sure what exactly triggers it- some combination of time open and doing a lot of SFTP transfers (and, quite possibly using the queue... though I haven't noticed if it ONLY gets stuck when starting a transfer pending in the queue- or if that doesn't matter....). But yeah, feel free to send me a debug build and I'll happily try to reproduce using that.

1 Like

Hello,
I still encounter this fairly often. Any progress on a solution?

None yet, and I wasn't able to reproduce the problem, but it is still on our list to make a debug build or similar. We need to finish some things first before we can spend proper time on it.

Hello- do you happen to know if this bug was fixed in v13? DOpus remains the only app I use that exhibits this behavior.

Sorry for digging up an old thread but this has been happening for me as well in v13.

Like stated before, it's not easily reproducible but it does happen at the start of a transfer.

The original thread was mentioning the file counting process but in my case it also happens right after the connection has started and the file is ready to be copied.

As mentioned, clicking Skip, Abort or anything else does not work, a process kill is required in order to shut down the popup.

Since I am moving quite some files from SFTP to local HDs I can have weeks where it doesn't happen and then one occurrence would have it happen twice in a row.

Not sure what to do other than hoping that this issue can be troubleshot and fixed.

We're in the process of replacing the SFTP code, so it'll probably be fixed by that.

1 Like

That is some pretty great news! I will be looking forward to playing with the new implementation once it's available!

Thank you for the news :+1:

Yes SFTP.
I have a folder of 4096 semi transparent homemade chess board gifs in a folder uploaded to a domain that I think is administered in the UK ( Regery ).
They had to help me set up the account and I think my tech support was UK.

I logged in and was able to select all 4096 using Ctrl A.
I dragged with a right click to a local folder and selected 'Copy'.
It is still working just fine and the Dopus progress dialog with graph indicates another 1 hour and 9 minutes remains in the copy. The file copy progress count is working just fine.

Perhaps it is the file sizes that cause it ?
Or something else ?

Well, FWIW that's what I have here.

Edit Note: Still no Giraffes or anything here. :laughing: