Unattended Copy/Move Retry options?

Help & Support
#1

Hi,
Seeking a way, or looking to suggest a new option, for the following situation.

I have a remote location, serviced by a solid VPN.
The nature of the connection is very good, but can experience incredibly minor interruptions. That's ok, the VPN redials if it has to automatically. (router to router setup)

Anyhow, I need to regularly move "larger" files across this.
Regular windows copy/move, when it hits an interruption gets upset as you may expect, and RETRY never works. Yet, SKIP then goes on to the next file with success as though the conneciton was never lost.

DOpus is at least copying more efficiently (I think? Looks like it is?) and if you get the same thing, RETRY doesn't resume, but at least starts re-trying the same file. With each file taking between 15-45 minutes to copy, I would LOVE IT if there was a way to trigger it to automatically retry.

If that's not presently a setting (that I have not found yet).... how about another unattended option that says: Retry on failure? and has a dropdown that maybe says how many times to retry? (1,2,3,5,10,25, Forever)

I also assume the fact that it cannot RESUME the copy when a network interruption occurs, is a requirement. Resume would be even nicer... much the way FTP can. But maybe that's why resume says "FTP only".

Thoughts?
and thanks.
Andrew

#4

An auto-retry option is interesting, but not currently available. I think the best you could do right now, if you're copying multiple large files over an unreliable network connection, would be unattended mode, just so that one file failing doesn't stop the next one from being tried.

(That said, it if takes a while for the VPN to redial, then if one file fails all the ones after it probably will as well.)

Resume support for normal file copies isn't impossible, but would impact performance (even when resume was't needed in the end), so it would need to be a special mode. It's something we've been thinking about on and off for a while, but I don't know if we'll ever implement it, as the trade-offs don't make sense for most people and situations.

If it was me doing this, I would probably split the files up so I could transfer them in chunks, then rejoin them at the other end (if needed), or maybe install an FTP or SFTP server on the destination so that I could FTP them.

#5

Yes, I'll try the unattended options, just so it doesn't sit there on "retry?" until I next walk by.

The splitting/rejoining, is a lot of work... I should explain that it's not a high-tech reason, but simply to be able to push my Dad's favourite TV episodes to his PC, living in isolation... :wink:
He has Starlink now, so the speed is good, just occasional hiccup. Direct streaming not an option though for many of them.

FTP, I thought of.... and maybe DOpus would help make that easy, it is a FreeNAS box I am pushing to, so I know it has it... might have to look at that.

In any case - auto-retry might be useful for others as well, in a straight up network copy, where resorting to setting up FTP services, is more work than it's worth.
If you're going to put it on the bucket list for DOpus, I realized after posting, that you'd also need a setting somewhere (either in "settings" or in the actual unattended screen) that also says how long to wait before retrying. Maybe the default is 3 seconds, that sort of thing.

Resume would be cool - but yes I can see how intensive that might be, and then you also get into the problem that many browsers face, when they have to label partial files as things like ".partial" until the download completes, and then rename them.

Still, fun to discuss and suppose about! :wink:

Thanks, and would still love to see a retry times/rate option in unattended. :slight_smile:
andrew

#6

Is syncthing.net an option? Works well for me on Windows machines. Haven't tried the FreeNAS plugin, though.

#7

Yeah, I have looked at sync ideas in the past, having used a number of them even going back to the ones that first came with windows server.

But given the occsional and usually manual work for this, well... wasn't worth the working out all the sync stuff, but maybe time to look again?

#8

Just as a follow-up to this thread/idea, I wanted to share two things:

  1. Unattended options, don't seem to help in this case. They only deal with "when file exists"... not network interruptions. Seems to still sit there for hours at "Retry?"
  2. I've now tested/installed about a dozen other copy utilities, from various sources, and honestly of various ages. :wink:
    Not one could tolerate the interruption, or had a resume file option.
    Some had unattended options, but didn't seem to cover anything more than DOpus does.

So - if you're looking ever for something to be "the only one that does this".... well... I have an option! :wink:

thanks all
Andrew

#9

Oh and Sync thing is interesting, but doesn't seem to tolerate existing strucutres well, from what I can see?

that is, there is already 7tb of data in place... so asking it to sync that, well... :wink:

#10

Interesting update.

Took the one machine I was using for this file moving, from Windows 7 to 10 today. (don't ask)

And now... when the network has a hiccup that interrupts the queued files, retry doesn't work any more.
Something about the interaction with Windows 10, is different.
It used to be, it could sit there waiting for me, and as long as the network is back when I hit retry, it clears the old file and carries on, starting the latest file over again.

The only option(s) that work, is to skip or abort. And if I skip, it goes on to the next file, and I have to seek out the missing one.
I suppose it could be somehting else, but that was the only thing that changed today. :slight_smile:

?

#11

Yes, confirmed.
Maybe Leo can comment on this?
Retry not working.... is strange, and unexpected in my books.

#12

It the server/network goes away, Windows can't really cope with that and all file handles involved become invalid. When/whether things start working again is also somewhat random and down to what Windows decides to do, which isn't always predictable.

Windows network shares aren't really designed to be used on unreliable networks. I would use FTP or SFTP or something similar that's designed for it.

#13

Sorry, Leo, I am not explaining correctly.
I have found a bug, that perhaps windows 10 introduced on you, with regards to your "Retry" button.

Skip the file, goes on to the next, and obviously re-establishes a fresh copy instruction, which allows the queue to continue. (I just did it again now)

Retry USED to do that in Windows 7, or, the code that was being used to "Skip to next file" wasn't needed for Retry in Windows 7... but in Windows 10, Retry is NOT trying to re-establish a new copy, but just asking the OS to do it for it maybe?

So maybe the Retry code, needs a little look?
Drop the partial file (incomplete), and try to send it again?
"Reset and send again." is what's needed to retry?

(honestly just trying to explain constructively) :slight_smile:

Andrew