Slow network file copy

Help & Support
1

For a long time I was perplexed about file transfers to my NAS being capped around 50 MB/s. My NAS is fast Synology DS1010+ RAID6, my main PC is fast as well with RAID and SSDs.
Seems that the issue is Directory Opus. It is about 50% slower on network copies compared to windows explorer on win7 and win 2008R2.
I verified using DUmeter and Synology DSM 3.2 performance monitor. Explorer consistently posts about 95-100 MB/s transfers while Opus is 50MB/s.
I tried adjusting copy buffer size and also tweaking NIC drivers, but no improvement.

Problem can be fixed by creating an iSCSI target on my NAS. Then speeds go back to what I see with the explorer.

Any ideas? Seems that issue is Opus copy implementation. Is there a way to make a button that copies files using windows explorer (similar to the TeraCopy button)?
BTW TeraCopy is equally slow.

Thanks.

2

If it affects both Opus and TeraCopy, and if the problem can be solved by changing something on the NAS side (creating an iSCSI target), then I'd say the problem is in the NAS.

Any program copying files will simply loop reading and writing data as fast as the disk/network will allow. It sounds like the NAS is not accepting data as quickly as it could under certain circumstances. Without the source-code to Explorer (or the NAS) it's hard to guess what the difference between Opus/TeraCopy and Explorer is that is triggering the speed difference.

Yes, you can do that. I've just posted the info here for easy reference: Copy files via the shell (Windows Explorer).

3

Thanks for the help. I am not convinced though.
iSCSI fundamentally changes the way storage is presented to the Opus. NAS is now a seen as a physical disk, thus speed is similar to that of writing to the disk. So it is more likely that Opus simply handles network file transfers differently and is slower for whatever reason. I also verified that I can get a full speed when copying files from virtual linux to NAS on the same PC. Bottom line is that there is a tool (windows explorer) that copies files fast and that there is another tool (Directory Opus) that copies network files slow. If I was making the slower tool I would look into it because performance difference is so drastic, 50 vs 100 MB/s. I like Opus, have been using it for many years, but this is basically unacceptable.

Thanks.

4

Opus has absolutely no idea how the data is mounted at the other end. All it does is CreateFile, WriteFile (in a loop), CloseFile.

(Unless we are talking about FTP transfers vs network drives.)

5

Thanks for "copy via shell button", it does the job and is decent stopgap measure.

Nevertheless I feel like you are just deflecting and failing to acknowledge the problem. I searched for other posts about the slow network copy and got the same feeling, there was never any acknowledgement by Opus about the problem, just deflection:(

Please look into this, fix it, and stop making up excuses. I went out of my way to write up a case for you guys, at least try to replicate it.

Thanks.

6

Send us your NAS and we'll have a look.

7

Nice one!

8

What do you propose that we do, realistically?

We do not have that model of NAS to test against and the problem seems specific to that model of NAS and its configuration.

Have you complained to Synology that, depending on the mode their NAS is in, the speed you can copy to it with two different programs is drastically reduced? "If I was making the slower NAS I would look into it because performance difference is so drastic, 50 vs 100 MB/s."

Below is some more background here, that was in a separate support thread, copied here for the benefit of anyone else who finds this forum thread:

[quote="I"]There's some background reading on tuning network filecopy performance here, about changes in Vista RTM which were subsequently almost all undone in Vista SP1:

blogs.technet.com/b/markrussinov ... 26167.aspx

So even Microsoft, who make the OS, the network drive protocol and the filesystem and network drivers that implement all of it, don't really know what they are doing here, tried to do something more clever, then ended up reverting back to simple code similar to Opus in almost all cases.

From that article, the only documented differences I can see in how Explorer and Opus copy the data is whether or not the client-side enables caching when copying to/from a network server, and whether or not SMB2 pipelining is used.

The buffering should only affect client-side memory usage, not overall speed (unless so much memory is used that the client starts paging). [And leaving buffering on should allow the OS and filesystem drivers to make more informed choices about how much data to read/write over the network at once, than what Opus could do by guessing, at least in most situations.]

SMB2 pipelining is not widely supported by NAS devices, and is documented as being useful with "high bandwidth, high latency" networks, which is unlikely to apply to a locally attached NAS.

Beyond that, there are all sorts of things that could be going wrong. Some devices are sensitive to buffer sizes, where one buffer size works great with one device and poorly with another. Some devices work great if the file data is written via a particular API or mode and terribly in any other mode. Perhaps Explorer is using slightly different buffer sizes or APIs to Opus and one works well with your NAS (when not in iSCSI mode) while the other works poorly. Unfortunately, without access to the source-code to Explorer or your NAS, we can only guess what the difference is.[/quote]

9

Just to mention I have got 2 Synology's, too, and don't have any probs (whether network nor eSATA, latest DSM, JFrame standard)!

Got a prob couldn't copy some audiofiles to it, but that was gone after reformatting/reinstalling the NAS.

10

Curious, when you see the slow copy is the processor being maxed out?
I in the past seeing DOpus struggle to copy some files. I could see that the reason was due to the PC I was using being pretty low spec and the processor was limiting the speed. This was on an older version of DOpus that seemed at times to be a little more processor intensive when copying. At the time I use used explorer or Teracopy to copy the files, I don't have this issue any more.

11

Just updating the thread for other readers, based on further support conversation: The speed problem isn't happening when copying using Opus to a different network device.

12

Passing on another update for anyone following the thread: