File Move speed is slow

Hi all,

Since upgrading to DO12 my file move speed has drastically reduced, it's slower than it was with DO11 and WAY slower than Windows Explorer which is a shock !

I regularly copy large (1.5Gb - 12Gb) .mkv video files from my download drive (Samsung 850Pro) to my storage server (Server 2012R2 - 6 disk storage pool - REFs) over a Gigabit LAN.

Today is actually the first time since the upgrade that I needed to move some stuff over to my server, and thought to myself as I was doing it "hang on this doesn't normally take this long" so started to note down times and speeds.

For 1.2Gb file size with DO12 I am getting an average of 33.2 MB/s taking 38 seconds
Same file size with Explorer I am getting 108 MB/s taking 11 seconds.

What is going on ! ?

I really like the file copy API of DO, being able to queue multiple move operations and such like, but at this loss of speed this is going to be very painful when I start to do things like moving VM's around :angry:

Any suggestions are greatly appreciated.

For information - I have only very recently installed DO11 (after trialing it both on Win7 and then Win10) and have not "fiddled" with it very much other than changing the colour of the file and tree windows.
I'm running Windows 10 64bit on an extremely high spec PC.

Many thanks in advance

Damian

The way Opus reads/writes files when copying and moving has not changed at all from Opus 11 to Opus 12. (We don't touch that code unless we really need to, since it is both critical it doesn't go wrong, and mature enough that we almost never have any reason to change it.)

The two settings within Opus which can affect copy speed are:

Preferences / Miscellaneous / Advanced: copy_buffer_size
Preferences / Miscellaneous / Advanced: copy_nonbufferio_threshold

The copy_nonbufferio_threshold setting's default did change some months ago, to make things more compatible with some non-standard filesystems, and that could slow things down a bit in some cases. You could try setting that to "1 MB" to get back to how the old default was. But you would not see that as a change from going from Opus 11 to Opus 12 unless you moved from a very old version of Opus 11.

Aside from those two settings, most of what affects copy speed in Opus is external: The physical hardware, the drivers, network, anti-virus, firewall, etc. Some of those things can be very sensitive to buffer sizes. Some of them (anti-virus/firewall software) auto-update and can become pathologically slow with certain operations. The same things can also treat different executable files differently, or treat different data being copied differently. On our side, nothing has really changed in this area so any sudden slowdown is probably an external factor, assuming you went from a recent Opus 11 to Opus 12.

Hi Leo, thanks for the reply.

I have tried your suggestions regarding buffer size and io_threshold and nothing makes any difference, it seems that any file transfer over my LAN is capped at 33 MB/s +/- .5 (yes I did exit out of DO completely before retrying and also tried a full reboot.

But File Explorer \ Syncovery and Robocopy all copy to the same location and same file at ~110 MB/s

I have turned my AV off (Kasperski) I don't run any software firewalls at all, I only use a hardware firewall on the external facing router.

I'm running on a relatively fresh (clean) install of Win10, all drivers are up-to-date, and as mentioned I have no issues with file moves outside of DO.

I have created a temp share on the C:\ drive of my server which is an SSD with standard NTFS formatting, and am getting exactly the same speed, so it not an issue with REfs.

I'm at a loss as what is going on here !!
I should mention that I work in ICT, at the thick end of things - as a secondline analyst, I know that doesn't mean that I know anywhere near what you do regarding this short of thing, but I'm not a numpty either !! :slight_smile:

Is there anything else you can think of that I could try - before I wipe out DO12 and go back to 11 and try that again ??

Many thanks

Damian

How are you comparing the network copy speeds?

The most accurate way is to open Task Manager or Resource Monitor and display a graph of network activity. Assuming nothing else is using the network, start the copy, then time how long it takes before the network activity returns to normal. Do it a few times using the same files in both programs, to account for caching etc.

The speed and time numbers which each program displays may not be directly comparable as they may measure speed differently, and may show the time taken to buffer the data rather than flush it to disk (or at least through the network). In some cases a copy progress dialog may appear complete and close when a lot more is still to be done.

Network utilisation graphs can also be tricky since you may see a lower but steadier throughput in one program and interwoven spikes/troughs in another, depending on how things are being buffered. But the clock time from when the network activity starts to when it stops is a good indication of speed.

With the buffer size settings, which sizes did you try? It's possible Opus is using a buffer size which triggers slowness in some part of the network, and it's also possible that the network drivers changed recently with the Windows 10 Anniversary Update (which seems to have brought with it a host of new drivers and a bunch of issues; my popular-model SSD kept disconnecting from the bus for 5 seconds at a time, for example). That's probably more likely than the change from Opus 11 to Opus 12 affecting copy speeds, if a-v/firewall treating the new exe differently has already been ruled out.

Hi Leo,

My timing of the copy event are both accurate and consistent - like I said I'm a systems analyst and have been doing this for many many years, there is no user perception issue here.

Things I have tried today:

removed DO12, total registry clean out (not that much left after uninstall)
Re-Ran moves with Explorer - avg 110 MB/s
Installed DO11 = exactly the same speeds as before ~33-34 MB/s
removed DO11 and installed DO12 - seeming as there is no difference between them.
Tested again - same result.

Played with copy_nonbufferio_threshold in conjunction with copy_buffer_size.
copy_nonbufferio_threshold :- seemed to make no difference at all - tried small values and high values up to 2GB
copy_buffer_size :- tried from 128KB right up to 10MB - 256kb to about 4MB made no difference at all, but from 7MB to 9MB there is a marked improvement (at 9MB i'm getting 57MB/s) over 9MB things start slowing down again.

This made me think about what Jumbo Frames might do, so I enabled JF (On all network devices involved) and played with the values there as well.
There was a slight improvement, but the improvement extended to Explorer as well, so the speed differential between DO and explorer was the same.

Disabled Jumbo Frames as JF only really helps with large file transfers and is know to cause more issues with other types of transmissions than it's worth.

So I've made a slight improvement (DO is now just under half the speed of Explorer !) but this is a long way off what it should be - and was under Windows 7.

I did find that the NIC on my server had an updated driver available, so installed that (but this is server 2012R2 and the drivers are very stable) incidentally the NIC on the server is the same as on my PC.

The only thing that I haven't yet tried, is to install DO12 on my HTCP - W7 and test its speed transferring the same files to and from my server.

Beyond that I'm running out of realistic tests and ideas.
I understand what you're saying that it's likely affected by something outside of DO - BUT it's only DO that's having this problem.
I can stream Blue-ray quality movies from my server across my network to a number of devices and transfer\synchronise documents and vhd's with syncovery and explorer across the LAN at pretty much near as damn it max network speed, just not with Directory Opus !!

Thanks

Damian

Looks like you have tried most of the things I could think of.

At the start you said it seemed to coincide with moving from Opus 11 to Opus 12, but now having tried Opus 11 again it's the same speed. So that at least rules out any change on our side, but it seems we are mostly ruling things out, not in. :slight_smile:

Have you checked in event log to see if there are any warnings in there? It's a long shot, but it helped track down one problem I had with the Windows 10 Anniversary Update randomly freezing for a few seconds at a time (turned out to be due to a new AHCI driver which did power-management things that my SSD really did not like, causing it to keep resetting its connection).

One unexpected thing which sometimes has an effect is the progress dialog itself. (Usually only when there's a problem with other drivers in the system to do with graphics of the PCIe bus.) Minimising the progress dialog might be worth a quick try, just to rule that out.

What's the CPU usage like while the copy is happening? If it's more than a couple of %, Process Explorer's Threads tab might reveal something. (Right-click the process, then choose Properties, Threads, select the thread using most CPU and click Stack. It'll tell you which DLLs are involved in the current thread, which can sometimes reveal low-level components doing odd things.)

And while it's ongoing, if you start another copy in parallel to the same server, do the two affect each other's speeds? That kind of experiment might help to check if the bottleneck is somewhere in the network or local to one machine or the other.

More testing done - more confused !!

CPU utilisation is at about 1.2% for normal copying and up to 1.4/7 for parallel copy - so not an issue I would of said.
The interesting this is that when I do a parallel copy, both copies were getting 39 to 40 MB/s each, so above what the single copy was achieving.
On a single copy with DO I'm only getting 432Mbps network throughput - and with parallel it was 720Mbps - 64% utilisation.
With Explorer doing the same tests I'm hitting 912Mbps on both single and parallel copying at about 97% utilisation.

Copying back from my server to PC with DO, I'm getting 908Mbps

So it really does look like something is preventing DO from fully using the available network bandwidth.
I will run more test ... got a headache and need to eat now tho :slight_smile:

Cheers

Damian

I am having a similar (I think) issue. After upgrading to Opus12 file copy speed off my desktop hard drive via a USB 3.0 connection to a portable external hard drive of x2 large files, @ on of @ 3Gig, the other @7 Gig has slowed to a CRAWL. From 40+ Mb p/s speeds to less than 500K. What does one do? This is making my transport and backup of the files unfeasible. Copying is going from a couple of minutes to estimated hours.

Nothing about how Opus 12 copies files has changed at all from Opus 11, except the addition of the speed graph in the progress dialog. (Which could be connected to problems if there is a driver issue, either with the motherboard (chipset/storage/bus) or the GPU. This has happened before, although not for some years.)

Other external factors include antivirus (is something scanning the files and slowing things down?) and recent Windows updates (the Anniversary Update changed a lot of drivers, in many documented cases swapping working drivers for terrible ones which cause problems until corrected).

Inside of Opus, if you have columns or other features enabled which cause the contents of files to be read, and that is happening while the copy is going on, then that can slow down some devices which don't cope properly with parallel requests. Similarly, try closing the folder tree or even all listers and just leave the progress dialog open on its own, to see if the slowdown is being triggered by something looking at the folder. (This could also be something outside of Opus, of course.)

The questions in my post above are also worth looking at (CPU usage etc.)

Hi, Leo. I installed 12.2 on a Windows 2008 R2 server today in conjunction with some Windows Updates (same reboot). For more than a 3/4 of an hour file transfers in Dopus between drives was abnormally slow (1.45 mb/s). File transfers between file explorer windows were normal. File transfers from an explorer window to a dopus pane was very slow. This was also done with > 1GB files.

However, now 1 hr after boot, file transfers are okay again in Dopus. Weird.

On a second machine, file transfers with Dopus 12.2 are okay (windows 10 au x64).

I'm not saying it was Dopus but today was abnormal and I could not get anything done. We will see tomorrow on the next boot. Hopefully not Dopus. Best wishes.

Sorry for revisiting an old thread but I found this while searching for an answer to my problem... which is MUCH slower comparative transfer speeds in Directory Opus than Explorer. I bought 2 licenses specifically for copying large files/folders and managing media.

I'm running 12.5 64bit on a Windows Server 2012 Essentials machine which is my primary home server. My data store is a Synology DS3615xs NAS box connected to the Windows server by a dedicated 10GBe connection using iSCSI.

If I copy a file from the iSCSI drive to itself (ie from and back to the NAS) using Explorer I get anywhere from 1.5 to 2.5GBps. Yes that fast. :slight_smile:

If I use Directory Opus for the copies I am lucky to get 100-200MBps. 1/10th the performance of explorer.

I've played around with the buffer sizes with no real significant improvement.

I am afraid that DOPUS is simply not coded to copy quickly...I hope that this not the case... but I paid 90ish bucks for what I thought would be a high performance file management replacement for Explorer. Is this the best I can expect or is there s something I am doing wrong that can fix this issue? No offense but I simply would not expect OPUS to be that much slower than explorer.

I should note that if I copy a file FROM OPUS to Explorer it opens the Explorer dialog and the copies are super fast as well. Only if the copy occurs within OPUS is it super slow.

This seems to be the behavior others are seeing as well.

Opus copies the data as fast as the OS, hardware, drivers, antivirus, etc. allow it to.

If you think there are speed differences, please show the test and data, and tell us exactly which settings you have tried.

Speed differences usually turn out to be because one program was configured to do different things to another. (e.g. Explorer copies let metadata than Opus by default, so if you are copying/moving a lot of small files, there may be a big speed difference until you configure things differently.)

They can also come from antivirus treating different programs differently.

They also often come from filesystem caching, if you are running tests back to back and one program ends up reading data that is already cached in memory instead of reading it from disk.

They can come from buffer size and non-buffered I/O differences, too, which are largely dependent on hardware/driver/network quirks and need experimentation to work out the best sizes on some systems.

There are also major differences in how the progress dialogs in different software works. Opus won't say something is complete until it is flushed to the destination. Other things say the job is complete a long time before it actually is, and if you look in Task Manager you can see the data is actually still being copied.

It's a complex subject. Usually the speed is the same in every program you try. When it isn't, or doesn't appear to be, there are a lot of things that need to be looked into to understand why.

Leo,

I'm talking 40gb files here... there no chance of caching causing the HUGE discrepancy. It's not hardware, driver or network. Xplorer2 does not have this issue.

I already explained what my tests were, very large files being copied from one folder to another on the same iSCSI drive connection.

Considering that many others seem to be seeing this issue with OPUS I have to assume it's OPUS and certainly not my system. To be frank I'm not interested in messing around with settings (unless you can point ones out that can improve performance 10 fold) to figure this out. It should perform roughly equivalent to Explorer and other file managers (as I mentioned I tested xplorer2 to validate) out of the box imo. That it is consistently slow only in OPUS and not elsewhere pretty much tells me it's not my system.

Don't mean to be argumentative here but I'm a software engineer myself and I understand how software "should" work and if this is a "settings" issue then it should have been identified a long time ago and the information on correcting it should be available. If it was environmental then that also should be well known by now as it appears others have had the same issue (those that have reported back and/or tested.)

As for every program should be the same... that is completely untrue. You are using your own code for the main manager interface and for the copy process (you aren't using the Windows Explorer APIs to copy the files for example) and so it's your copy code that is the problem. I've tested both Xyplorer and Xplorer2 and neither of those products has a problem.

I really want this to be a resolvable problem. I love Directory Opus overall but this is a deal breaker. Any ideas (other than setting the copy buffer size as I've tried 1mb to 9mb and no joy) would be appreciated. If I can't get Explorer copy speeds then Opus won't work for me.

Thanks.

Agreed, assuming normal amounts of RAM/caching. That detail was not in the original post so the possibility of lots of small files was still there.

Did you test speeds with your setup before buying and find they were OK then, but then ran into issues when trying other thing after buying?

I am wondering if speeds are fast in some tests/situations and slow in others, which may point to a cause and possible solutions.

I understand that, but without seeing the data or being able to reproduce the setup, I have nothing to go on. I can tell you that when I copy large files, speed is identical to everything else. Which of is is "right?" Probably both of us, and the differences come down to the details of what is being done, which are important.

Sometimes differences also come down to how the data is being interpreted and differences in how different programs represent progress, and so on. Possibly not in this case, but I can't rule it out without knowing the details.

Have you checked copy speeds with other types of device? How do things perform when copying between two local SATA drives? If performance is good there then it means we should focus on the type of device involved, and that we may need to get hold of similar hardware to test against ourselves. On the other hand, if the same performance issue exists with a vanilla local SATA->SATA copy, then we can rule out anything to do with the type of device and look in other directions.

The vast, vast majority of people find copy speed is identical between programs when doing proper tests (avoiding caching, and per-file overheads that are different in each program, and so on). This should generally be the case because Opus is literally just sitting in a loop waiting for the OS to copy the data, and not doing the low-level work of reading/writing each bit itself.

There will always be edge cases where one program is faster than another for particular scenarios, but we aren't aware of any major ones. You may have found one, in which case we'd like to know more about it to help us reproduce what you're seeing and look into where the difference is coming from.

Many other file managers just ask the Windows shell to copy files for them, in which case (unless antivirus is doing different things to different programs) performance is identical to Explorer as it's literally the same code being used. (Which also has the same limitations, as the downside. You can call the shell file copy APIs in Opus if you wish, but it blocks off a lot of functionality. Still, it can be useful when doing basic file copies to devices that are only fast when using the shell APIs.)

As an example: Companies who make NAS devices do a lot of testing and optimising with Explorer, since it's what most people will use and judge them by, but sometimes leave giant performance issues (or worse) in their devices when other programs do things even slightly differently. I'm not saying that is the case in this situation, as I do not know and cannot test it, but it has been found to be the case in others.

e.g. People checked how fast a drive performed when Photoshop loaded and saved data vs when Explorer copied files, and they saw the drive was very slow for everything except Explorer (and other things using Explorer's file-copy API). Sometimes those issues are resolved by updating the device's firmware, but curiously they sometimes come back in new devices, indicating that some brands do not carry forward all of their fixes from one device to another.

If you don't want to try things then there is not much we can do. I can suggest settings that may help (e.g. try turning on the non-buffered I/O option in Pefs/Misc/Advanced, setting it to 1MB), but copy speed issues can be caused by a large array of issues, and we have no way of knowing which one without working through the different possibilities and trying various things.

I am willing to try things here... my main issue is that I'm not seeing what I can try that will give real results. Please tell me which tests/logs/etc you will need to diagnose.

I still need to point out that performance in every other app I've tested with show no performance issues (as noted above) with the difference being they use Windows Explorer's copy code to actually copy (dialog box etc). I wonder... is there a way to tell Opus to let Explorer do the actual file copy operation itself?

Anyway let me know what info/tests you'd like me to do. As I said I love the program but the slow copies make it unusable for me. I do wish I had tested this prior to purchase and I'm hoping sincerely we can resolve it :wink:

Note: I'll test with the unbuffered IO setting change and report back.

No change in performance with copy_nonbufferio_threshold set to 1MB. Also tested copy_buffer_size at 1024 and at 8192. No meaningful change.

42gb file copy test:
Opus - 160-200MBps.
Explorer/Xyplorer/Xplorer2 - 1.6-2.2GBps (1600-2200 MBps)

Any other tests/settings?

To confirm, we are talking about writing 2 gigaBYTES per second? That makes this test and the device very unusual.

I am assuming the DS3615xs achieves this by writing to all 12 drives at once in parallel, since no single normal HDD or SSD exists that can come close to maintaining that speed. That could also make the DS3615xs very sensitive to how the data is sent to it.

Where is the data coming from that can read that fast? Another DS3615xs, or a RAM disk or similar?

We'd like to set up a similar test, but devices that can even approach those speeds are rare and costly, and we could end up buying them just to find the problem was outside of Opus.

What happens if you use xcopy or robocopy from a command prompt? Those are examples that won't use the shell file copy API (although they won't be identical to Opus, either).

To back up what I'm saying about some Synology devices being sensitive to minor details of how data is written to them, as well as being improved by Synology's software updates, here are a couple of threads:

Synology have a File Server Performance area on their forum. It may be worth posting there to see what they think.

Another thing to check: Is there high CPU load during the copy? From dopus.exe or another process? That could indicate something is causing a bottleneck.

Have you tried extremely large buffer sizes? Things generally default to 64KB but that will be extremely small if you're transferring at 2GB/s. Setting the copy buffer size to 1MB, 10MB or 100MB to test with those is worth a try. (If you're familiar with Process Monitor, you can also use that to see which buffer sizes Explorer is using when it copies to the device. It may have logic to switch to larger sizes for extremely fast devices.)

I would also try those buffer sizes in combination with the non-buffered I/O set to both 1MB and 0. (So 6 tests in all.)

(Non-buffered I/O being turned on often helps, and will also result in the read and write sides being done in parallel by Opus rather than by the OS, but this can cause pathological issues with some devices as well, as some are only tested for buffered I/O. Note that the copy buffer size is independent of the non-buffered I/O setting; they are about different buffers: One in Opus itself, and the other in the OS/filesystem stack. The non-buffered I/O setting is more an on/off than a buffer size: It specifies the minimum file size before non-buffered I/O is used to copy that file, since it will often slow things down with very small files, even on devices which work well with non-buffered I/O.)

Yessir, GigaBYtes. The copies I conducted were From-To the NAS over a 10GB connection using iSCSI. So the data was never (or should not have been) written locally, just back and forth. The 3615xs is definitely capable of some crazy speeds.

I'll run some further tests as you mentioned. The closest I could think of you trying would be read/writes to a super fast M2 drive. I'm going to actually test that today on my main desktop that has 2 Samsung 950 drives and see how Opus compares to Explorer for the same file sizes.

Will report back...

Copy from M2 to M2 was 800-900MBps using Explorer and 400-500MBps using Opus on my desktop machine. Tested both with a fresh file copy (not cached on write) and the same file copied twice to compare cached writes. File size was 6.54GB and free system memory was 12GB of RAM.

BTW not sure if you saw my question on who handles the copy operation. Any chance there's an option to have Explorer actually handle the operation (ie opening the Explorer copy dialog) like the other products I mention here do? Assuming there is some overhead/issue/behavior with Opus that Explorer doesn't have this would potentially be a nice way to use Opus great interface without the slow down. I do know that if I drag and drop FROM OPUS to Explorer then Explorer handles the copy and I get expected performance.

Thanks

How can you get 2.2 gigabytes / sec over 10 Gb LAN?