I am willing to try things here... my main issue is that I'm not seeing what I can try that will give real results. Please tell me which tests/logs/etc you will need to diagnose.
I still need to point out that performance in every other app I've tested with show no performance issues (as noted above) with the difference being they use Windows Explorer's copy code to actually copy (dialog box etc). I wonder... is there a way to tell Opus to let Explorer do the actual file copy operation itself?
Anyway let me know what info/tests you'd like me to do. As I said I love the program but the slow copies make it unusable for me. I do wish I had tested this prior to purchase and I'm hoping sincerely we can resolve it
Note: I'll test with the unbuffered IO setting change and report back.
To confirm, we are talking about writing 2 gigaBYTES per second? That makes this test and the device very unusual.
I am assuming the DS3615xs achieves this by writing to all 12 drives at once in parallel, since no single normal HDD or SSD exists that can come close to maintaining that speed. That could also make the DS3615xs very sensitive to how the data is sent to it.
Where is the data coming from that can read that fast? Another DS3615xs, or a RAM disk or similar?
We'd like to set up a similar test, but devices that can even approach those speeds are rare and costly, and we could end up buying them just to find the problem was outside of Opus.
What happens if you use xcopy or robocopy from a command prompt? Those are examples that won't use the shell file copy API (although they won't be identical to Opus, either).
To back up what I'm saying about some Synology devices being sensitive to minor details of how data is written to them, as well as being improved by Synology's software updates, here are a couple of threads:
Synology have a File Server Performance area on their forum. It may be worth posting there to see what they think.
Another thing to check: Is there high CPU load during the copy? From dopus.exe or another process? That could indicate something is causing a bottleneck.
Have you tried extremely large buffer sizes? Things generally default to 64KB but that will be extremely small if you're transferring at 2GB/s. Setting the copy buffer size to 1MB, 10MB or 100MB to test with those is worth a try. (If you're familiar with Process Monitor, you can also use that to see which buffer sizes Explorer is using when it copies to the device. It may have logic to switch to larger sizes for extremely fast devices.)
I would also try those buffer sizes in combination with the non-buffered I/O set to both 1MB and 0. (So 6 tests in all.)
(Non-buffered I/O being turned on often helps, and will also result in the read and write sides being done in parallel by Opus rather than by the OS, but this can cause pathological issues with some devices as well, as some are only tested for buffered I/O. Note that the copy buffer size is independent of the non-buffered I/O setting; they are about different buffers: One in Opus itself, and the other in the OS/filesystem stack. The non-buffered I/O setting is more an on/off than a buffer size: It specifies the minimum file size before non-buffered I/O is used to copy that file, since it will often slow things down with very small files, even on devices which work well with non-buffered I/O.)
Yessir, GigaBYtes. The copies I conducted were From-To the NAS over a 10GB connection using iSCSI. So the data was never (or should not have been) written locally, just back and forth. The 3615xs is definitely capable of some crazy speeds.
I'll run some further tests as you mentioned. The closest I could think of you trying would be read/writes to a super fast M2 drive. I'm going to actually test that today on my main desktop that has 2 Samsung 950 drives and see how Opus compares to Explorer for the same file sizes.
Copy from M2 to M2 was 800-900MBps using Explorer and 400-500MBps using Opus on my desktop machine. Tested both with a fresh file copy (not cached on write) and the same file copied twice to compare cached writes. File size was 6.54GB and free system memory was 12GB of RAM.
BTW not sure if you saw my question on who handles the copy operation. Any chance there's an option to have Explorer actually handle the operation (ie opening the Explorer copy dialog) like the other products I mention here do? Assuming there is some overhead/issue/behavior with Opus that Explorer doesn't have this would potentially be a nice way to use Opus great interface without the slow down. I do know that if I drag and drop FROM OPUS to Explorer then Explorer handles the copy and I get expected performance.
This thread was about moving data ("File Move speed is slow"), which would be an instant rename operation if the source and destination were the same device, so I did not even consider copying from and to the same NAS.
Explorer (via the Shell file-copy API) may be telling the NAS to do the copy itself, without any of the data going over the network at all. What does Task Manager's network bandwidth graph show while the copy is happening in Explorer? I believe there is a very low-level API for doing that with newer versions of SMB (or iSCSI apparently*) if the server supports it.
(*Edit: From doing some research, SMB and iSCSI both have different but similar features which were recently added to do this, circa WIndows 8.)
That is not something Opus or many other programs support, but you can call the shell file copy APIs from Opus in cases where you want do that kind of operation, which would give you the same result as in Explorer or other programs that use those APIs. You'd lose extra functionality when doing that, but most of the extra functionality would not be compatible with a file copy which the NAS was basically performing by itself anyway;
It's bi-directional. Copy from/to combined. Each side is around 1GBps.
I would guess that the "peaks" of 2.5 were probably something wonky with the calculations...but I can only report the numbers. I can say that it's real as I just copied a 44GB file and it took right about 25 seconds.
The copy most likely is not going over the network at all, which probably accounts for all the differences. See my previous post above.
(Copy speed also would not combine from/to speeds together. It would be the maximum of the two, at most. With a normal copy, it's the round-trip speed, always, as that is how fast data is copied from A to B. But in this case, it's probably not a normal copy and the network is probably not relevant to the 2GB/s speed you're seeing.)
I don't know enough to agree/disagree with you Leo, you are probably right, it's possible that it's not using the. That doesn't change the challenge I'm facing however. I just want to get Opus to do these with the same kind of numbers as Explorer.
Unfortunately I don't have a fast M2 drive on the server that's connected to the NAS over 10GB so I can't easily validate if it's doing the transfer completely within the NAS or not. I'm going to see if I can use Aida64 to monitor LAN utilization on the NIC with Opus and Explorer and see if Opus is using the LAN vs Explorer not (ie the iSCSI commands being processed only on the NAS.) Be an interesting test...
Opus copy of ~40GB file from/to the iSCSI drive used the LAN fully ~125-150MBps transfer with LAN at 125000-150000KB/s.
Explorer at 2-2.5GB/s transfer speed but LAN was... about a few KB so NO LAN traffic at all on the 10GB connections. Interesting! Note: There is traffic when copying to another drive...I need to get those numbers as well. But for now this narrows it down for sure.
So it seems the issue is related to how Opus is handling iSCSI based file transfers maybe? At least when to/from the same iSCSI connection?
So, hey people, I recently installed dopus and was faced with the same problem. I have samsung qvo 860 and the copy/move speed was around 50MB/s, I did 2 things, I enabled Rapid mode through samsung magician and gave the program full admin rights by default. this fixed it and now dopus is faster than FExplorer and the speed hangs around 220MB/s. I hope this helps some people who face the problem. Though I do nothing crazy and I'm a complete normal user.
I tested by moving multiple files with the total size being 50GB.
Try this program that someone suggested to me. I said sure and I tried it and Directory Opus started go blazing fast with respect to network stuff. It's called MSI Mode Utility Version 2. After installing it and trying it, I was like What the F....
Maybe it doesn't work for everyone and my computer was just so misaligned or something. Evidently, I know nothing about computers.
I do know that previously, when I would press the network button on the side, it would take minutes for the process to end. Now it takes seconds. And, when I access my NAS, it opens up everything right away. It's not a virus or anything like that. That was my first suspicion but evidently a lot of gamers use it to speed up something or another.
I don't have the link. You'll have to find a safe copy yourself. Actually, I might have a copy which I'll see if I can attach to this post.
I didn't do the following but the guy had said it works better if you disable fast boot and hibernation. I think I did this (I can't remember really), I don't think it made much difference. Or maybe he was saying that you need to turn it off to work. I don't know because I AM AN IDIOT. Anyway, try it out. It seems that everything on my computer goes faster. What I am wondering was why didn't people tell others about this small program years before? Maybe they did but I never knew because I am stupid.
Here is the link to the Github page for the tool. Link.
And, here is a white paper from Intel that explains what it does. Link
You'll see that everything on your computer just opens cleaner and snappier. It's pretty cool. Wondering why Windows doesn't license from Intel and incorporate it in their system.
A Reddit post is still not a great source for a utility like that. The home of the project is here. I'm happy with my computer, so I'm not going to install it. The post I linked goes into how to do this stuff manually if you wish.
