@ Leo
All drivers are absolutely the latest ones.
I was reading data from an external eSATA RAID-5 and writing over network to an external 4lane SAS RAID-5 (connected to a post pro workstation).
I will try playing with the copy buffer size in Opus.
@ tbone
I found this setting to be the number 1 reason for slow GB networks during my researches all over the web.
Unfortunately I cannot change the NIC on my laptop workstation.
Just wanted to note that changing the copy buffers of Opus did not help. It is getting quite annoying that I am not able to copy files over LAN via Opus.
Any other suggestions what to try? Is there anything I can log so the Opus developers can trace/reproduce the problem?
I can't think of anything that could be logged, unless you're seeing very high CPU usage during the copy (which could indicate something is going wrong, and could be traced using Process Explorer; shout if that is happening).
Since the network connection is actually dropping sometimes it seems like an issue with the network hardware and/or drivers (or firewall/av) becoming overloaded/confused to me, since Opus would not be disconnecting the network.
No - CPU load is normal.
As I wrote above - copying through Windows Explorer works without any network disconnections. So I would assume there is something in the way how Opus does the file transfers which results in a network drop.
I don't doubt that Opus is doing something different which is triggering the problem, but I can't see how it's doing anything but triggering a problem in something else, since the network connection itself is dropping when all Opus is doing is reading and writing to files, and since this isn't a problem everyone else is seeing.
Thanks guys for the fix,
My network is 100% GigE but DO was only averaging 30-40MBs on 25GB file transfers to and from a large raid 60 2008 server. Windows 7 Ent could hit and sustain 130MBs. After discovering this thread (thank you!) I bumped the copy buffer up to 16348KB and my transfers jumped up to an 112MBs average on DO, but increasing DO's buffer beyond 1634KB doesn't get anymore throughput.
The server is using an Intel quad NIC server card teamed for 4.0GB, the workstation is using an Intel dual NIC server card teamed for 2.0GB. Using a LAN speed test I've recorded transfers that can hit around 150MBs at peak w/ 130MBs on average.
Only issue I see now is that about 2/3 of the way through the throughput starts coasting down to between 70 & 80MBs, and I don't why. The server isn't doing anything that I can tell, but the slowdown is always about the 2/3 mark.
Has anyone else seen this?
What happens when deactivating Jumbo Frames?
@BadKarma
I wish I had your problems.. o) 30-40MB/s is what I want to achieve.. i'm still stuck at about 5-10MB/s.
Ok, this is no teamed GBit-Ethernet on my side, but.. o)
Did you test your transfer speeds with Explorer or another filemanager as well ?!That might give you
some different results or further hints where to look. I checked the web for any filecopy-caching
mechanisms one could alter in Windows-Server-OSs, but it looks like are no options for that.
Regarding my tranfers-speed-topic:
In the meantime I fully patched my Problem-System to the latest Microsoft-Patchlevels, but that did
not help much. But at least I seem to have figured out that strange caching behaviour, it seems it is
related only to truecrypt volumes. I'll continue my research.. thanks so far.
ps: Further tests with that CopyBuffer-Size let me go back to 32KB.
Sasa, disabling jumbo frames drops the speed back down under 100MBs. Starts at just under 100MBs and steadily falls to around 55-80MBs.
TBone, Yes , Explorer can sustain anywhere from 125-130MBS all the way through the transfer (with small slowdowns that last only 10-15 seconds) . I'm not sure why but DO just starts slowing down after 2/3 -1/2 way into the transfer. It's still an ok speed but it just makes me wonder what's happening.
I’ve experienced similar issues: I edit HD video via HD video network server. Just using the fastest network components didn’t give me the required speed at first. I found that different brands and types of network cards and routers played a big part. (Software seems to vary in communication methods via these network devices) I just had to research, test and find what was the most compatible with the software I use. Some of the software works better with different network cards of the same speed. To address this, I had to install multiple cards on the same systems and set the software to use the card it’s most efficient with. Because of these issues, systems like Avid and Autodesk Flame Premium use propriety dedicated lan built to their system to maximize the performance, thus reducing driver, hardware and software compatibility issues.
My troubleshooting checklist:
Driver updates
Use a different computer with the same (i.e. opus software on the same network to test)
Check the cables and connections (bad cables can reduce the functions network cards try to use. Some software like Opus might try to use some of the more advanced features crippled by the bad cabling)
Remove router or switch from chain to test
Swap network cards around if possible
Change network card types brands, etc. (laptops use an external to test)
Just switch things around and using a simple process of elimination always works great for me.
One surprising find, was another device on the network can affect the performance of another or all devices, even though it’s not being actively used to move files, etc. Just the way the router communicates collaboratively with all network devices can play a big part; the managed routers a most prone to this. One port with identical settings can work better than the other with various software; very strange!
This might not be the cause, but maybe it can help. Cheers
@ tbone
Did you try my hint with disabling "IPv4 (TCP / UDP) checksum offload" (or in German "IPv4 Prüfsummen abladen")?
Before I did that my network was stuck to <10 MB/s.
@ all
Which security software are you using on your computers (AV & firewall)?
I am using MS security essentials and just saw that it has realtime network inspection - so I will do some tests if that might interfere with Opus.
Am I just blind or is there no possibility to edit my own postings?
Anyway, @ tbone again: You might consider buying an Intel NIC - the Intel Gigabit CT is quite cheap but very fast (I am using that on 2 of my computers)
-> geizhals.at/deutschland/351749
@tbone
I don't use per workstation security software. I choose to hardware firewall the WAN with a Fortinet FortiGate 60C (Integrated Threat Management for Front Line Locations). Catches everything! I've never had a virus on my network since for 4 years now. No slowdowns on workstations via bloated AV software. Before this though, I used Panda Security; very low overhead and never had virus’s escape quarantine for two years on 12 research workstations. Other low overhead AV's are: VIPRE and Online Armor ++. I have also used Paket Fence and Network Security Task Manager; for real-time monitoring and reporting across the network. Through testing: Norton, AVG and AdwarePro, interfered the most with performance issues for all 30 AV's we tested; Panda Security and Vipre performed the best across the board for network and single workstation setups.
@freezer
I did that now (again?), i cannot really recall if I did this change to the NICs anytime before,
I also checked for the latest drivers for the NICs and put some blank unfragmented hardrives
into the involved systems. I'm now at nearly 60MB/s. I guess your hint was the most efficient
change I did so far, although I think I already played around in this corner, but nonetheless,
thank you for stressing this setting again!.. o) Sidenote: I disabled ANY checksum offloads,
for UDP as well as TCP and enabled those JumboFrames @8K again.
BUT!..
DOpus is still behaving weird. I can copy the same file over and over again back and forth with
Explorer.exe and always get these 60/MBs, sometimes even more when this strange caching,
(or what is is) kicks in, then I can even get around 100MB/s shortly. This sudden speed up or
boostet filetransfer is NOT related to truecrypt volumes. I was wrong with that, this behavior
occurs even with freshly formatted partitions.
DO on the other hand is not capable of reliably copying with 60MB/s. It would do this one time,
and the next two attempt it falls back to 20MB/s. If I stop and restart the transfer with Explorer.exe
it is 3 times faster @ those 60MB/s. CopyBuffer is at 8MB right now. I noticed that it might have
something to do with HOW I copy the file. Drag&Drop vs. Copy&Paste or the CopyButton. As
I don't think that there is an internal difference on how DO copies files.. I must forget about this, it
might just have something to do with the abnormal situation, where a filecopy is fast, and the
next is slow.
So whatever I change.. settings, harddrives, patchlevels.. it is still DO being slower than Explorer
in most of the situations.
@kgibbs
I am not a fan of security-software as well, with the latest MS-Update, that Microsoft-Security-
Essentials got onto my Windows7-machine.. I let it be right now, as I did not find any slowdowns,
but I will pay further attention.. o)
To sum things up:
I still think there is something "in" DO, that keeps us from getting a steady and reliably fast file copy.
It cannot be the CopyBuffer-Setting anymore, because the same setting is fast.. and then slow for the
next try.
Are you sure nothing else is going on with the folder (or network) while this is happening?
For example, if Opus is configured to show media duration in the status bar (i.e. the total playtime of all the files in the folder), or set to show thumbnails, or anything like that, then it will be working through all the files in the folder(s) you are viewing, using the network (and disk etc.) in the background.
That is one possible explanation for the speed varying when you do the same copy each time in the same program. There are many others, of course.
Looking at the various things the Resource Monitor tool (comes with Win7) shows being used may reveal where the bottleneck is, too. (e.g. The disk, network, memory, etc. may be hitting 100% usage when the slowdown occurs, which could point towards what to investigate. The tool gives a detailed breakdown of e.g. which process and file is responsible for the disk access. Of course, it may be worth looking at this on both the client and the server machine.)
Nirsoft: Processmon, FileMon and Diskmon could help analyze deeper, if you know how to use them and want to spend the time required. By comparing Explorer and Dopus differences (i.e. eliminating common denominators via filters), you can start pinpointing how each differ in there execution. I’ve had my entire network fine tuned on every workstation, server and router to maximize bandwidth via viber-optic 2gig workstation and 10gig server cards, for uncompressed, quad-band/layered, real-time HD video editing, via up to 4 paralleled HD workstations. What I’ve discovered in the process, is the faster your equipments bandwidth potential is, the more finely tuned and compatible all components need to be, throughout. Most hardware, software and drivers weren’t designed with super high bandwidth, quad NIC cards in mind. Implementing these newer types of specialized components quickly starts revealing all kinds of bottlenecks and shortcomings throughout the entire network chain. Just because you buy a high bandwidth network card, doesn’t guarantee, today, that your system setup will be able to utilize it efficiently. You may get lucky or not. Some of my workstations performed near full potential, where as others, with almost identical components, did very poorly without modifications and tweaking. Be prepared, the closer you want to get to consistently achieving maximum bandwidth potential, the more time you will spend to achieve it. Another point, the cheaper high-bandwidth cards are more of a hit or miss to their compatibility with your system. Their drivers tend to be minimally written to adjust and conform to the wider spectrum of system architectures (i.e. get what paid for, lol). As I found out, owning this type of technology is a real curveball to your initial expectations, but well worth it in the long.
I have to agree with the other posters, file transfers are much slower than Opus v9.x. In fact I've un-installed v10 and gone back to v9.x because of a number of problems that appear to be attributable to v10.
When v10 is transferring a lot of files over the network, the rest of Opus's UI is very unresponsive for one thing. Another, I've just tried using the latest v10 beta 10.0.1.4 i think it was, had to uninstall this because of peculiar mouse click behaviour that wasn't occurring before installation.
I don't think anybody has mentioned Opus 9 until now in the thread. People are talking about Explorer vs Opus, as far as I can tell.
That's also strange since the way Opus copies files didn't change between Opus 9 and Opus 10. A lot of other things have changed, but not that.
Maybe one of those 9->10 changes has triggered a problem on your setup. (e.g. Some graphics/motherboard drivers are really sensitive to the way progress dialogs are updated, in my experience. It can be quite random things like that. Or it can be more obvious things like antivirus or firewall software that works using signatures and sees two versions of Opus (or whatever) as different, and treats them differently.)
If that is due to high CPU usage, Process Explorer may be able to indicate what is using the CPU.
(See W0lfdale's suggestion in an old thread for an alternative description of what to do, with screenshot. That's linked from the general troubleshooting FAQ which may have other things worth trying.)
Have you reported that problem anywhere? It can't be explained/fixed unless it's reported, and it doesn't sound related to this thread.
Thanks for the reply Leo. I'll set up a few experiments and see if I can figure out why there's a difference. Apologies for posting about the mouse behaviour in the wrong thread. Before I report the 'bug' about mouse behaviour I need to check to make sure nothing else caused it first. But having un-installed v10, my mouse is now behaving itself again.
Just want to share my latest experiences with file copies over networks:
I found out that one single feature of my NIC is affecting the stability of file transfers:
RSS Receive Side Scaling
If this option is set to OFF - I have no problems with losing the connection anymore!
Furthermore: A few days ago Microsoft released a security update for networking and since then I have file transfers near 100 MB/s via Dopus.
Maybe that infos help someone else.