I'm on Win7 64bit using Directory Opus 10.0.3.4 x64 (International, 64 bit) on an I7 quad core laptop.
Windows Search is turned off.
I have a few folders (~70MB) to be copied to a different place on the same harddrive. "allow files in this folder to have contents indexed" is unchecked for each of the folders to be copied as well as for the drive itself ("allow files on this drive to have contents indexed").
During the copying (takes 5+ minutes) dopus is completely unresponsive.
Inspecting the attributes of the copied folders I see that "allow files in this folder to have contents indexed" is now checked!
If Opus is unresponsive while copying files, try closing all the windows except for the copy progress window. If that solves it, what kind of files are you copying? There's likely a problem with a shell extension, viewer plugin or video codec and how it handles those files (possibly to do with how it interprets half-written files). The problem may or may not be part of code in Opus itself, depending on the exact details.
If Opus is still responsive when the only window open is the progress dialog, then I would look into whether the problem is caused by your anti-virus scanner or similar tools, since they may block the process as it creates new files, while the files are being scanned. (e.g. The same thing sometimes causes web-browsers to lock-up for a while after download a large file.)
My suspicion is that dopus itself is causing its own unresponsiveness by turning this "allow files in this folder to have contents indexed" attribute on when copying.
what kind of files are you copying
4 folders with about 400 subfolders and about 7000 files. latter are mostly text files.
Is there a way to completely disable this "allow files in this folder to have contents indexed" attrib being turned on via preferences settings?
thx.,
H.
P.S. the primary reason for having windows search turned off and not using the indexing attribute was the slow file operations performance in the first place
Updating the Windows Search index should not cause slow-downs. Isn't it done in the background anyway, not done as soon as the files are written? If there was a general problem with search indexing then we'd all be seeing it.
The index/don't-index attribute is probably inherited from the parent directory, so try clearing it on that before copying files in. But I'd be surprised if that is what's behind the slow-down, with or without Opus doing the copying.
Opus only copies the "standard" file attributes (archive, compressed, encrypted, hidden, readonly and system) so extended attributes like this will probably be inherited from the target folder (although I'm not sure if that's how the system works).
I really doubt if the indexing is behind the slowdown you're experiencing. The indexer runs in the background at a low priority, and from what I've seen doesn't kick in immediately for new files anyway.
If you really don't want to use Windows Search, disable the service and the indexer will be disabled completely.
just installed version 10.04 64bit: noticable change : dopus stays responsive during copying
Let me recap.
I have a source folder with 4 subfolders (containing ~ 400 folders, 7000 files, 70MByte) to be copied to a target folder on the same disk.
Both source and target folders have the "allow files in this folder to have contents indexed" attribute unchecked, as do each of the folders to be copied.
AFTER the lengthy copy process (measured on freshly rebooted otherwise idle computer: 120 seconds), the copies now have the "allow files in this folder to have contents indexed" attribute CHECKED
In Settings -> Preferences -> File Operations -> Copy Attributes: "Preserve the attributes of copied files" is checked!
Obviously, the latter setting is NOT honored!
The same copy process takes only 17 seconds in Windows Explorer.
Repeating the experiment with Settings -> Preferences -> File Operations -> Copy Attributes: "Preserve the attributes of copied files" UNCHECKED:
the copies now have the "allow files in this folder to have contents indexed" attribute UNCHECKED
(measured on otherwise idle computer: 80 seconds)
Good news is that deleting these folders takes the same time on dopus and explorer (8 seconds).
[b]So, obviously two things are not as desired:
dopus does not honor certain preferences settings
file copy performance is dismal (compared to explorer)
How can I address both (and justify the money I spent for dopus)?[/b]
If you're copying 70MB in 7000 files then it's likely that simply setting the attributes on them is what accounts for the time difference.
You are copying lots of files, but a tiny amount of data, so the per-file overhead of setting the attributes is going to be significant and the actual data-copying is going to be insignificant. (Copying 70MB would take about 1 second, so it's not even a factor when we're talking about 80 seconds vs 120 seconds. The per-file operations are what are slowing things down.)
This speed difference has nothing whatsoever to do with the indexing service. It's just how long it takes to set attributes on 7000 files.
Note that other programs do not always set the attributes when you copy files, so if they are quicker than that is often why. Configure Opus not to set the attributes and it should be the same speed.
dopus sets the indexing attribute during copying when preferences "Preserve the attributes of copied files" is checked, even if the attribute is not present in the source to be copied
dopus file copying is significantly slower that explorer's ("Preserve the attributes of copied files" unchecked: 80 vs 17 seconds)
The first item looks like a bug that needs to be addressed.
The second item looks like a performance regression/issue that needs to be addressed.
One obvious solution would be not use dopus for file operations at all.
Pls advise!
We'll look into that. (Note that the "don't index" attribute should not be copied, even if other attributes are; it should always be either cleared or inherited from the parent folder. I think we may be clearing the attribute at the moment, which might be wrong, but we need to take another look and think on it further. It really shouldn't matter anyway as most folders are excluded from indexing -- via the Windows Search settings dialog -- regardless of the file/folder attributes. The attributes are for micro-managing the indexing of individual items within folders that are indexed, which is very unusual.)
Turn off all the optional things in Opus's file copy preferences and compare the speeds again. Also, make sure you are doing the copies several times in each program, and not allowing filesystem buffering to skew your results. (e.g. If you copy the files in Opus, then copy the same files again in Explorer, it's likely that the entire set of files will still be cached in RAM.)
Also make sure that anti-virus tools are not a factor, as they may treat Opus and Explorer differently.
And close any open windows looking at the folders that you are copying into, in case there is a problem with the files being inspected (e.g. to get their icons and other properties) as they are being copied. (If there is, we can look into it, but it will be clouding the issue if we don't identify that it is happening.)
At the end of the day, Opus just opens files, writes data into them and then (optionally) copies attributes, permissions, descriptions, etc. The opening and copying is done using exactly the same APIs that Explorer uses, so any speed difference will almost always come down to other factors (such as those options, or something else reacting to the copied files that causes a slow-down).
About every six months someone says that Opus copies slower than something else, and (if my memory serves me) literally every single time it has been because of one of the above things. Usually, a simple config change so that what Opus isn't being told to copy more metadata than the other program is all that is needed to make both programs copy the same speed. (Sometimes it's not the config at all and just people not accounting for filesystem caching, too.)
(Ignoring weird cases like USB HDDs which for some reason go really slow if the copy is done using certain buffer sizes, but those can be configured as well.)
Did you try closing the Opus file displays after starting the copy (so the only Opus window still open is the progress dialog) to see if that makes any difference?
Using Process Monitor to see which operations Opus is performing vs which Explorer is might reveal something.
What kind of files are you copying?
How are you doing the copy? (Drag & drop? Copy & Paste? Copy Files button?)
Is it possible that the drive you're using is mounted as "removable"? That may cause Opus to wait for each file to be fully-written to disk (not just to the write-back cache in memory, but flushed to the physical disk) before moving to the next file (or closing the progress dialog after the last file), which could explain things.
Seems unlikely, but it's probably going to be something weird like that. Some small difference in the way Opus and Explorer work that is tripping things up somewhere. (Obviously everyone tests their stuff with Explorer, so issues triggered by quirks of Explorer get found & fixed. Quirks of how Opus does things are less likely to, but shouldn't matter and do not matter in most cases.)
Changing the Preferences / Miscellaneous / Advanced: copy_buffer_size might have some effect on large file speeds, but it's usually irrelevant apart from devices with poor drives (e.g. some USB devices are very picky about buffer sizes), and would not affect the case with lots of small files.
Have you checked that your motherboard drivers (specifically the SATA/IDE controller), and HDD/SSD firmware, are all up to date?
Is it possible that the drive you're using is mounted as "removable"?
NO.
Changing the Preferences / Miscellaneous / Advanced: copy_buffer_size might have some effect on large file speeds
upped the setting from 65KB to 32MB - no change
Have you checked that your motherboard drivers (specifically the SATA/IDE controller), and HDD/SSD firmware, are all up to date?
Everything is current.
I did do a few comparative tests with 3rd party file copy tools (fastcopy, teracopy, extremecopy). Each of these tools give results comparable to dopus (for the 2.1GB iso file), and trail significantly behind explorer.
Yet another test: repeated dopus vs explorer comparison with 2.1GB iso file on my boot disk (SSD).
dopus: 25 sec
explorer: 20 sec
Well, two possibilities:
explorer is cheating and hides its progress dialog BEFORE copying is actually complete
explorer is using some trick that speads up file copy operations and Micro$oft missed bragging about it.