I'm in the process of copying a whole folder structure from one drive to another. It contains a little over 1 million files in 120,000 folders, the files averaging around 460kb, so a total of about 460Gb.
It took 2.25 hours for the first 100,000 files to copy, this being 10% of the file count, 14% of the folder count and about 5% of the total data (so average file size to this point was about half that overall).
So an estimate of the time remaining based on file count would be about 9 * 2.25 = 20.25 hours and based on data would be about 19 * 2.25 = 43 hours. I'd assume that whatever algorithm is used to estimate time remaining takes both file count and total data into account, but either way the estimate should be somewhere between 20 and 43 hours. However the copy dialog is showing around 12 MINUTES remaining.
I know the estimate calculation is complex especially when there are lots of very small files involved as in this case (Leo, we've previously corresponded by email on this - you sent a response on 25/10/2013 2:29 PM), but given the numbers involved here, it's really hard to see how the estimate can be so far out - i.e. it really looks like a bug to me). Could you explain why a more accurate remaining time estimate cannot be provided?
The time estimate is based on how fast the lower progress bar moves. What the lower progress bar represents depends a bit on the operation and configuration, but it's usually either the total size to be copied (if Opus is configured to calculate files before copying) or the number of top-level items selected in lister when the operation started (if Opus is configured not to count files before copying).
With an example like this, with a million files each of which is very small, and the progress bar moving so slowly due to the amount of files and slowness of the drives (2 hours to create 100,000 files and copy 23GB), there will be a lot of inaccuracies, but you're right that it is a long way out in your screenshot.
Exactly why it is that far out, I could only say by trying to re-create what you're seeing (assuming I can re-create it). I'll make a note to do that but I don't know when I'll be able to.
Also, the average rate calculates to about 1.8 MB/s not the 8.71MB/s shown.
There's clearly an issue with my HDD speeds, but the average shown in the dialog still seems wrong.
When either time is 10 hours or greater, it switches to showing HH:MM instead of HH:MM:SS (or MM:SS).
Makes sense to drop the seconds when the hours get huge, but it's obviously confusing when the same xx:xx format is used for both cases. I'll put in a change to make it switch to saying things like 37h 28m when there are 37 hours remaining etc. That won't go into the very next version though, since it will need translation work for other languages.
Ah, that makes much more sense - and also explains the confusion I had with your emailed answer to my previous question about the format the time was shown in.
It doesn't necessarily explain the average transfer rate error though.
Also, after realising how slow the process was through Opus, I did the remaining 80% through Explorer and it completed in 10 hours (less than Opus did 20%). I've seen mention of copy_buffer_size and copy_nonbufferio_threshold in the Slow Copy Function thread ([url]Slow copy function?]) so will look into that.
File copy speed is only calculated while data is actually being read or written. It's a measure of how fast the pair of drives can read and write data, not how quickly they can open/create files. When copying a million tiny files, the speed will be fairly irrelevant as the time it takes to open/create each file will dominate the negligible time it takes to read/write the data.
The speed difference between Explorer and Opus will not be due to buffer sizes in this case. They're almost certainly irrelevant with files so small. It will be because Explorer is not copying file attributes and other metadata which Opus is configured to copy, which adds a significant per-file overhead when copying a million tiny files. If you configure Opus not to copy the extra metadata then it should be as fast as Explorer. See Preferences / File Operations / Copy Attributes.
