Can DO have a verification when finishing copying?

Can DO have a check when finishing copying? Because HardDisk sometimes may produce bugs which can cause the copied files to be not right and even broken. But a check could help find out the issue for the first time. Maybe not everyone need the feature, so you can try just introducing an option in DO's copy settings.

Hope to get improved. It would be perfect that DO can have a verification like the fastcopy before copying finished.

Meaningful verification is actually difficult, since both the operating system and physical hardware cache written data, and re-reading it may just re-read the cache.

We could easily add something which makes you feel better but turns out not to mean anything and just wastes your time, but we would rather not add that kind of thing. :slight_smile:

Running a folder comparison tool after the copy is easy to set up if needed, however.

But I think something such as md5 is not that difficult. Get the md5 before copying and Check the md5 for the copy after copying. The thing similarly like this have been integreted in Fastcopy and Totalcommand.

They are not very meaningful if the operating system or hardware caches the data and you read it back and MD5 it from the cache instead of physical storage.

The OS filesystem cache can be huge (gigabytes) in Windows, since it will potentially use all memory that isn't being used for anything else. The drives themselves also often cache data. SSDs move data around constantly as part of other writes to the same blocks. HDDs move data around by themselves as well, to a lesser degree. SSDs and HDDs also have write buffers and sometimes multi-tier storage to speed up writing which also means if you don't wait long enough you may not be reading the data back from its final place. There are so many layers of caching in modern storage and filesystems that reading data back just after it has been written is not a guarantee of anything.

The OS and hardware already guarantee that data written to a device has been correctly written at that moment, so testing it at that point is largely a placebo, and not very useful. The idea is naive, in my opinion.

The hardware may also fail just after you do the test (or even as a result of doing the test, since the test will put extra load on the hardware). The data on an SSD may go wrong when something else causes it to be moved. If you want to verify data integrity, you need to do it regularly, not just once, and doing it right after the copy finishes is the least useful time to do it.

If you are worried about storage hardware failing, it's probably time to replace the hardware.

Devices which support SMART will often tell you when things start failing, as long as the controller and drivers aren't blocking the information from reaching the OS. (Windows will report catastrophic SMART failures, but there are SMART monitoring tools which you can use to get a heads-up about other conditions which might indicate a failure is about to happen but aren't yet catastrophic.)

2 Likes

Thanks for replying. Due to my English I think I need take minutes to read.