Best practise for "manual backup operations"?

Help & Support
1

Hi Dopus friends :slight_smile:

Ok, this post is kind of a mix of questions and maybe feature requests. Sorry about that.
What I am looking for is your "best practise" when it comes to "manual backup operations".

I am currently facing some problems you might have some input or ideas to. I also share some of the problem the user from this post also faces: Most secure way for a major sort out: [url]Most secure way for a major sort out]

Here is the story.
Over the years I have tons of backups spread over different drives, DVDs and external drives.

Now I want to MERGE all backups on my new HDD drive.

What I miss is a:

  1. "copy but skip files with identical MD5" function.
  2. in addition this function should work together with a "rename file if filename is identical" function.

I think I have read most posts here about using the syncronize and delete duplicate function. But I have some comments to those functions and some general comments to the copy functions.

  1. I think the 'find duplicate function' (when using MD5) also needs a function which "keeps the folder structure when comparing files". This function should only compare the files within same folders and subfolders and not across folders. Or am I missing something here?
    Is there a smarter way?

  2. Don't get me wrong, but in many cases the syncronize and duplicate and move functions are just too slow and unreliable. I always end up doing stuff like this:

Instead of MOVE I use COPY and then DELETE. What do you do?
Also it's much faster on external USB drives. Generally I think it's faster in most cases to copy+delete than to move. And it's also more secure because I use to check the sum of files before deleting the original folder. What is your best practise?

Instead of SYNCRONIZING I often do manual copy both ways with e.g. the UPDATE ALL function.
For 1TB of data and with hundreds of thousands of files it's just faster and doesn't really kill the system. Also I can pause in between and add more events to the COPY QUEUE. This means I can leave the computer for half a day or more. My system freezes when I activate 2 or 3 syncronize operation which even might involve the same HDs. There is a queue function needed here. I am new to dopusrt and I guess I can build that using the @async modificator (since I always need to sync fixed folders).

Instead of DUPLICATE I use other ways. The problem is that I have hundrets of folders with my company logo inside. They all have the same name, size and date. I want to keep the logo together with my InDesign files in the folder. So when I want to merge the whole folder structure with several backup places which also have the same folder structure the duplicate function should also have a "switch" to only look for dublicates within the folder structure and not across folders. Thas way I can use the MD5 check without deleting all logos from other folders. Is that possible today or did I miss some essential function?

THanks :smiley:

2

Doing a move between volumes is a copy-then-delete. They're identical operations (except that you might copy everything and then delete everything, vs copying and deleting each file before moving to the next; but that should not matter on most HDD filesystems), so it's strange that the speed would be so different.

3

Hi leo :slight_smile:

Strange. I should do some measurements to verify. Can it be that all my HDDs are set up in a RAID1 or RAID10 system? And moving stuff on the same RAID makes is slower? hmmm - I will do some testing about that and get back.

Maybe I think it's faster because all the files are copied faster so I can continue my work while I delete 500 GB in the background (and don't notice it because I am already working on something else). While then I move stuff I cannot proceed with my work until the whole operation is completed. Ok, that would of cause be an unfair comparison from my side :slight_smile:

4

My 2 pence is that when doing important backups, it is always better to manually copy everything you need, then once everything is done and you are completely satisfied (i.e. you do a second pass checking everything), delete the original. This is not so much for a technical reason, but in case:

  1. you decide you need to do something else with the original copy of the data. i.e. Back it up to another location, or duplicate specific folders on the localmachine or copy to another remote machine. Obviously this is quite dependant on your situation.

  2. It is IMO opinion better to delete the original backup after a week or two after creating your single machine backup. This is prob a bit of paranoia on my part, but a lot of the time the new backup is taking place on a new HDD which can be checked & validated by software, but the SMART info is not that good a metric on drive failure as the google data centre paper shows and you never know. So unless you need to free up the space, keep the original backup for as long as possible. 2 copies are better than one. :slight_smile:

5

Hi ET

For me - after loosing important stuff which I actually had backed up on two different drives and both drives died almost simultaniously after some crazy incidents - paranoia is my best friend!

So your advise is... wise! :smiley:

6

.. oh ... and both backup devices were RAID1 systems - so all in all the data was "secured" on 4 HDs located two different places. Still not good enough! .. so paranoia it is :open_mouth: