The Transferring of Metadata is Confusing Me

Help & Support
1

First, I need to know what is included in Metadata. I have been using Directory Opus for several months now and I need to wipe out my hard drive and reinstall Windows. I usually do this perhaps 1-2 times a year.

Now, Directory Opus had its own way of doing things that Windows 10 did or did not have with respect to layout. For example, I would label certain important document in folders with Directory Opus's own method of labeling. I would sort files and folders in my own particular way in Directory Opus in many folders by using manual sort. Are all of these things that I just stated considered metata? Where are they stored? When I backed up my files via a backup program, did it backup this information too or was it lost?

For example, I have a Synology NAS and they use exFAT. It's unfortunate but I know I lose all of my Windows Alternate Data Stream and ACL permissions. That is something I really hate about my NAS. I wish there was a way to preserve it. What part of Dopus's metadata is saved and will be back there when I reinstall my hard drive?

2

It depends on the type of files, and the type of metadata. If we know how to store it in the file itself, we do that. If we don't, or there isn't a way, we store it in an NTFS ADS stream.

Labels are either stored in NTFS ADS or in your Opus config, depending on Preferences.

Folder formats are store in your Opus configuration and wouldn't normally be considered "metadata".

Custom sorting is stored in NTFS ADS against the folder itself. Not usually considered "metadata".

If you need to reinstall Windows, there are backup tools which will preserve the NTFS ADS data, if that's what you're worried about losing. Most tools intended for system backups (not just ad hoc file and document backups) should handle that. Or just copy the files to another NTFS drive (using something which preserves ADS, of course).

(Needing to reinstall Windows twice a year suggests something is wrong. Maybe you should look into using a virtual machine to try things out before changing your real machine? That can often save you from bad situations and is well worth setting up.)

3

Hi, Leo,

Thank you for your helpful information. I've been thinking about this problem for the past few days. I really appreciate you advice as this is a little heart-renching for me. I never knew that my Synology NAS (EXFAT format) wasn't backing up my metadata for things such as my family photos and videos (date, time, GPS location, and device used). I never knew that this information is totally lost forever.

I don't know what to do with the NAS. It's got about 20 Tb of space left and I have Windows 10. But, I definitely have to get and use external hard drives that are formatted NTFS because the date, time, and GPS location is a huge factor when you are taking pictures for legal evidence, for example. I would like to get a QNap NAS (formatted NTFS) but I don't have the funds for another NAS.

I feel so relieved that labels and folder formats are stored in Opus configuration files. I put a lot of time into sorting files into a specific order, then adjusting slightly with manual sort, and then saving the folder format. So, when I back up my files onto my Synology NAS, reinstall Windows 10, and then put the my data on my Synology NAS back, I hope most of the folder and file view and sort order should be about the same, right? If not, can you suggest the best way to preserve all the things that I considered as "metadata"? Should I back Directory Opus data onto an NTFS formatted drive?

You suggested backup tools which would be able to preserve just the NTFS ADS data. I've purchased a Synolgy NAS, Macrium Reflect and Resilio Sync Hom Pro to be my backup programs.

If you can suggest a few solid backup tools that you were referring to, which would preserve the NTFS ADS data, I would be extremely grateful. I'm looking for them as we speak and I don't know what I'm looking at.

You also recommended that I just copy all my files to another NTFS drive and then just reinstall it back. I'll definitely will do that from now on. It really sucks that Synology didn't warn us. It's a shady move. I must have spent $3k worth of backup hardware that is flawed. Synology should say something on their product boxes warning that it won't backup metadata especially since metadata tags are developing into such a big search tool.

I'm definitely going to look for NTFS drives that can properly copy all Dopus files. Is there a preference for discrip.ion or ADS that you think works best with Dopus "metadata" info like labeling, the specific order of the type of files, etc.?

I've been reinstalling Windows at least once a year on New Years since for decades. I do it when my computer starts to slow down a bit. I know there is virus and maleware on my computer even now because I use a lot of 3rd party programs and the odds are that at least a few of them have planted something. So, I sort of "reset" my computer so I can keep the baddies to a minimum.

My bro in law is in cryptostuff and he said never download 3rd party programs. But, I can't listen to his advice. For example, I love Directory Opus! It's really too helpful.

4

For photos, date, time and GPS location data are stored within the image itself.

This information isn't lost unless you do something that removes it (e.g. edit the image and resave it in something that doesn't preserve it). Copying the image file to another device won't affect that information because it's stored within the file itself.

5

From what I see in help, Backup4all does this.

6

Xyzzy, that's a program that I don't use. Are you sure that Backup4all would back up the ADS data? How? My understanding is that ExFAT (the format that Synology NAS's use) doesn't accommodate ADS at all. I might be wrong.

I think I'll convert to descript.ion and slowly convert all my metadata to it. I can see that as the only solution. I hate computers.

7

You'd have to backup into an archive or disk image of some kind, not to copies of each individual file, for the NTFS metadata to be preserved. There are backup tools which do that though, and some will also preserve the metadata. (I don't know about the particular one in question.)

8

From the help I see this is limited to "mirror" type backup (a simple copy) + NTFS as target drive. Synology supports NTFS on external disks only.

9

Leo,

Ah...I get it. I have Macrium Reflect which copies my hard disk as an entire image instead of a set of files so the ADS gets copied. But, Resilio Sync which I also use is copying by file by file so the ADS probably won't get filed.

So, when I reinstall, I should rely on Macrium Reflect to make a backup of my hard drive. And when my Windows 10 is reinstalled, I should open the disk image created by Macrium Reflect and then slowly move my old folders and files to the new hard drive.

The Resilio Sync acts as an instantaneous backup for me but if I ever need it, I would lose all my ADS data.

Leo, let me ask another question. (I'll repost it.) What is the "best" way to store metadata so I could use it to help me search for files and folders? Which will last many years into the future and development of the system might even progress. These are the following methods:

  1. NTFS's ADS
  2. Directory Opus's method of saving just comments, ratings, and tags.
  3. Descript.ion
  4. one of the numerous other 3rd party tagging system.
10

Xyzzy,

Yep. My synology NAS does support NTFS but only on external drives which means it's a dead end for me. I have about 30 Tb and I can use external drives to slowly transfer 30 Tb. I want a metadata system that will really become permanent for the foreseeable future.

For example, what if I had a picture and I put comments with it. The could be really important comments that I need to rely on later when I go back over the pictures. Or, what if I had a picture from 30 years ago, it would be nice to have comments that stated what was going on.

11

[Moved back into this thread as it's best kept in one place, I think. --Leo]

I posted this same question at the bottom of the previous thread. I thought it should have its own thread. Metadata discussions have been posted a lot but I was wondering what people thought which metadata system is the "best". The criteria is as follows:

  1. It makes it easier to find what you're looking for. Some search engines will give results for any word that pops up in the document. For example, if it's in an obscure field, it will list it with the same importance as a file in which the search term is in the title.

  2. It make tagging a folder or file very easy to do, instead of taking so much time. For example, I don't like Windows's method of using tags in File Explorer because you have to add "tag:" before the tags that you're looking for. I end up never using it.

  3. Longevity. The tagging system is going to probably last for a very long time so you don't have to worry about losing all the metadata that you've saved so far. For example, I have a feeling that they are going to go rid of ADSs in the near future. If so, a lot of metadata will be lost. So expected longevity is important. I'm speculating that Directory Opus's descript.ion system is going to last for a while or else they will provide a way to transfer the old metadata to the new system.

  4. Other cool features.

The ULTIMATE question is what metadata system do you think that I should rely on to organize my data? I think I'll be using Directory Opus for a while.

12

The answer will depend on the person and what the are doing and why, and which which file formats, and which other software.

The subject is too broad/complex to really answer without a lot more specifics and, even then, probably something you just have to work out for yourself. Only you will really know all the details, and you probably won't think of some of them yourself until you try something and run into situations you hadn't thought of.