Add multiple filters and tags to files using ADS

PURPOSE:
This script addin allows multiple On–Off filters, and multiple text tags, to be added to any file. When present, they are stored in two alternate data stream (ADS) files attached to the file.

FILTERS:
I have several situations where I need different syncs of unsystematic subsets of the items in a particular directory. This requires multiple On–Off filters that are independent of all the other metadata of the items. The addin can create a hidden alternate data stream file 'MultiFilters.txt' to hold any number of independent filters, which can then be used for syncing, filtering, searching, or whatever. This ADS file with its filters may also be added to directories, but that is not recommended (see below).

A filter actually has three values, On ("Yes"), or Off ("- -"), or 'not set'. For filtering purposes, a filter is 'not set' if the value in the ADS filter file is only spaces, or if there is no ADS filter file; this sets the value in a column to "␣␣␣" (three spaces).

TAGS: A second ADS file called 'MultiTags.txt' holds multiple independent tags. These tags could hold, for example, marks, or comments, or the date the item was processed. As with filters, a tag may be 'not set'.

COLUMNS: The addin contains functions to toggle multiple filter or tag columns corresponding to each defined filter or tag.

REFERENCE ITEM FOR FILTERS AND TAGS: Often a nearby item, which I have called a 'reference item', already has the filters or tags required for some newly created items. Identifying this reference item and copying its filters or tags can be done very quickly within this addin.

ADS FILES: Some help is given with displaying and editing ADS files in general. A column gives the number of ADS files attached to each item, the names of the ADS files can be displayed, new ADS files can be created, and text ADS files can usually be edited. Other operations require third-party tools (see below).

This addin replaces the 'Sync' addin that I posted in December. Immediately after posting, I found that I needed multiple syncs in the same directory, which required multiple filters. I then adapted the code to multiple tags as well. Many thanks to leo, jon and tbone for their continual advice about subtle details and very basic things.

DOWNLOAD:
Everything is contained within the one addin script. Download the script addin ADSFiltersAndTags.js below. Open 'Preferences - - > Toolbars - - > Scripts'. Drag the file onto the right-hand panel.

USER CONFIGURATION:
First, go to 'Preferences - - > File Operations - - > Copy Attributes' and make sure that the following two checkboxes are turned on:

  • 'Copy metadata (comments, keywords, etc)'
  • 'Copy all NTFS data streams'

Secondly, click on 'Edit' in 'Preferences - - > Toolbars - - > Scripts - - > ADSFilterAndTags', and follow the four steps below and at the top of the script file to customise the addin:

Configuring Filters and tags:

  • Edit the first two lines for each required filter, giving the filter name and the default column width. Example values have already been given for the first few filters, and they should be edited first.
  • Then do the same for the tags, which also have a third line allowing left, right or centre justification.
  • The filter and tag indices should be consecutive, without gaps.
  • NEVER delete a filter or tag already used with files, just ignore it.
  • If more than 10 filters or tags are needed, the body of the script will need to be edited. For each extra filter (tags are analogous):
    — First, add an extra block below the block starting: if (nFilterNames > 9) {
    — Then add an extra function below the block starting: function OnFilter009 (scriptColData) {
  • Each filter already has the three matches necessary for filtering — they are the last lines of each filter cmd in 'OnInit', and have the form cmd.match.push_back ("..."). Each tag past the second, however, only has the trivial match cmd.match.push_back ("␣␣␣") for an empty tag (three spaces), but more matches can be added if needed.

Three further minor configurations:

  • By default, each item's 'Date Modified' is advanced by 1 minute so that the item will be included in any sync. This can be changed: each item's 'Date Modified' can be set to now, or it can remain unchanged.
  • The text editor for editing ADS text files needs to be specified. The default 'Notepad' will always work, but another editor may be used. The full path to the .exe file, rather than an alias, may be required.
  • The default name of the hidden files for filters and tags may be changed, but this is not advised. One reason for doing this could be to 'start again' with the addin.

Copy and store the relevant configuration part of the addin file. This may be needed in case of In case of re-installation or upgrade.

THE SEVEN AVAILABLE FUNCTIONS, AND THE POSSIBLE COLUMNS:

  • SetADSFilters allows any number of filters to be turned on or off simultaneously for all the selected items. Once filters have been set, start and configure a DOpus sync between source and destination in the usual way. Then click 'Define' to the right of 'Filter' and select 'Name'='Script column', select 'Match'='Match' or 'NoMatch', select the filter in the third column and "Yes" or "- -" or "␣␣␣" (three spaces) in the fourth column.

  • SetADSTags allows one tag to be set or removed for all the selected items.

  • ChooseReferenceItem sets the full path of a single selected item as a persistent global variable named 'ReferenceItem'. This could have all kinds of purposes, but its purpose here is:

  • PasteADSFiltersOrTags pastes all the filters or all the tags of the reference item to all the selected items. This is useful when, for example, a directory is updated with new items.

  • ToggleADSFilterColumns allows any subset of the filter columns to be toggled in the lister display. It also allows all filters to be shown in a single column, and allows a column giving the number of ADS files attached to each item.

  • ToggleADSTagColumns is the analogous function for the tags columns.

  • EditADSFile allows (usually) editing of any text ADS file, and the creation of a new ADS text file.
    — If no item is selected, it displays all the ADS files in the directory, allowing editing of each.
    — If one item is selected, it displays all the ADS files of that item, allowing editing of each and the creation of a new ADS file.

LIMITATIONS AND WARNINGS:

  • Alternate data streams only work on Windows NTFS file systems, and cannot exist on Fat32. In particular, any USB or portable hard drives used for sync or storage should be formatted NTFS.
  • Cloud backup services may or may not preserve ADS files reliably.
  • ADS files associated with directories cannot be deleted, even with third-party software, except by deleting the directory. THE USER IS THEREFORE WARNED THAT ONE SHOULD PROBABLY RESTRICT THE FILTERS AND TAGS TO FILES, AND NOT USE THEM WITH DIRECTORIES. (To delete an ADS files attached to a directory, create a new directory, then copy the contents of the old directory to it, then delete the old directory, then rename the new directory.)
  • A 'Permission denied' error occurs when trying to add an ADS file to some protected items, such as some Windows Update Standalone Installer files.
  • The Clipboard is used in the calculations for the column showing the number of ADS files. This was done for speed in large directories, but it may be a nuisance for users without third-party Clipboard software such as ClipCache. (Search the script for "/R" to find how to do this using a temp file rather than the Clipboard.)
  • Some software, such as Notepad++, strips any ADS files when saving a file. Notepad does not, and UEStudio has an obscure setting to preserve ADS files: 'Advanced -> Configuration -> File Handling -> Advanced -> check Copy file instead of using rename when creating backup file'.
  • Hidden ADS files are used in Windows for many things, and should not normally be removed. Some operations with them are tricky. Tools for manipulating them can be found within Windows and at DOpus here and here, and also at the bottom of FlexHex.
  • The most straightforward tool for .csv listing and deleting individual ADS files seems to be Nirsoft's tool. It cannot, however, delete ADS files attached a directory.

ADSFiltersAndTags.js V1.1
06/03/16
ADSFiltersAndTags.js.txt (66.5 KB)

2 Likes

Thanks for the script! How to delete Tag000: , Tag001: , Tag002? In Everything 1.5 Alpha I needed brevity for easy viewing.
I tried for three hours to learn and revise, but I haven't learned. :upside_down_face:

I spent hours and days upon just trying to get to the bottom of having a consistent means of storing metadata to every single file on windows C: and NTFS drives. In a manner that is open ended above all the metadata can be readily accessed by:

  • Powershell
  • Directory Opus
  • SearchEverything

and atleast manuplated in:

  • Powershell
  • Directory Opus

I went through a number of tools trying to figure this out:

  • Exiftool
  • MediaInfo
  • Nirsofts PropertySystemView
  • Directory Opus Metadata Panel and corresponding command set
  • Windows Pre Powershell era Tool called Wmic

Nirsofts tool PropertySystemView is broken for Windows 11, am certain of it. it does not get any of the System.[Property] nor cannot append any data to them. I spent allot hours on this as I have allot of confidence on his utilities but I just concluded its broken, maybe its Windows 11

Directory Opus threw me of and gave me a long goose chase. Its Metadata Panel CAN write data to the Tags, Comments and Rating "Properties" of any file but it does this in a (clever) discriminate way. for files that do not support these properties it will write them its NTFS ADS stream and for files that do, it writes it in a standard way.
This threw me of, no matter what external tool i used to read, say tag data I simply could not find what Opus supposedly wrote to the file. it was so paintfull. I can now only hope to heed future individuals from this pitfall.


All in all your tool is perfect. Does all that I needed done and more.I can acces the data in Powershell and in Search Everythings Custom columns. I am so happy and estatic over this :smiley:
As for this:

ADS files associated with directories cannot be deleted, even with third-party software, except by deleting the directory. THE USER IS THEREFORE WARNED THAT ONE SHOULD PROBABLY RESTRICT THE FILTERS AND TAGS TO FILES, AND NOT USE THEM WITH DIRECTORIES.

Its perfectly fine for me but for those that may want an alternative, consider this the Desktop.Ini soltution. With it I was essentially able to write metadata into any folders for virtually any properties there exists on windows using some registry hacks. This metadata can be accessed from anywhere (Powershell, Directory Opus, SearchEverything)
Here is an example of metadata being written to Tags, Rating and Comments columns:


The advantage to this method is, of course is your just dealing with a simple text file manupulation (of that Desktop.Ini file), as well as their not being risks of data being lost overt the cloud etc.

The registry hacks to hijack the System.Properties columns and others are over at SuperUser site:

I hope this helps the next person.
I saved your source code as well, it looks to be usefull just as a learning case.
Thanks allot for sorting all this out.

1 Like