Print/Export Folder Listing - Filter folders by tags

[SColquhoun]

Version: DirOpus 12.22 x64

Issue Summary:

Print/Export Folder Listing - Compound filter criteria involving Metadata don't return any results (ex. Folder Type + Metadata tags. Includes AND/OR + literal values or Regex wildcards).

When the same Filter criteria are used with the Find/Search functionality, the desired results are returned, but without the option to export/save the associated "Tags" metadata values.

Use-Case:

DirOpus Folder export-listing is used as the input for scripts that automate creation and maintenance of music composition/production Templates in DAW recording software (Digital Audio Workstation).

Work Process:

1- After installation of Sample Libraries on local pc, I use the "Tags" metadata object to assign a genre/category for the library. The Tags (ex. STR, AMB, WWD, BRA, GTR, KEY, VOC, ETH) are defined for each library's primary folder(s).

(Note: Currently I only use a single "Tags" Metadata value for each Folder. Multiple delimited Tags values are not in scope for this use-case)

2- The Print/Export folder listing utility is then used to generate a clipboard or text-file at the Folder level, where each line is the full path of the primary Library folder alongside respective "Tags" values.

This is where the problem occurs. No results are returned.

Desired Results (mock up):

Tags	Full Path
BRA		V:\VSTi\Orchestral Tools\Berlin Brass
BRA		V:\VSTi\Orchestral Tools\Berlin Brass\BBR - Main
BRA		V:\VSTi\Orchestral Tools\Berlin Brass\BBR A - Additional Instruments
BRA		V:\VSTi\Orchestral Tools\Berlin Brass\BBR B - Muted Brass
BRA		V:\VSTi\Orchestral Tools\Berlin Brass\BBR C - SFX French Horns
HST		V:\VSTi\Orchestral Tools\Berlin Harpsichords
PRC		V:\VSTi\Orchestral Tools\Berlin Percussion
PRC		V:\VSTi\Orchestral Tools\Berlin Percussion\BPC - Main
PRC		V:\VSTi\Orchestral Tools\Berlin Percussion\BPC A - Timpani
STR		V:\VSTi\Orchestral Tools\Berlin Strings
STR		V:\VSTi\Orchestral Tools\Berlin Strings\BST - Main Collection
STR		V:\VSTi\Orchestral Tools\Berlin Strings\BST A - Special Bows I
STR		V:\VSTi\Orchestral Tools\Berlin Strings\BST B - Special Bows II
STS		V:\VSTi\Orchestral Tools\Berlin Strings\BST D - First Chairs
STR		V:\VSTi\Orchestral Tools\Berlin Strings\BST E - SFX
WWD		V:\VSTi\Orchestral Tools\Berlin Woodwinds
WWD		V:\VSTi\Orchestral Tools\Berlin Woodwinds\BWW - Main Collection
WWD		V:\VSTi\Orchestral Tools\Berlin Woodwinds\BWW A - Additional Instruments
WWS		V:\VSTi\Orchestral Tools\Berlin Woodwinds\BWW B - Soloists I
WWS		V:\VSTi\Orchestral Tools\Berlin Woodwinds\BWW C - Soloists II
WWD		V:\VSTi\Orchestral Tools\Berlin Woodwinds\BWW D - SFX
ENS		V:\VSTi\Spitfire\Spitfire Audio - BBC Symphony Orchestra
BRA		V:\VSTi\Spitfire\BBC Symphony Orchestra - Brass
PRC		V:\VSTi\Spitfire\BBC Symphony Orchestra - Percussion
STR		V:\VSTi\Spitfire\BBC Symphony Orchestra - Strings
WWD		V:\VSTi\Spitfire\BBC Symphony Orchestra - Woodwinds

3- The text file from step 2 above is used as the input for a script that splits the Vendor, Library Name, and Category for each text file line, and automatically creates all required nested Folders within the DAW software.

4- Steps 1 through 3 are performed "en masse" initially when setting up the preliminary DAW Template, but are also used to apply periodic updates to the existing templates upon acquisition of new Sample Libraries or changes in categorization.

Desired Print/Export Filter Results:

Print/Export utility should export...
1- Folders only, and
2- Only Folders having a "Tags" Metadata value.

(Note: Files are ignored for this step as they are processed subsequently, and desired export Filter criteria work perfectly for Files).

Filter Definitions attempted via Print/Export Folder Listing:

Notes:
- Most, if not all of these work as desired in the Search/Find functionality
- Additionally, in cases where results were returned, Folders not matching the criteria were listed. Such results included Folders without "Tags" values or Folders with "Tags" values different from the Filter criteria (ex. listing ETH when only GTR was specified.)

1. Folder + Regex Wildcards

   Type	Match	Folders
   	AND
   Tags	Match	Regex  .*
   

   -Note: various regex criteria were used separately in different attempts to detect any "Tags" chars.

   Ex.  .*  	or  	/./  	or	 ^(?=\s*\S).*$
   

   

2. Folder + (literal single value A OR literal single value B, OR literal single value C, etc.)

   Note: I fully realize this definition is likely only pairing the Folder Type with the first "Tags" value, i.e. (Folder + literal single value A) OR literal single value B OR literal single value C etc.
   However, without knowing exactly how DirOpus parses the multiple criteria, this combination was simply a test to see what would result.

   Type	Match	Folders
   	AND
   Tags	Match	GTR
   	OR
   Tags	Match	VOC
   	OR
   Etc.
   

   

3. (Folder + literal single value A) OR (Folder + literal single value B) OR (Folder + literal single value C) OR etc.

   Type	Match	Folders
   	AND
   Tags	Match	GTR
   	OR
   Type	Match	Folders
   	AND
   Tags	Match	VOC
   Etc.
   
   

   

Debugging Steps:

• Applied same Filter criteria to the Search/Find functionality where the desired results were achieved (may not be a work-around since I need the additional capabilities of the Print/Export listing utility).

• Simply inverting the "Print / Export Folder Listing" Filter does appear to show the correct results for the opposite settings. i.e. where Tags No Match Regex .*

• Attempted various combinations of the above Filter criteria alongside Wildcard, Regex, and Case-sensitive options

• Attempted various filter criteria alongside all 4 Source-Folder options (Flat-view unmarked, Flat-view Mixed, Flat-view Mixed (no Folders) -doesn’t make sense for this use-case-, Flat-view Grouped)

  

Any suggestions would be greatly appreciated.

Thanks in advance for your time and consideration.

[Leo]

It seems to work OK in my tests:

Your filter definitions may be the issue.

You should not combine AND and OR in boolean logic without grouping the expressions.

a AND b OR c AND d is ambiguous and could mean
((a AND b) OR c) AND d or
a (AND (b OR (c AND d)) or several other things.

In normal boolean logic you'd use brackets around things to prevent the ambiguity. In Opus filters, sub-clause does the same thing.

But you don't really need that here, as you could do your examples using wildcards to make things easier:

• Type Match Folders
• AND Tags Match (GTR|STR|KEY|BRA|WWD) Use Wildcards

[SColquhoun]

Leo,

Thanks for your kind instruction. The details are greatly appreciated! I especially appreciate mention of the “Subclause” option. I don’t know how I never noticed that previously, but it will be incredibly useful.

Root Cause #1 – Incorrect Filter Criteria (Resolved)

Ahhh…..I was so close on an early attempt. I had initially tried…

AMB|BRA|BRS|ENS|ETH|GTR|HST|KEY|PRC|STR|STS|SYN|VOC|VOS|WWD|WWS

…but without ( ).

With your guidance I was able to achieve the desired results. While initial implementation of your suggestion did not return any results for my particular work process, the details you kindly provided led me to discover yet another misconception that I have about the Print/Export functionality.

Root Cause #2 – Incorrect assumptions about export functionality when...

• Filter set to “Type = Folder”
• Any folder within a given Folder’s hierarchy does not match additional Filter criteria

Folder Hierarchy

In my scenario, I have the following folder structure...

{Drive}:\VSTi\{Vendor}\{Library}\{Library Sections}

Ex. 
V:\VSTi\Orchestral Tools\Berlin Strings\BST - Main Collection

i.e. 
Drive:\One\Two\Three\Four

Tags Assignment

I assign Tags at the “Library” (level Three) and “Library Section” (level Four) levels.

Ex. 
V:\VSTi\Orchestral Tools\Berlin Strings [Tags:STR]\BST - Main Collection [Tags:STR]

i.e 
Drive:\One\Two\Three [Tags:STR]\Four [Tags:STR]

No Tags are ever assigned at the Vendor level (level Two), only at levels Three and below.

Statement of Problem

If any folder (level 2) within the chosen Source Folder (Level 1) does not match the Filter criteria, then no folders are returned, even if sub-folders (levels 3+) do match the Filter criteria.

In other words, it appears that folders process “top-down”, and a folder hierarchy’s evaluation is terminated upon the first occurrence of a folder that does not meet the Filter criteria. This is also true for the initial folder(s) within the Source Folder, regardless of whether any sub-folders do match the Filter criteria.

Example

Recalling the previous screen capture depicting the Folder Hierarchy, it appears the following Filter criteria causes folder evaluation to terminate at the first set of folders. I would assume this is because the string “Berlin” was not found in the “Orchestral Tools” folder name.

	No Results

However, if I alter the Filter criteria to then include the string “ra”, that is common between the Vendor folder “Orchestral Tools” and the Library sub-folders for “Berlin Brass”…

…I am met with the following results.

'Tags	Full Path
		V:\VSTi\Orchestral Tools
BRA		V:\VSTi\Orchestral Tools\Berlin Brass
BRA		V:\VSTi\Orchestral Tools\Berlin Brass\BBR B - Muted Brass
STR		V:\VSTi\Orchestral Tools\Orchestral String Runs

Note: I intentionally tested using Folder Name criteria in order to rule out any potential differences in behavior between Metadata (ex. the Tags cited in my use-case) and common Folder properties. However, additional testing showed this behavior also applies for Filter criteria using Tags. Thus in my work process, no folders were considered since the Vendor-level folders do not contain Tags.

Desired Functionality

Folder Filter criteria would evaluate all sub-folders instead of terminating at first occurrence of a non-match.

I had hoped to simply run the Print/Export utility with the criteria you kindly provided, using the root “V:\VSTi” folder as the Source Folder. The objective is to have the Print/Export functionality evaluate all folders within the Source Folder hierarchy, resulting in a list of all folders and sub-folders having Tags Metadata values. In other words, Folder evaluation would consider all sub-folders even when a parent folder does not satisfy Filter criteria.

Additional Investigation

In my investigations, I discovered many topics mentioning various ways to limit sub-folder evaluation via implementation of the “Subfolder” and “Location” options (below).

I attempted to manipulate the Location option, but to no avail. It seems designed to limit subfolders based on the occurrences of “\”, but it is not likely intended to force evaluation of subfolders.

Final Question

Is there any such functionality within DirOpus?

I am able to resolve the issue by assigning dummy Tags at the Vendor level, but in the interest of furthering my knowledge and understanding of DirOpus, I would greatly appreciate any guidance on what might be the best approach, if this is even possible.

Thanks again for all your efforts.

[Leo]

@SColquhoun

If any folder (level 2) within the chosen Source Folder (Level 1) does not match the Filter criteria, then no folders are returned, even if sub-folders (levels 3+) do match the Filter criteria.

Use Tools > Find Files to find what you want, then use Print / Export Folder Listing to turn the list of results into a text/csv file.

[SColquhoun]

Leo,

This is excellent...Many thanks!

This now resolves the first two issues and addresses 99% of my desired work process.

If I might kindly beg your indulgence, I have one final challenge around sorting the "coll://Find Results" collection sent to the Print/Export Folder Listing utility.

Statement of Problem
The "Print/Export Folder Listing" utility seems to ignore user-defined Sort configurations when printing a Collection ("coll://Find Results") from the "Find Files" utility.

Print / Export Folder Listing Sorting
I have defined the following Sort properties in attempts to export the Folder listing, sorted by Path.

Find Files Collection Format
I also configured the "Find Files" Collection Default Format to sort first by Location, then File.

Additional Context - Example Folder Content
For simplification purposes, suppose I have the following three vendors...

Vendor A
Vendor M
Vendor Z

...and I have three Sample Libraries within each Vendor folder.

v:\VSTi\*A*\Concert Grand Piano
v:\VSTi\*A*\Uilleann Pipes
v:\VSTi\*A*\World Strings

v:\VSTi\*M*\Cathedral Organ
v:\VSTi\*M*\Symphonic Brass
v:\VSTi\*M*\Viola da Gamba

v:\VSTi\*Z*\Asian Percussion
v:\VSTi\*Z*\Chamber Strings
v:\VSTi\*Z*\Didgeridoo

Desired Results

I would expect to see the "Print / Export Folder Listing Sorting" results to appear as dictated by the sort definitions shown previously.

For example, the listing should sort on Vendor first, then by Library within each Vendor.

v:\VSTi\*A*\Concert Grand Piano
v:\VSTi\*A*\Uilleann Pipes
v:\VSTi\*A*\World Strings
v:\VSTi\*M*\Cathedral Organ
v:\VSTi\*M*\Symphonic Brass
v:\VSTi\*M*\Viola da Gamba
v:\VSTi\*Z*\Asian Percussion
v:\VSTi\*Z*\Chamber Strings
v:\VSTi\*Z*\Didgeridoo

Find Files Results (Perfect)
The Find results are perfect as they adhere to the Default Collections Format configurations, thus exactly matching the listing depicted by the Desired Results immediately above.

Print / Export Folder Listing Results (Incorrect Sort)
However, when the Find results collection is sent to the Print/Export utility, the exported results do not seem to obey the sort configuration defined within the Print/Export utility.

The example below shows that the results are actually sorted by Library first, then by Vendor.

v:\VSTi\*Z*\Asian Percussion
v:\VSTi\*M*\Cathedral Organ
v:\VSTi\*Z*\Chamber Strings
v:\VSTi\*A*\Concert Grand Piano
v:\VSTi\*Z*\Didgeridoo :o)
v:\VSTi\*M*\Symphonic Brass
v:\VSTi\*A*\Uilleann Pipes
v:\VSTi\*M*\Viola da Gamba
v:\VSTi\*A*\World Strings

Hypothesis

Perhaps the Find Files utility adds the Library folder either as the Key value (or only value) in a sorted Collection, thus overriding the correct sorting applied to the Find results (which would make perfect sense)?

Ex. Assumed contents of Find Files results Collection that are sent to the Print/Export utility:

Asian Percussion
Cathedral Organ
Chamber Strings
Concert Grand Piano
Didgeridoo
Symphonic Brass
Uilleann Pipes
Viola da Gamba
World Strings

Regardless, the sorting applied to the Find results does not seem to be applied to the Collection. Additionally, it appears as though the Print/Export Sort configurations are also ignored? This second part was unexpected.

Any final thoughts, suggestions, or guidance would be greatly appreciated.

[Leo]

Editing the format (via the Edit button in the Print Folder Contents dialog) and sorting by location, then name, works fine here:

It's possible I am missing something in all the detail here.

[SColquhoun]

Thanks Leo.

Please allow me to clarify...

Correct

When used individually, "Tools-->Print / Export Folder Listing" does return correctly sorted results.

Incorrect

However, when used in tandem with "Tool-->Find Files", the above Print/Export Sort definitions (Location, then Name) are ignored and the results are instead sorted by Name only, alphabetically ascending.

In other words...

Step 1 - Tools-->Find Files
Results of Find are properly sorted based on Collection preferences)

Step 2 - Tools-->Print / Export Folder Listing
Please note the Source Folder is the Collection (coll://Find Results) passed from the Find utility executed in Step 1.

In this scenario, the results of the "Print / Export Folder Listing" are sorted alphabetically by Name only, ignoring any Print/Export Sort configurations.

Ex. of incorrect results (actual results are difficult to read owing to varying path lengths)

v:\VSTi\*Z*\Asian Percussion
v:\VSTi\*M*\Cathedral Organ
v:\VSTi\*Z*\Chamber Strings
v:\VSTi\*A*\Concert Grand Piano
v:\VSTi\*Z*\Didgeridoo :o)
v:\VSTi\*M*\Symphonic Brass
v:\VSTi\*A*\Uilleann Pipes
v:\VSTi\*M*\Viola da Gamba
v:\VSTi\*A*\World Strings

Expected / Desired Results

v:\VSTi\*A*\Concert Grand Piano
v:\VSTi\*A*\Uilleann Pipes
v:\VSTi\*A*\World Strings
v:\VSTi\*M*\Cathedral Organ
v:\VSTi\*M*\Symphonic Brass
v:\VSTi\*M*\Viola da Gamba
v:\VSTi\*Z*\Asian Percussion
v:\VSTi\*Z*\Chamber Strings
v:\VSTi\*Z*\Didgeridoo

I tested with several Sort configurations on Path, Location & Name, etc., but in all cases the Print/Export Sort definitions were ignored when used in conjunction with the Find Flies utility.

Apologies for the confusion, and I hope this is helpful in illustrating the issue.

As always, I truly appreciate all your time and efforts!

[Leo]

The example I gave was from Tools > Find Files.

[Leo]

Looking at your screenshot, you still have Flat View > Grouped turned on in the Print dialog. That doesn't really make sense when working on a Find Results collection (it's already flat), and may be what's changing the sort order.

[SColquhoun]

Thanks Leo!

Unfortunately, un-ticking the "Flat-View" option did not alter the way the results are incorrectly listed.

Files v Folders?
I can't help but wonder if there might be a difference in functionality between Files and Folders.

In other words, in your testing of the Find Files functionality I noticed you searched for .exe Files, whereas I am searching for Folders having Tags Metadata.

Additional Test Results - Sorting Files - Successful; Folders - Unsuccessful
While I realize there shouldn't be any difference between Find and Print of Files v Folders, my testing shows that execution of "Find" on Files does indeed result in a properly sorted Print/Export listing of the Collection; However, when executing a Find on Folders, the Print/Export Sort definitions are ignored.

In a previous test regarding the Folder Metadata, you were kind enough to have tested with the following example.

Final Test
If I might impose upon you one last time, would you be so kind as to test the following scenario?

1. Tools-->Find Files on Folders having Tags Metadata Match (Cat|Dog)

2. Tools--> Print Export Folder Listing on Collection and confirm sorted results

In order to properly simulate my folder structure, you may have to create additional folders with the following hierarchy.

Folder A
	Dog Folder
Folder Z
	Cat Folder

If properly sorted, your Print/Export results should be...

Folder A\Dog Folder
Folder Z\Cat Folder

...But I suspect they will actually be...

Folder Z\Cat Folder
Folder A\Dog Folder

Apart from the Files vs Folders difference, I am not aware of any factors that might cause an incorrect sort order.

Continued gratitude as we bring this saga to a close!

[SColquhoun]

@SColquhoun

Final Test
If I might impose upon you one last time, would you be so kind as to test the following scenario?

1. Tools-->Find Files on Folders having Tags Metadata Match (Cat|Dog)
2. Tools--> Print Export Folder Listing on Collection and confirm sorted results

Hi Leo,

I was wondering if you had a chance to confirm whether or not the Print/Export utility's Sort definitions are correctly applied to a collection of Folders returned by the Find Files utility.

Per my previous post, my tests indicate that this process correctly sorts a "Files" Collection, but not a "Folders" Collection.

If you are able to confirm my findings, then I will create a Bug/Enhancement request.

Thanks for your time and continued efforts.

[Leo]

Haven't had a chance to check it yet, but I'd be surprised if folders were handled differently to files unless you have turned on the "keep folders sorted alphabetically" option in the format, which would of course do what it says.

(If you can't get the order you need, you could always export to CSV and then sort the columns using Excel or a similar tool, as well.)

[SColquhoun]

Hi Leo.

I scoured my DirOpus Folder Format Display Settings and can confirm that none of my settings are using the “Keep folders sorted alphabetically” option. I checked all Formats for good measure.

As much as I appreciate your suggestions regarding sorting via Excel, if his were solely for use in my studio, I would not have any issues doing so. However, I am preparing a presentation for a few on-line music fora where I hope to outline my process of using Directory Opus to manage music Folders’ categories via Tags, followed by a folder list export to a text file for use with my scripts that auto-generate Template folders and instrument presets.

For composers with hundreds or even thousands of sample libraries, the process of setting up a template can easily take several weeks. Many composers hire assistants to help manage and maintain these templates since even adding new instruments or folders can waste valuable time. It is said that for every eight hours of composing, only 2 minutes of usable music are completed. Thus it is easy to see how composers prefer to stay focused on the composition process instead of dealing with technical activities or inefficient work processes.

Thus far DirOpus has been a perfect fit for my proposed work process, but this Folder-sort issue is the final piece of the puzzle that could help bring my approach to fruition. If this last 1% can be made to work in DirOpus, it will truly make for a highly effective and efficient process. Otherwise, I fear the moment additional software is introduced to manage the sorting, the target audience will likely deem the process too tedious and potentially lose interest. For one-time activities this might seem a trivial issue, but when part of a highly iterative process, the impact increases.

Apologies for my stubbornness, but this final piece will be of tremendous benefit should we discover how to make it work.

As always, I really appreciate all of your time and efforts.

[SColquhoun]

Hi Leo,

I bother you this last time only to report a final update. Based on your previous feedback I discovered the root cause of my Folder sort issue.

I re-investigated your previous comments from 10 Dec (seen below) regarding the "Keep folders sorted alphabetically" Format option.

Frustratingly enough, I reviewed all of the following Format options...

..but I mistakenly assumed the Print/Export Folder options were inherited from a corresponding configuration amongst one of these Formats.

Even so, I cannot imagine how I overlooked the most relevant and (what should have been) most obvious configuration...the "Print Folder Contents" Format.

In summary, as you so accurately assessed, this configuration was indeed set to "Keep folders sorted alphabetically". Once this setting was removed, the results appear exactly as desired, for Folders as well as Files.

I offer my sincere and humble apologies for pestering you and wasting your time. Additionally, I would like to express my sincere gratitude for your generous support, guidance, and feedback.

I am updating the Title of this thread to more accurately reflect the issues herein.

Best wishes for a happy and safe holiday season.