Column - Custom Text and Regexp

wowbagger · July 13, 2014, 2:03am

This script adds support for creating custom columns using regexp against the file name and path. The columns are configured using a dialog, with all Dopus column types and custom column settings being supported.

Example output

The first column are the actual folders, the rest are RegExp extracted columns from the folder name.

Column config for above example

Normally you would create the columns using the dialog, however to try the above example you can paste the below config in to global variable RegexpCustomColumnsConfig global-variable-management-dialog.

Folder Names

photo shoot - black cat 4 - 2017-01-01 - 20
photo shoot - cat 2 - 2017-10-01 - 100
photo shoot - cow 5 - 2016-01-02 - 100
photo shoot - ginger cat 4 - 2016-12-01 - 95
photo shoot - max 3 - 2016-12-02 - 100
photo shoot - puppy 1 - 2017-01-01 - 75

Column config

{
  "columns": [
    {
      "name": "Shoot Date",
      "displayName": "RegExp Sample 2",
      "nogroup": true,
      "nosort": true,
      "datetimeformat": "yyyy-mm-dd",
      "regexp": "(.*?)(\\d\\d\\d\\d-\\d\\d-\\d\\d).*",
      "captureGroup": "\\1\\2",
      "returnTheFirstNonEmptyGroup": false,
      "type": "date",
      "defwidth": "20",
      "skipEmptyCaptureGroup": true,
      "justify": "right",
      "inputItemProperty": "name",
      "filter": "All",
      "firstValid": false,
      "label": "Shoot Date",
      "header": "Shoot Date",
      "maxstars": "",
      "grouporder": "",
      "sorting": "normal",
      "infotiponly": false,
      "output": "$2"
    },
    {
      "name": "Shoot Rating",
      "displayName": "RegExp Sample 3",
      "nogroup": true,
      "nosort": true,
      "datetimeformat": "yyyy-mm-dd",
      "regexp": ".*?\\s-\\s.*?\\s(\\d)",
      "captureGroup": "",
      "returnTheFirstNonEmptyGroup": false,
      "type": "stars",
      "defwidth": "10",
      "skipEmptyCaptureGroup": false,
      "justify": "right",
      "inputItemProperty": "name",
      "filter": "Folders",
      "firstValid": true,
      "label": "Shoot Rating",
      "header": "Shoot Rating",
      "maxstars": "5",
      "grouporder": "",
      "sorting": "normal",
      "infotiponly": false,
      "output": "$2"
    },
    {
      "name": "Shoot Processed Relative",
      "displayName": "RegExp Sample 1",
      "nogroup": false,
      "nosort": false,
      "datetimeformat": "yyyy-mm-dd",
      "regexp": ".*?(\\d{2,3})$",
      "captureGroup": "0",
      "returnTheFirstNonEmptyGroup": false,
      "type": "graph",
      "defwidth": "20",
      "skipEmptyCaptureGroup": false,
      "label": "Shoot Processed Relative",
      "header": "Shoot Processed Relative",
      "maxstars": "5",
      "justify": "left",
      "grouporder": "",
      "sorting": "normal",
      "infotiponly": false,
      "output": "",
      "firstValid": true,
      "inputItemProperty": "name_stem",
      "filter": "All"
    },
    {
      "name": "Shoot Subject",
      "label": "Shoot Subject",
      "header": "Shoot Subject",
      "type": "text",
      "defwidth": "20",
      "justify": "left",
      "infotiponly": false,
      "maxstars": "5",
      "datetimeformat": "yyyy-mm-dd",
      "noGroup": false,
      "grouporder": "",
      "sorting": "normal",
      "regexp": ".*?\\s-\\s(.*?)\\s\\d",
      "inputItemProperty": "name",
      "firstValid": true,
      "output": "$1",
      "filter": "Folders",
      "nogroup": false
    },
    {
      "name": "Shoot Processed",
      "label": "Shoot Processed",
      "header": "Shoot Processed",
      "type": "percent",
      "defwidth": "20",
      "justify": "left",
      "infotiponly": false,
      "maxstars": "5",
      "datetimeformat": "yyyy-mm-dd",
      "noGroup": false,
      "grouporder": "",
      "sorting": "normal",
      "regexp": ".*?(\\d{2,3})$",
      "inputItemProperty": "name_stem",
      "firstValid": true,
      "output": "",
      "filter": "All",
      "nogroup": false
    },
    {
      "name": "Shoot Friendly Name",
      "label": "Shoot Friendly Name",
      "header": "Shoot Friendly Name",
      "type": "text",
      "defwidth": "20",
      "justify": "left",
      "infotiponly": false,
      "maxstars": "5",
      "datetimeformat": "yyyy-mm-dd",
      "noGroup": false,
      "grouporder": "",
      "sorting": "normal",
      "regexp": ".*?\\s-\\s(.*?)\\s\\d\\s-\\s(\\d{4}-\\d{2}-\\d{2})\\s-.*",
      "inputItemProperty": "name_stem",
      "firstValid": false,
      "output": "$1 - on $2",
      "filter": "Folders",
      "nogroup": false
    }
  ]
}

How to use

Download the attached .osp file (below) and drag it to the list in Preferences / Toolbars / Scripts.

(Alternatively, drag it into into your Script AddIns folder /dopusdata/Script AddIns, which does the same thing.)
Use the provided button (below) to display the dialog and create you columns.
Once the columns are created you will need to add them to your Lister like you would any other Dopus Column.
The apply changes button, will live update the column if it has been selected.

Configuration Dialog

Download

Requires DOpus 12.7.3

Script:

RegExpColumns.osp v1.13.0 (36.0 KB)

Older Versions

RegExpColumns.osp v1.12.0 (35.9 KB)

RegExpColumns.osp v1.9.0 (35.6 KB)

RegExpColumns.osp v1.8.0 (34.8 KB)

RegExpColumns.osp v1.6.0 (33.9 KB)

RegExpColumns.osp v1.5.6 (33.3 KB)

RegExpColumns.osp v1.5.5 (33.3 KB)

RegExpColumns.osp v1.5.2 (33.2 KB)

RegExpColumns.osp v1.5 (32.6 KB)

Config Button:

This button will display the configuration dialog.

Raw command: RegExpColumnsConfigure

XML button data you can copy & paste from the forum directly to a toolbar:

How to use buttons and scripts from this forum

<?xml version="1.0"?>
<button backcol="none" display="both" textcol="none">
	<label>RegExp </label>
	<tip>Manage RegExp Columns</tip>
	<icon1>#RegExpColumns:RegExpBlue</icon1>
	<function type="normal">
		<instruction>RegExpColumnsConfigure</instruction>
	</function>
</button>

Notes

As I am using a global variable this script might also be helpful (though its not necessary) global-variable-management-dialog.
Sometimes apply changes button does not refresh the columns, Press F5 in dopus and it should update correctly.
I hope you find this useful and easy to use .

tbone · July 14, 2014, 8:41am

Hi wow! o)

Thanks for this!

It seems you had the same idea I had, to kind of introduce some prefix/namespace to keep the script columns menu more tidy, so you wrote this somewhere in the init-section:

cmd.name = scriptNamePrefix + config.columns[i].name;

Where scriptNameprefix = "CustomTextColumns:"

But that prefix does not show up for me anywhere, does it for you?

ps:
You got those

    cmd.match.push_back("Yes");  // when searching, these are the only two options
    cmd.match.push_back("No");

example lines in there again! o))

wowbagger · July 14, 2014, 12:13pm

I didn't use the prefix in the label only the name.

//Set column name
cmd.name = scriptNamePrefix + config.columns[i].name;
//Set label, what is displayed
cmd.label = config.columns[i].name;

I considered also including the prefix in the label. But the Label is what is displayed as the column header title and column selection menu, I did like how it looked. The prefix might not have been needed for the name, however I wanted to identify what columns came from this script. Also I was not sure if name clashes would be an issue.

It would be nice to have the custom columns grouped by script in the selection menu. A layer of sub menus matching the script name would work well.

I updated the script to remove those extra lines thanks.

Leo · July 14, 2014, 12:51pm

That's coming in the next beta.

tbone · July 14, 2014, 2:58pm

Is this necessary? I would think the script is called only for those columns which were introduced by itself.

I was about to suggest something similar, this is a welcome enhancement! Thanks! o)

Leo · July 14, 2014, 3:48pm

It shouldn't be necessary, since the script name already has to be included when referring to the column in a command.

e.g. With Jon's first example script, if you use the button editor to build the command, you get this:

Set COLUMNSADD="scp:Is Modified Column/IsModified"

tbone · July 14, 2014, 3:57pm

How to reference the column is good to know, thanks!

But I think we slighty misunderstood here, I was refering to this sentence of wow:

I think there's no need to find out if a column came from this or any other script, as those callback methods are always called for script specific columns only.

Leo · July 14, 2014, 3:58pm

That's right.

wowbagger · July 14, 2014, 9:03pm

What about for the name? I didn't test, but would it cause issues if there are two columns with the same name? As this script allows the user to add multiple columns, I thought prefixes would prevent any name conflicts with columns added via other scripts.
I agree that the column name check in the callback is not needed. I will remove it from the sample scripts to avoid confusion.

Nice

Leo · July 14, 2014, 9:21pm

The script's name is part of the column's name in any context except inside the script itself (where it is implicit).

See the COLUMNSADD example in my reply above.

wowbagger · July 15, 2014, 1:47am

Makes sense, thanks.

Miran · July 16, 2014, 1:37pm

Hi Wowbagger,

I already started a very "similar" script myself, for exchange of experiences - if you are interested - you could find it attached to this post.

RegExColumns.zip (7.7 KB)

Bye,

Miran

wowbagger · July 16, 2014, 9:38pm

Nice script, mind if I steal some ideas? I like that you support other column types.

Curious, wont this line of code take the second match?

result = result[1];

This is a good idea. This would be a good thing to have built in to Dopus.

	// DOpus Version Check 
	if(!DOpus.version.atLeast(MIN_VERSION)){DOpus.OutputString("ERR: DOpus Version " + MIN_VERSION + "required to run this script!")}

Miran · July 17, 2014, 7:35am

I want you to "steal", that's the main reason I postet the script

Yes it would, and it should I only need the first group for the custom column, i first though about extracting "all" custom columns with one regexp, but as DOpus calls the function for every column individualy (because it is the only valid way) it does not make sense.
The only feature that require to catch all groups would be for concatenated column values, where the input for the new column is not one connected string in the filename but two parts of the filenamen....while writing, this sounds like a valid use case

V1.1 with Multi capture group support
RegExColumns.zip (7.97 KB)
V1.3 added type rating
RegExColumns.zip (9.03 KB)

wowbagger:

This is a good idea. This would be a good thing to have built in to Dopus.
	// DOpus Version Check 
	if(!DOpus.version.atLeast(MIN_VERSION)){DOpus.OutputString("ERR: DOpus Version " + MIN_VERSION + "required to run this script!")}

Indeed, or even better, not just a log output, but a greyed out entry in the preferences page. I will submit this as a feature request.

playful · February 1, 2015, 6:44am

I was confused about where to place the patterns because the Configure button is greyed out. Finally understood where to configure the script so here goes in case someone else is wondering.

By renaming the osp file to zip (or adding osp in Prefs/zip files/zip extensions) and extracting the js file, the script becomes editable and it's possible to paste patterns to configure the script.

This should be immensely valuable.
Thank you!

playful · February 3, 2015, 2:50am

Hi wowbagger,
Your script is very useful. I would suggest a small improvement to allow the use of capture groups. The reason is that JS probably has one of the most crippled regex flavors. To work around the lack of lookbehind or \K, you need capture groups. This could be important for instance in a file where you want to create a column using the nth group of digits.

With the current script this is a minor change. I'd suggest updating the column definitions to allow an optional "group" property.
Then we only need to replace line 65 with something like:

var capturegroup = (typeof config.group === 'undefined') ? 0 : config.group; 
scriptColData.value = result[capturegroup];

Here is a quick example of definitions for a hypothetical file convention that would include a phone number and name:

  columns: [{
    //Name of column, Title.
    name:"area",
    //input values
    itemProperty: "name_stem",
    //regexp to extract value
    re: /^\d+/,
    // group to be returned (optional if whole match, i.e. group 0)
    // group: 0
  }, {
    name:"phone",
    itemProperty: "name_stem",
    re: /^\d+\D+(\d+[- ]\d+)/,
    group: 1
  }, {
    name:"who",
    itemProperty: "name_stem",
    re: /^\d+\D+\d+[- ]\d+\s*(\S+)/,
    group: 1
  }]

(Attaching these incorporated changes for the convenience of anyone interested. Didn't know what to do with the version number and naming convention, sorry if it is wrong.)

Other Thoughts
Here we're running three match attempts when one pattern would be enough to fan the data into three groups, and that's a bit of a shame. It would be better if we could use a single regex for all three columns, using various capture groups as results. I haven't studied the scripting interface so have no idea if processing multiple columns at once is possible.
CustomtextColumn with Groups.js.txt (2.38 KB)

playful · April 18, 2015, 10:39am

Another small addition I'm finding useful:
When creating a naming convention with a view of parsing long file names into regex columns, one risk is to run out of characters for the file name (the Windows max for path + file name is 260).

The following tweak adds columns so you can keep track of the length of file names.

In the column definitions:

  { name:"c.namelen",
    // this column give you the length of the file name
  },

  { name:"c.charsleft",
    // this column give you the number of chars still available to use in the path + file name
    // Windows imposes a 260 maximum length for the Path+Filename
  },

// no need for a custom column for the length of file + pathname as that column is available under General / Path length

In OnCustomText:

  // filename length column?
  if (config.name == 'c.namelen') {
    scriptColData.value = scriptColData.item.name.length.toString()
  }

  // characters left column?
  else if (config.name == 'c.charsleft') {
    // Windows imposes a 260 maximum length for the Path+Filename    
    // how many more chars available in the path + filename? 
    var PathLength = new String(scriptColData.item.realpath).length
    scriptColData.value = (260-PathLength).toString()
  }

  else { // it's a regex column
    if(!config.itemProperty)
    // etc

Leo · April 18, 2015, 11:07am

There's a built-in Path Length column which should be better/quicker for that one of the three. I think the other two are new & useful, though.

playful · April 18, 2015, 10:48pm

Ahhh, indeed! I missed that. Removed it. Thank you.

wowbagger · July 26, 2017, 10:05pm

Thanks for all the feedback @playful and @Miran. Both your scripts were good reading.
A new version V1.5 has been created that uses a dialog to configure the columns. See First post.