I've been thinking about this for the past month. A few weeks ago, I made a prototype of OpusCustomFields that uses a sqlite database, and I definitely think that's the way to go. At least for me, it's been quite useful in practice, and having an actual database makes searches much faster (e.g. I was moving my labels and tags, set unmovable timestamps to files, .etc).
In this case, I need some testers to help out before I simply release it, since it could end up changing the design itself. I may have overlooked something, etc. Once it's published, it'll be harder to make those kinds of changes. Anyone interested?
FEATURES:
- Manage custom fields through an easy-to-use interface.
- Almost any item can have custom fields, including FTP, MTP, items inside archives, etc.
- Full support for all data types supported by Opus.
- Custom fields can be used as regular script columns, e.g. for searching, filtering, etc.
- Built-in support for setting values in registered fields.
- Officially, these are the first user columns in Opus that aren't read-only!
- Support for exporting/importing custom fields.
- Support for advanced actions: PULL, PUSH, FLUSH, .etc.
- Seamless data management using a sqlite database.
- Support for directly searching the database, including an advanced mode using SQL SELECT syntax.
MANAGING CUSTOM FIELDS:
To use this script, you first need to have some fields enabled. This can be done from the Configuration window.
To manage your custom fields, you can access it:
- From the Script Management window
- Using the
OPUSCUSTOMFIELDS CONFIG command
You can disable fields. Their data will remain in the database, but they won't be accessible through any command (their status is changed to inactive in the database).
You can customize various properties for a column, including adding a short description.
Once you've made your changes, you need to press Save to apply them (including to the database).
NOTE 1: When creating a custom field, you'll be asked for a name and a data type. These CANNOT be changed later due to the difficulty involved in changing the data type in the database.
NOTE 2: When configuring the fields, the fields reported by the database and those stored in your Opus configuration are synced to keep both sides in sync. No columns are deleted during this process.
NOTE 3: Deleting a column from the Configuration window IS A DESTRUCTIVE PROCESS. It means DELETING the column from the database ALONG WITH ALL OF ITS DATA.
AVAILABLE COMMANDS:
Set:
Allows you to set values for any active fields. Its syntax is similar to how Opus's SET META command works.
Pull:
Creates a copy of all custom fields and their values for a given file.
Push:
Sets values for active fields by reading the data from a file.
Flush:
Removes values from active fields for a given file.
Find:
Searches directly using the database. It supports a simple syntax based on the field data types, as well as an advanced mode using SQL SELECT syntax.
Purge:
Cleans up orphaned data in the database, or removes data for files matching a given wildcard. It can also run the SQL VACUUM command.
NOTES:
-
It is NOT possible to track a file when it is moved/deleted if it has data in the database. To mitigate this limitation, the script uses:
- The unique file ID of each file (for local filesystem files only) is stored in the database, so if a file is moved within the same drive, it can theoretically be found and updated automatically.
- There is an option (which must be enabled in the script's Options window) to save a copy to an ADS and/or a
.dopcf file next to the file whenever the database is modified, when possible (local filesystem files only).
- There is a column that is always enabled and shows the number of fields for a given file. This can be useful if you plan to move the file somewhere else, so you can first use the
PULL command to create a copy.
- There is a column that is always enabled and shows the last time the data was saved to the database.
-
Custom fields can be used in regular Opus searches, although this obviously means using the database indirectly.
-
Datetime, date, and time values are always stored in UTC using ISO-8601 format, but are transparently converted back to the local time zone when used through any method (except PULL, where they retain their original format).
-
When setting data (PUSH and SET), you can use placeholders for any property available in Opus for a file, including script/Evalautor column values. To invoke a placeholder, enclose it in $_value_$.
Example: OpusCustomFields SET "mytags:$_keywords_$;other tag" "timestamp:$_created_$" sets values for mytags and timestamp on the selected files in the source file display, retrieving the keywords and creation date of each file at runtime.
If a value is set to empty, that field is removed for the given file (similar to how SET META works).
-
You can use FLUSH to remove fields more easily. Example: OpusCustomFields FLUSH=mytags,timestamp.
-
The schema for PUSH is a simple JSON object with key:value pairs.
-
Valid values according to the data type are:
- Text: Any string value.
- Datetime:
"now" or formatted as ISO-8601, with optional milliseconds when a time component is present.
- Date:
"now" or formatted as ISO-8601.
- Time:
"now" or formatted as ISO-8601, with optional milliseconds.
- Number: Any positive integer.
- Number,signed: Any integer.
- Double: Any double number.
- Boolean:
true|yes|1 are equivalent to true. Anything else is converted to false.
- Stars: Integer between 0 and 5.
- Percent|Graph: Integer between 0 and 100.
- Duration: Integer or string in the format
xd HH:mm:ss. Days, hours, and minutes are optional.
- Durationh: Integer or string in the format
xd HH:mm:ss. Days are optional.
- Size: Integer or a numeric string.
-
Some values may be converted if they don't follow the guidelines above.
-
No command can CREATE or DELETE fields, or modify fields marked as inactive. Use the Configuration window for those operations.
-
Column import is currently only available from the Configuration window.
. Regarding the format and everything else, currently the companion files are just a JSON schema that stores the value and internal name, with all your custom fields in the same file. So it's easier for DOpus to retrieve them, but probably not so much for other tools like Everything.
. It can parse any .epub, .pdf, and .cb* natively, with support for parsing .opf files as well. So in theory, you could also use those files for custom metadata.