FAYT suggestions list

I quickly wrote a FAYT script which implements the request from this thread for @mgn but I hit a major roadblock.

Given these directories under current lister path:

  • subdir1\my first directory

  • subdir2\my second directory

  • subdir3\foobar

This FAYT script takes an input like !m e, matches them via a regex like m.*e (simplified for demonstration) and attempts to list the first 2 in the suggestions popup. At all times, I can see the entries in the suggestions list in Output window, where I put an identical copy of DOpus vector into a JS array and print it out.

According to docs, the suggestion popup can have up to 2 columns: a simple string array, a map of key+value pairs or a vector of 'key\tvalue's. DOpus does few things though:

  1. The suggestion list is internally converted to a map (hash-table), even though a string array or Vector is given, e.g. ["match\tvalue1", "match\tvalue2"] results in only match\tvalue1 being shown.

  2. DOpus tries to match the user input with the keys itself, instead of just showing the suggestions list and letting the user choose.

  3. If I start typing the keys exactly as they're shown and start backspace'ing and enter nonsense and keep playing with the input, at some point the suggestions list stops being shown although I can see the results in the console.

  4. The suggestions list does not seem to get updated in real-time (example below); again I can see the results in the console.

There are easy ways around #1, but not with #2 constantly getting in the way. #2 is by a mile most disruptive of these and also most likely the cause for both #3 & #4; both are easily reproducable, I can upload a video if you want.

If suggestions list is set to only the full path or just subdir names like ['my first directory', 'my second directory'], the input !m e clearly ignores suggestions altogether because of #2, because 'm e' != 'my first/second directory'. So if instead ['m e 1\tmy first directory', 'm e 2\tmy second directory'] is used to compensate for #2 (and numbers added because of #1), then they're shown but a. it is ugly but can live with it, but b. backspace breaks it when I type !m e 1 (still 1 result) and backspace to ! then !xyz and backspace to ! and to !m again, and voila, the script stops being called. If #2 wasn't there, #3 would be less of a problem. Also if I want to match foobar folder above, by typing !foo. DOpus finds a match right after !f and shows the suggestion (['f 1\tsubdir3\\foobar']), but as soon as I type !fo, the suggestions list disappears (because 'fo' does not match 'f 1'?), although I can still see the result in the console, I have to backspace back and forth to get 'fo 1' shown. By the way, in #3 the stops getting called entirely, in #4 the script is getting called but the suggestion list disappears, but probably at their core #3 & #4 might be the same issue even: #2.

I imagine #2 makes sense in most use cases, where results start with the FAYT input, but not for this one, or am I missing something very trivial, like a simple, user-friendly solution? Can we maybe have a flag or something to control #2 where script author can decide if DOpus should try to match the results itself or stay out of it and let user choose? This would increase the power of FAYT scripts immensely. I can already see a universal launcher via FAYT :smiley:

Here's the script if you wanna test - Output window shows what the inputs & outputs, incl. suggestions list:

// @ts-check
/* eslint quotes: ['error', 'single'] */
/* eslint-disable no-inner-declarations */
/* global DOpus Script */
///<reference path="./_DOpusDefinitions.d.ts" />

var DEBUG = true;
function $dbg(/**@type {any}*/str) { if (DEBUG) DOpus.output(str); }
function $dbgClear() { if (DEBUG) DOpus.clearOutput(); }

$dbgClear();
var CACHE = 'CACHE';

function OnInit(/** @type {DOpusScriptInitData} */ initData) { // eslint-disable-line no-unused-vars
    initData.name = 'CuGoSubdir';
    initData.desc = 'CuGoSubdir';
    initData.default_enable = true;
    initData.group = 'cy';
}

function OnAddCommands(/** @type {DOpusAddCmdData} */ addCmdData) { // eslint-disable-line no-unused-vars
    var cmd = addCmdData.addCommand();
    cmd.name = 'TestListAllSubfolders';
    cmd.method = 'OnGotoSubdir';
    cmd.hide = true;

    var fayt = cmd.fayt;
    if (!fayt) return;
    fayt.realtime = true;
    fayt.enable = true;
    fayt.key = '!';
    fayt.backcolor = '#3c3c3c';
    fayt.textcolor = '#ff9900';
    fayt.label = 'Go to Subdir';
    fayt.realtime = true;
    fayt.flags = DOpus.create().map();
    fayt.flags.set(1, 'do not use case-insensitive comparison');
    fayt.flags.set(2, 'do not ignore .git folders');
}

/**
 * caches the subdirs of any given path
 * @returns {string[]}
 */
function _getSubdirsCacheForPath(/** @type {string} */ path) {
    if (!Script.vars.exists(CACHE)) {
        $dbg('initializing cache');
        Script.vars.set(CACHE, DOpus.create().map());
    }
    /** @type {DOpusMap} */
    var cache = Script.vars.get(CACHE);
    if (!cache.exists(path)) {
        $dbg('initializing cache for path: ' + path);
        var allDirs = readSubdirsOfPath(path);
        cache.set(path, JSON.stringify(allDirs));
    }
    // $dbg('cached dirs: ' + JSON.stringify(JSON.parse(cache.get(path)), null, 4));
    Script.vars.set(CACHE, cache);
    return JSON.parse(cache.get(path));
}

/**
 * this is the method where you can define the default behaviour as you wish
 * current implmentation creates a regex like:
 *
 * /^path.*(inputToken1|inputToken2)/
 *
 * where input is tokenized by space characters, e.g. 'Foo Bar' => ['foo', 'bar']
 *
 * the path itself as prefix is not necessary since we get only the subdirs anyway,
 * but I left it for future Everything Search integration
 */
function _buildRegexFrom(/** @type {string} */ path, /** @type {string} */ input, /** @type {boolean}*/ noLowerCase) {
    var joinedTokens = input.split(' ').join('.*');
    return new RegExp(preg_quote(path) + '.*(' + joinedTokens + ')', noLowerCase ? '' : 'i');
}

/**
 * {@link https://stackoverflow.com/questions/280793/case-insensitive-string-replacement-in-javascript#280805}
 * @param {string} str
 */
function preg_quote(/** @type {string} */ str) {
    // http://kevin.vanzonneveld.net
    // +   original by: booeyOH
    // +   improved by: Ates Goral (http://magnetiq.com)
    // +   improved by: Kevin van Zonneveld (http://kevin.vanzonneveld.net)
    // +   bugfixed by: Onno Marsman
    // *     example 1: preg_quote("$40");
    // *     returns 1: '\$40'
    // *     example 2: preg_quote("*RRRING* Hello?");
    // *     returns 2: '\*RRRING\* Hello\?'
    // *     example 3: preg_quote("\\.+*?[^]$(){}=!<>|:");
    // *     returns 3: '\\\.\+\*\?\[\^\]\$\(\)\{\}\=\!\<\>\|\:'
    return str.replace(/([\\\.\+\*\?\[\^\]\$\(\)\{\}\=\!\<\>\|\:])/g, '\\$1'); // eslint-disable-line no-useless-escape
}

/** poor man's sprintf, only %s */
function pmsprintf(/** @type {String}*/ template) {
    var idx = 0, args = Array.prototype.slice.call(arguments, 1);
    return template.replace(/%s/g, function () {
        return idx < args.length ? args[idx++] : '%s';
    });
}

function OnGotoSubdir(/** @type {DOpusScriptFAYTCommandData} */ faytCmdData) { // eslint-disable-line no-unused-vars
    if (!faytCmdData.suggest) return;

    var tokens = faytCmdData.cmdline;
    var currentPath = '' + faytCmdData.tab.path;
    var noLowerCase = faytCmdData.flags & 1;
    var noGitIgnore = faytCmdData.flags & 2;
    var _re = _buildRegexFrom(currentPath, tokens, !!noLowerCase);
    $dbgClear();
    $dbg(pmsprintf('path: {%s}, noLowerCase: {%s}, noGitIgnore: {%s}, tokens: {%s}, regex: {%s}', currentPath, noLowerCase, noGitIgnore, tokens, _re));

    var allDirs = _getSubdirsCacheForPath(currentPath);
    if (typeof allDirs !== 'object' || typeof allDirs.length !== 'number') {
        $dbg(pmsprintf('got from cache: %s, %s', typeof allDirs, allDirs));
        return;
    }
    /** @type {DOpusVector} */
    var someDirs = DOpus.create().vector();
    var someDirsJS = [];
    for (var i = 0; i < allDirs.length; i++) {
        var subdir = allDirs[i];
        if (!noLowerCase) subdir = subdir.toLowerCase();
        if (!noGitIgnore && subdir.indexOf('.git') >= 0) continue;
        var matches = _re.exec(subdir);
        if (!matches) continue;
        // var sugstr = pmsprintf('%s\t%s', tokens, allDirs[i]); // not working because apparently all entries are converted to a hash-table
        var sugstr = pmsprintf('%s\t%s', tokens + ' ' + (someDirs.length+1), allDirs[i]); // we'll use allDirs[i] not subdir in case original path name was lowercase'd
        someDirs.push_back(sugstr); 
        if(DEBUG) someDirsJS.push(sugstr);
    }

    $dbg(JSON.stringify(someDirsJS, null, 4));
    faytCmdData.tab.updateFAYTSuggestions(someDirs);
}


function readSubdirsOfPath(/** @type {String} */ currentPath) {
    function isValidDOItem(/** @type {DOpusItem} */ oItem) {
        return (typeof oItem === 'object' && typeof oItem.realpath !== 'undefined' && typeof oItem.modify !== 'undefined');
    }
    function isDir(/** @type {DOpusItem} */ oItem) {
        return (typeof oItem === 'object' && typeof oItem.realpath !== 'undefined' && oItem.is_dir === true);
    }
    var allDirs = [];
    $dbg('currentPath: ' + currentPath);

    var fsu = DOpus.fsUtil();
    var fenum = fsu.readDir(currentPath, 'r');
    if (fenum.error) {
        DOpus.output('fsUtil.readDir - cannot read:\nError: ' + fenum.error);
        return;
    }
    var icnt = 0, imax = Math.pow(10, 7); // just as a runaway-limit for while loop
    while (!fenum.complete && icnt++ < imax) {
        var subitem = fenum.next();
        if (isDir(subitem) && isValidDOItem(subitem)) {
            allDirs.push('' + subitem.realpath);
        }
    }
    return allDirs;
}

I had already pointed out something similar to point 2 in two previous posts that were ignored. Let's see how it goes if someone else but me asks for it.
I agree with you that it would be great if there were more liberty in this matter. By default, DO could handle it as it does currently, but also allow the script to implement its own option.

For example, suggestions only appear when the user starts typing something. I would like a different list to appear when they haven't done so yet (similar to the command history), but currently that's impossible.

Given that there are few scripts of that type published on the forum (apart from mine), I imagine this must be of ultra-low priority right now.

1 Like

Not ignored, there are just hundreds of threads to work through at the moment.

1 Like

Yeah, I don't think there is any malice or whatever behind it. As Leo said they must be flooded with so many questions, feature requests and all the todos already in their own lists. It is perfectly normal in my experience, once you release your product to mass usage, the people find gaps in your design or want changes, and extensions, etc pp.

Overall I am pleasantly surprised with the FAYT scripting. I wasn't expecting DOpus to perform this well in the speed department, but GG job. FAYT definitely needs a few extensions and few fine cuts here and there, then they'll be one of the most sought after features.

I didn't mean any of that either :laughing:

IMO it's one of the best features of DO13.

1 Like

100%. And add Everything integration to that! It also needs some more features imho, but I am holding back with the requests until the dust is settled a bit :wink: