Script to run external commands hidden and capture their output without temporary files

cyilmaz · July 1, 2021, 3:16pm

This is not DOpus scripting per se, however, it's designed to be called from DOpus scripts mainly and can also be used in your other projects, Windows shortcuts, etc.

For a project I needed to run external commands and capture their outputs without using a temporary file. It was a trickier than it seemed, because the usual WSH gives 2 options:

Run() allows you to run programs hidden but cannot capture the outputs
Exec() can capture the output but it shows a shell window

The 2 together seems to be implemented very rarely together, and the few examples I found around were always limited to a single, hard-coded command. This script does exactly that: combining both in one.

You might want to remove the block "// rebuild and consumption example", it's just for demonstration purposes.

-- README.md

WHAT IS IT?

This WSH JScript runs a Windows shell command via %comspec% /c <your command> completely hidden and captures the output without using temporary files, usually implemented as <your command> > %TEMP%\mytempfile.txt.

While running a command hidden and capturing output of commands without temp files - usually with WSH.Exec() - are both universally used, the combination seems to be much rarer. There are programs which do the console hiding perfectly, but for eliminating the temp files, I found no satisfactory and reusable solution, so I wrote mine, particularly my favorite file manager Directory Opus in mind. However, you can use this script universally. Simply replace the final step where environment variables are set with a WSH.popup(), a balloon tip or copy the output to clipboard and you're done.

Normally I'd write this with AHK since it can achieve the same perfectly and can be converted to an .EXE file. Since I write many DOpus scripts which I eventually release for public use, I instead needed a small .js script which my DOpus script create once and reuse for all spawned commands, e.g. launching mediainfo.exe, b3sum.exe, etc. for hundreds of single files without creating a temp file for each one. If temp files do not bother you at all, check out AHK Run command or the suggestions on this page.

It is best suitable for callers which call a lot of small external commands.

One possible usage is to run it on read-only/protected media, e.g. a survival USB drive, which does not create and delete files on the host system or USB drive.

It also helps if you use an MVC, QVC-type SSDs which are highly inefficient for writing and you are cautious about TBW & SSD wear and tear.

HOW TO USE

The output is put into one or more environment variables, depending on if there are 1 or 2 arguments passed to the script.

1st variant (without prefix):

DirectHiddenRunner.js "your command goes here"

Output Environment Variable	Meaning
%DirectHiddenRunnerOUT%	standard output of the command (STDOUT)
%DirectHiddenRunnerERR%	standard error of the command (STDERR)
%DirectHiddenRunnerEC%	Exit Code of your command, usually 0 indicates success

As you can reckon, every call to the script would overwrite old values! Be careful when using this variant.

2nd variant (with a custom prefix for EnvVars):

  DirectHiddenRunner.js "yourEnvVarPrefix" "your command goes here"

Output Environment Variable	Meaning
%yourEnvVarPrefixOUT0%	number of lines standard output of the command (STDOUT)
%yourEnvVarPrefixOUT1%	1st line of STDOUT
%yourEnvVarPrefixOUT2%	2nd line of STDOUT
%yourEnvVarPrefixOUT3%	3rd line of STDOUT
...	the suffices are not zero padded, i.e. they go like OUT9, OUT10... OUT99, OUT100...
%yourEnvVarPrefixERR0%	number of lines standard error of the command (STDERR)
%yourEnvVarPrefixERR1%	1st line of STDERR
%yourEnvVarPrefixERR2%	2nd line of STDERR
%yourEnvVarPrefixERR3%	3rd line of STDERR
...
%yourEnvVarPrefixEC%	Exit Code of your command, usually 0 indicates success

This is a very cheap emulation of arrays. The caller of this script should generate a unique prefix, e.g. based on timestamp, a normalized, cleaned up filename, etc. Then it can be called in parallel and each output will be separated.

While While Windows allows a single environment variable to be maximum 32767 bytes, command-line lengths to set environments are limited to 8192 bytes, so your command might exceed the limits very quickly! The pseudo-array overcomes this limitation if the output of your program is expected to be big.

More info: Environment Variables - Win32 apps | Microsoft Docs

PROs and CONs

Simple variant is much faster but command output length is limited. Use it for simpler commands.

Pseudo-array variant is slower but it's more flexible and command output length is practically unlimited.

IMPORTANT REMINDER

REMEMBER TO REMOVE THE PSEUDO-ARRAY ENVIRONMENT VARIABLES AFTER YOU CONSUME THEM, OTHERWISE YOUR ENVIRONMENT WILL BE OVERFLOWN IN NO TIME!

FOR CMD.EXE REMOVING IS AS EASY AS "SET MY_ENV_VAR_XYZ=" (without quotes).

The variables are created in volatile environment, i.e. not persistent over reboots.

TIPS:

To embed double quotes in the command argument use 2 single quotes, e.g.

tasklist.exe /fi "imagename eq firefox.exe"

should be passed to the script as

DirectHiddenRunner.js "myPrefix" "tasklist /fi ''imagename eq firefox.exe''"

Samples:

Pseudo-array, will work, output in myPrefixOUT and myPrefixEC (will be 0):

DirectHiddenRunner.js "myPrefix" "tasklist.exe /fi ''imagename eq firefox.exe''"

Pseudo-array, will NOT work due to wrong syntax, myPrefixERR and myPrefixEC will be filled:

DirectHiddenRunner.js "myPrefix" "tasklist.exe firefox.exe"

Standard call, will work:

DirectHiddenRunner.js "tasklist /fi ''imagename eq firefox.exe''"

Standard call, will NOT work due to wrong syntax:

DirectHiddenRunner.js "tasklist firefox"

HOME PAGE & CREDITS

cy-gh/CuDirectHiddenRunner · GitHub - June 2021

Portions inspired by

https://groups.google.com/g/microsoft.public.scripting.vbscript/c/VNPfLikhG60/m/nIpAfwKaEbYJ
adjusted to accept command-line parameters to the script and re-written in JS
https://stackoverflow.com/a/22752797: removed unnecessary stuff and added STDERR support

Although I haven't used it, a very simple script which shows the basic idea behind running programs invisibly and capturing their output can be found here: vbs exec hidden Code Example

github.com

cy-gh/CuDirectHiddenRunner/blob/master/DirectHiddenRunner.js

// https://github.com/cy-gh/CuDirectHiddenRunner
// June 2021

var bUseArrayOutput = false;

var iSleepStep      = 33;
var iElapsed        = 0;
var iMaxWait        = 10000;
var oWSH            = WScript.CreateObject("WScript.Shell");
var oEnvVolatile    = oWSH.Environment("Volatile");
var oScriptArgs     = WScript.Arguments;
var sCommand        = '';
var sPrefix         = '';

var myFullName      = WScript.ScriptFullName;
var myName          = WScript.ScriptName.replace(/.js$/, '');


if (oScriptArgs.length === 1) {
    bUseArrayOutput = false;

This file has been truncated. show original

local_man_1996 · July 10, 2021, 2:36am

Damn how did I not catch this thread.
I think this is just what I have been looking for.
Will do some testing in the coming days, thanks allot for sharing this!

Leo · July 10, 2021, 2:45am

If you want the method that uses a temporary file to capture the output, there's an example here:

(And a few more around the forum, but that's a self-contained one that's easy to experiment with.)

local_man_1996 · July 10, 2021, 2:47am

Sweet! I will check them out for sure.

cyilmaz · July 10, 2021, 8:00am

I just checked in the "hybrid variant" used in MediaInfo script. It combines best of 2 variants.

github.com

cy-gh/CuDirectHiddenRunner/blob/master/DirectHiddenRunner-Hybrid.js

/*
https://github.com/cy-gh/CuDirectHiddenRunner
This file runs given command hidden, captures its output and puts it into volatile environment variables.


From DOpus you can consume it like this:

    function(oItem) {}
        // VERY IMPORTANT: remove the volatile variables once you consume the output
        function removeVolVar(sVar) {
            util.shellGlobal.Environment("Volatile").remove(sVar);
        }
        function getUniqueIDForString(sString) {
            var blob = DOpus.create().blob();
            blob.copyFrom(sString);
            return DOpus.fsUtil().hash(blob, 'md5');
        }
        var OUTPUT;
        var exit_code;
        var cmd;

This file has been truncated. show original

However, as Leo points out to other scripts, stick to WShell.Run("cmd.exe /c blah.exe .... > tempfile.txt", 0, true) (that's all it takes, with some asterisks on double quotes handling) unless you really don't want any temp files. Passing the output from the runner's memory to DOpus script is brittle and very ugly.

Excluding the temp files, we have very few alternatives to pass data: Clipboard, Environment Variables, Re-entering the DOpus script via DOpusRT. That's it, only 3. No IPC or shared memory shenanigans (IPC with AHK is possible and quite powerful though). DOpusRT, although I use it for multithreading, seemed to be overkill, requires protecting the strings from shell and other changes to your DOpus script. Clipboard is out of question, because it messes your current clipboard contents and you cannot run multiple commands in multiple listers, because they'd overwrite each other's output instantly. So environment variables is left, but that requires reconstructing the output and removing the variables again... Nah, I don't like it.

One could say "it's ugly but after all it works", I see it as "it works but it's so buttugly."

local_man_1996 · July 10, 2021, 10:04am

I`ll bear that in mind!