SearchEverything: Getting Opus to work with Everything

>>> INTRODUCTION

  • Everything is, as some of you already know, a nice little program similar to Indexers like Google Desktop Search and Windows Search except it is a lot smaller and takes less than 10 seconds to index more than 5000000 files. It indexes only filenames and paths and relies on the USN journal of the NTFS filesystem to instantly detect and display changes.

  • Pros: Lightning fast, Light and unintrusive, instantly detects changes

  • Cons: Only works with NTFS, Only has Details view as of now

  • SearchEverything brings you those filesystem-wide search results to dopus as a collection where you are free to do whatever you wish with the full power of the opus.

>>> INSTALLATION

  1. Download and install Everything from voidtools.com.
  2. Go to Tools > Options > General --- make sure Everything service and Start on system startup are checked
  3. Download SearchEverythingCoreCLI from github releases and extract it to a folder (e.g. C:\Program Files\SearchEverything). Take note of the folder if you change it!
  4. Copy the following button code and paste it to a toolbar
Button code (click to expand)
<?xml version="1.0"?>
<button display="icon" label_pos="right" type="three_button">
	<label>Everything</label>
	<icon1>#actions:findf</icon1>
	<button display="both" hotkey="ctrl+f3" label_pos="right">
		<label>Everything (Dialog)</label>
		<tip>Search Everything</tip>
		<icon1>#default:find</icon1>
		<function type="normal">
			<instruction>@keydown:none</instruction>
			<instruction>&quot;C:\Program Files\SearchEverything\SearchEverythingCoreCLI.exe&quot; &quot;{dlgstring}&quot;</instruction>
		</function>
	</button>
	<button display="both" hotkey="shift+f3" label_pos="right">
		<label>Everything (Clipboard)</label>
		<tip>Search Everything for Current Clipboard Text</tip>
		<icon1>#default:find</icon1>
		<function type="normal">
			<instruction>&quot;C:\Program Files\SearchEverything\SearchEverythingCoreCLI.exe&quot; &quot;{clip}&quot;</instruction>
		</function>
	</button>
</button>

>>> USAGE

  • Left click the toolbar button (or Ctrl+F3 as default hotkey) to get a dialog where you enter search text.
  • If you want to use Regular Expression search enclose the search in slashes (e.g. /searchText/)
  • Additionally you can directly search on the text in the clipboard with a Right click on the toolbar button (or Shift+F3 as default hotkey).
  • Results if any will appear in a new collection called Everything

>>> OLD STUFF

Collapsed to save space
  • Download "Everything" and the Everything Command-line Interface (es.exe [not needed for 3.0.0+]) from here.

  • Download SearchEverything package and extract to a folder (ex. Program Files\Everything)

  • Start "everything"

  • Add the button code below to your toolbar. You WILL need to edit the paths where the buttons point to. If you're in portable mode use relative paths.

  • See appendix after button code for button function descriptions.

    >>> DOWNLOAD 4.0.0.0
    SearchEverything v.4.0.0.0.rar (62.3 KB)
    >>> DOWNLOAD 3.0.0.0
    SearchEverything.v3.0.0.rar (65.2 KB)
    >>> DOWNLOAD 2.2.0.0
    SearchEverything 2.2.0.0.rar (41 KB)

    **>>> UPDATE v.4.0.0.0 DOPUS 10+ ONLY IMPORTANT **

  • Fixed compatibility with 1.4.0.x beta versions of everything

  • Using net framework 4.5.2 (no need to install 2.0 on windows 10 anymore)

    **>>> UPDATE v.3.0.0.0 DOPUS 10+ ONLY IMPORTANT **

  • Completely rewritten (again) to utilize included Everything.dll enabling direct communication with Everything.exe via IPC (dll is from david's everything sdk)

  • es.exe is no longer needed

  • Instant retrieving of results (although it takes a while for dopus to initialize the collection).

  • No more codepage conversions. All (as opposed to everything) is now done in unicode via IPC. If a filename is garbled - blame the IPC Api.

  • Added manifest to require administrator elevation when run (required to interact with Everything.exe)

  • Added switches --install and --remove to respectively install/uninstall the "Everything://" protocol handler. Read more about it below.
    **>>> UPDATE v.2.2.0.0 DOPUS 10+ ONLY IMPORTANT **

  • Fixed searches with 1 result returning 0 results

  • Added commandline switch "--query" >> displays a search input dialog (defaults content to clipboard)

  • Added commandline switch "--clip" >> runs a search for the current clipboard text (if any)
    >>> UPDATE v.2.1.0.0 DOPUS 10+ ONLY

  • Fixed hang on searches with many results by adding default timeout on search = 500ms (tested with 12000 results, changeable in config)

  • Optimized search (not using cmd anymore)

  • Moved go path=coll://Everything/ to SE. Results appear as soon as we get them. Remove that line from your buttons to avoid double redraw

  • Added commandline option: "--genconfig" to generate sample config file which you can edit afterwards

  • Added config option MaxExecutionTime - maximum time in ms to wait for results from es (default: 500)

  • Added config option PopupDisplayTime - maximum time in ms to display result popup. Set to 0 to disable popup. (default: 2500)

  • Added config option MaxResultCount - placeholder only, not yet implemented

  • Fixed result missmatch of 1
    **>>> UPDATE v.2.0.0.0 DOPUS 10+ ONLY **

  • Completely rewritten in C# .net 2.0
    >>> TOOLBAR BUTTON for v.2.1.0.0 UPDATED [11-06-04 23:20 CET]
    (USAGE: copy code, right click toolbar in dopus > customize > right click anywhere on the toolbar again > paste > voila)
    NOTE: If you have UAC enabled you MUST have @admin present in the button function for SE to work so get the below updated code again if you added it before the update.

<?xml version="1.0"?>
<button display="both" label_pos="right" separate="yes" type="three_button">
	<label>Everything</label>
	<icon1>#default:find</icon1>
	<button display="both" icon_size="large" label_pos="right">
		<label>Everything (Dialog)</label>
		<tip>Search Everything for Specified Keyword(s)</tip>
		<icon1>#default:find</icon1>
		<function type="normal">
			<instruction>C:\Program Files\Everything\SearchEverything.exe &quot;{dlgstring}&quot;</instruction>
		</function>
	</button>
	<button display="both" icon_size="large" label_pos="right">
		<label>Everything (Clipboard)</label>
		<tip>Search Everything for Current Clipboard Text</tip>
		<icon1>#default:find</icon1>
		<function type="normal">
			<instruction>C:\Program Files\Everything\SearchEverything.exe &quot;{clip}&quot;</instruction>
		</function>
	</button>
	<button backcol="none" display="both" icon_size="large" label_pos="right" textcol="none">
		<label>Everything (Program)</label>
		<tip>Starts / Brings Everything to Front</tip>
		<icon1>#find</icon1>
		<function type="normal">
			<instruction>cd C:\Program Files\Everything</instruction>
			<instruction>C:\Program Files\Everything\Everything.exe</instruction>
		</function>
	</button>
</button>

>>> BUTTON FUNCTIONS

  1. [Left Click - Default Behavior] Displays an input box to enter search criteria (any part of a filename is ok)
  2. [Right Click - ClipboardString] Directly searches with the current clipboard text as keywords
  3. [Middle Click - Open Everything] Starts everything (if not running) and opens a new everything window
    >>> FAQ
  • Q: ES is not starting. Gives me cryptic errors. A: Install .net framework 2.0 for windows or reinstall your current one
  • Q: Why do I get an elevation prompt asking me for administrative privileges every time I search? A: You have UAC enabled. Everything.exe implicitly requires elevation and as such the tools interacting with it need that as well to be able to query it.
  • Q: ES does not run on Win98... A: And DOPUS does? Only NT based XP+ OS' supported.

>>> PROTOCOL HANDLER FUNCTIONALITY

  • You can register a protocol handler using:
    SearchEverything --install (remove it using SearchEverything --remove)
  • When installed you can hit Win+R and type everything://your search terms hit enter and see your results in dopus.
  • This becomes useful as you can code a page and make links using the same protocol handler so clicks inside your browser trigger searches.
    OLD STUFF STOP READING iF YOU ARE USING DOPUS10+

VERSION 1.2 [2009/12/08 21:40] KNOWN ISSUES
SearchEverything1.2build20091208-2140.rar (205 KB)

  • NO UNICODE SUPPORT (GET DOPUS 10 AND USE SEv2.0)
  • SLOW LOADING OF RESULTS (1 by 1)
  • LIMITED RESULTS (due to slow loading)
  • NOT SO PORTABLE
    VERSION 1.2 BUTTON CODE
<?xml version="1.0"?>
<button display="both" icon_size="large" label_pos="right" type="three_button">
	<label>Everything</label>
	<icon1>#default:find</icon1>
	<button display="both" hotkey="f3" icon_size="large" label_pos="right">
		<label>Everything</label>
		<tip>Search Everything for Specified Keywords</tip>
		<icon1>#default:find</icon1>
		<function type="normal">
			<instruction>Delete FILE=coll://Everything QUIET </instruction>
			<instruction>CreateFolder NAME=coll://Everything</instruction>
			<instruction>/home\Apps\searcheverything.exe {dlgstring}</instruction>
			<instruction>Go PATH=coll://Everything</instruction>
		</function>
	</button>
	<button display="both" hotkey="alt+f3" icon_size="large" label_pos="right">
		<label>Everything (Clipboard)</label>
		<tip>Search Everything for Current Clipboard Text</tip>
		<icon1>#default:find</icon1>
		<function type="normal">
			<instruction>Delete FILE=coll://Everything QUIET </instruction>
			<instruction>CreateFolder NAME=coll://Everything</instruction>
			<instruction>/home\Apps\searcheverything.exe {clip} </instruction>
			<instruction>Go PATH=coll://Everything</instruction>
		</function>
	</button>
</button>
12 Likes

Hello,

thanks for the scripts but I cannot get it to work. The only difference on my system that the installation path to DOpus is

C:\Program Files\GPSoftware\Directory Opus

Tried to modify the ahk-script with

if 0 < 1
{
    Run,%A_WorkingDir%\..\..\Everything\Everything.exe
    ExitApp
}
everythinges = %A_WorkingDir%\..\..\Everything\es.exe

But this doesn't help. Any help much appreciated.

Regards
xbprm

The following code works


#SingleInstance, force
; SearchEverything Wrapper by Devocalypse
; Uses CMDRet by corrupt (http://www.autohotkey.com/forum/viewtopic.php?t=3687)
if 0 < 1
{
    Run,C:\Program Files\Everything\Everything.exe
    ExitApp
}
everythinges = C:\Program Files\Everything\es.exe
cmdstring = "%everythinges%"

Loop, %0%
{
    param := %A_Index%
    IfInString, param, %A_Space%
  {
   cmdstring = %cmdstring% "%param%"
  }
else
  {
    cmdstring = %cmdstring% %param%
  }
}
results := CMDret(cmdstring)
Loop, parse, results, `n
{
if A_LoopField
   {
    RunWait,"C:\Program Files\GPSoftware\Directory Opus\dopusrt.exe" /cmd Copy TO="coll://Everything" COPYTOCOLL=member FILE="%A_LoopField%",,Hide
    clipboard = "C:\Program Files\GPSoftware\Directory Opus\dopusrt.exe" /cmd Copy TO="coll://Everything" COPYTOCOLL=member FILE="%A_LoopField%"
   }
}
CMDret(CMD)
{
  VarSetCapacity(StrOut, 1000000)
  RetVal := DllCall("C:\Program Files\GPSoftware\Directory Opus\Apps\cmdret.dll\RunReturn", "str", CMD, "str", StrOut)
  Return, %StrOut%
}

Now trying to make it independent from actual installation path.

1 Like

I know I should have made the paths absolute but i decided against it for better portability if you export to a USB.

You changed the correct paths. I do not see a reason for it not to work. The only possible explanation could be if you run it as a script instead of a compiled executable. I will test a few scenarios and let you know what i find out.

Edit: It seems the working directory is the problem. I will rewrite the ahk source to use the Script Dir and get dopusrt path from registry. Expect a new version within an hour or two.

Edit2: Updated to version 1.2. It seems there is no proper path in registry under HKLM or HKCU for Dopus and since "Everything" is portable there is no foolproof way to find these paths so you have to manually specify the paths in SearchEverything.ini contained in the version1.2 rar.
Note: Do NOT add a trailing slash \ after the paths

hi apocalypse,
i am trying to get your little tool to work, but something isn't working. i am only getting this output "Everything IPC service not running." how do i run the ipc service? i can't find any info about this problem.

thanks neophite

[quote="neophite"]hi apocalypse,
i am only getting this output "Everything IPC service not running." how do i run the ipc service?[/quote]

I checked and I get the exact same output when Everything.exe is not running in the background. This is just a fronted for it so it needs to be in the taskbar or system tray, besides database updates cannot happen if Everything is not running.

There is a possibility to start it just for the duration of the search but that would be beyond the point of having fast results. If you press the search button and then don't enter anything as a search term and hit ok it will start everything.exe itself.

hi apocalypse
everything was running, but i forgot to start it with admin rights :unamused: after doing that everything works fine...
i hope you / the dopus developers find a solution for the collection import.
btw great addon for 2 great programs :smiley:

I'm not an Opus developer, so I caution you to tread carefully with what I'm about to suggest. (Be sure to back up your Opus configuration before you try anything.)

Opus file collections are just plain text XML files named with a .col extension. So you should be able to you to create them externally from Opus, provided you get the content schema correct. From what little I know of this, there are a couple of issues you need to workaround.[ol][li] Certain meta data is stored for each file as an attribute for that file's element inside the .col file. Many of these attributes are not stored as plain text or decimal values.

The fields I see in my .col files are:[ul]
[li] access[/li]
[li] create[/li]
[li] modify[/li]
[li] attr[/li]
[li] size[/li][/ul]

I do not know if this is the extent of all meta data stored inside of a .col file, or if it is related to my Collections default format.

I believe size is stored as the number of bytes in octal format; thus, the file element for a file that is 26 bytes, would have an attribute of size="32". I do not know how the date/time fields are stored (access=, create=. modify=), nor the attributes (attr=).

For what it is worth, I have tried creating a collection .col file with no meta data for any files. The resulting file collection in Opus will show all the files with 0kb size and the date/time stamps fields will all be blank, et cetera. Opus doesn't seem to hang or anything on such lack of meta data. However, a collection with only file names and paths may be of little or no value.

Like I said , I'm not an Opus developer, so this is all just what I have gleaned from my own observations.
[/li]
[li] As you may already be aware the .col files can be copied to your /dopusdata\Collections folder, after which you must restart Opus before the new collections take effect.

This thread has some relevant discussion.[/li][/ol]

Hopefully this helps you some.

Thank you for the insight Kenneth.
I am aware of the location of the collection files and their xml schema. The problem is mainly that you have to restart dopus after editing any of those files before the changes are read by it so it becomes a problem. AFAIK there is no way to load or reload a collection with a command within opus without having to restart.

Concerning the metadata i think it only appears in special collections such as find duplicates or search results.

I made an experiment with a custom collection and copied some random files' paths to it. Restarted dopus and all the columns were correct. That is version 9.51 though so I am not sure about the older ones. I'm trying now to invoke some way of refreshing the collection from within dopus and will keep you posted with my progress.

Hi apocalypse,

now it works perfectly. Thank You.

Regards
xbprm

apocalypse,

This is the best tool I've found all week!

Thanks again.

Paul

Thank you for this guide apocalypse, however I do have a question about the behaviour of the Z-ExploreFile command.
In Everything v1.2.1.451a, if I use the 'Open Path' command on a folder, it opens the actual folder that has been selected not the directory that contains it.
Is there any way to get it to work such that DOpus will go to the directory containing the folder and select the folder?

Thanks again for this guide, it's been most helpful.

In

Go PATH="&filename&"
Select PATTERN="&filename&"

Idea was that when you pass a filename (not a foldername) dopus opens its parent folder regardless of that it's a file and not a folder path and then the command is issued to select the file.

It's not the case with archives and folders it seems. I'll have to write a script to intercept and split the path and then pass the parent path and the filepath / foldername or understand the cryptic everything.ini more and find a way to pass both arguments from there.

After some testing heres the AHK code:[code]
#SingleInstance, ignore
#NoTrayIcon

if 0 > 1
exitapp
if 0 < 1
exitapp
else
argpath = %1%

dopusdir = YouDopusFolder

SplitPath, argpath , filename, dirname, extension, namenoext
Run, "%dopusdir%\dopusrt.exe" /cmd Z-Everything FILENAME="%filename%" DIRNAME="%dirname%"
exitapp
[/code]
Command Code: [code]

<?xml version="1.0"?> Z-Everything Launch Lister and Focus File dirname,filename,filepath #cli Go PATH="&dirname&" Select PATTERN="&filename&" DESELECTNOMATCH [/code] Everything INI code:[code] open_folder_path_command=$exec("folder_where_you_put_it\ExploreFolder.exe" "%1") [/code]

This works good in general BUT here comes a BIG problem:
dopus SELECT always interprets the pattern as the dreaded Pattern Matching Syntax suggests so that if your filenames have characters like [ ] ( ) # ~ they get intepreted as operators and not as filenames and nothing gets selected.

I tried search and replace within the script but the escape character poses a problem for the commandline which gets passed to opus afterwards, I mean why the hell use ' when you can use backslash or ` as in every common language.

We need a simple /selectfilepath command which works on the entire filename as windows explorer does (explorer.exe /select,) or a way to switch the useless pattern matching syntax off)

Thanks for the guide, unfortunately I use the characters that you pointed out quite a bit so it's probably best for me to stick with the original command.

Have downloaded SearchEverthing portable version and extracted in F:\Everything.

How to make it work with DO ?

thanks

SearchEverything: Getting Opus to work with Everything

[quote="apocalypse"]

Before I start you will need both everything and the Everything Command-line Interface (es.exe) in the same folder (ex. Program Files\Everything). So if you dont have them, get them here.[/quote]

What is the difference of everything and "the Everything Command-line Interface". Sorry for the question apocalypse, cause I am a new user of Directory Opus... I really need to learn and undergo different tutorials to interact with DOpus features.

Thanks in advance,
JMcGrey

One is a GUI and the other is a CLI.

One is a GUI and the other is a CLI.[/quote]

I still don't get it! Can you refer me a thread which start the topic of GUI and CLI in this forum, but i will try to read the post in the link you provided. Thanks