This weekend I was using Everything's Opened-By column (I had barely used it before). While the data itself is fast, there are some considerations that make it not a great candidate:
The data is somewhat inaccurate. Or rather, it seems to go through some kind of filtering, as if only certain opened files are shown.
Keeping it constantly enabled, at least in my case, makes Everything unstable from time to time. I'm not entirely sure what's causing it.
Sometimes the results get kind of distorted. For example, the same process can be listed for the same file like 10 times, things like that.
This python helper got me thinking too. There are things you could do with python that you currently can't achieve with pure Opus scripting, and with PowerShell it's too slow.
I put into practice what I mentioned at the beginning: instead of querying each file individually, (when an unknown number of files require values) it's better to get the list of all open files once, and then just check whether those files are in that list.
This obviously has many advantages and some drawbacks:
It works very fast. The time doesn't increase exponentially with the number of files, since the list is only retrieved once anyway. In theory it could even be improved further by caching the data, but that would probably increase the chances of getting incorrect results.
It aligns with how user columns work in Opus. So with these you can do searches, filtering, and any process where Opus requires them.
To get a more complete list, elevated privileges are needed. While python can make this easier, if you enable this and have UAC turned on, you'll get a UAC prompt every time the column is used, which can be a bit annoying.
I was also looking into how to close handles without terminating the process. Basically, a full-on locksmith but built in Opus. My results are a bit mixed, but I think I'm making progress. I'd say the new test version is pretty usable, it can close most handles. There are a few things I'd like to improve, for example only prompting for elevation once per run (right now it asks every time it needs to close handles that aren't accessible with the current permissions), or being able to "politely" ask programs to close first before killing them.
You shouldn't do that. It is possible using Sysinternals Handle. But it causes unpredictable damage to the process that uses this handle. It is much better to terminate the process. Because once you forcefully close the handle, this process is likely going to crash.
instead of querying each file individually, (when an unknown number of files require values) it's better to get the list of all open files once, and then just check whether those files are in that list.
Yes, that is what I'm trying to do. And this is why it is so hard. There is a bug in WinApi that doesn't allow you to enumerate all handles in the system. You have to use a kernel driver. That is why I needed to reverse engineer ProcExp152.sys driver that Sysinternals Handle uses to provide information about handles.
My idea was:
Create a Windows Service that uses ProcExp152.sys to provide a list of all locked files
DOpus communicates with this Windows service via a socket. It gets the list of all locked files every 5 seconds and caches it
The JScript will request "who locks this file?" and it will get an answer from the cache. This is instantaneous.
No admin rights are needed because the Windows Service is run as an admin.
The only problem with this is that the information about handles is a bit outdated. But there is no way around it. I have a very fast PC, and getting info about all locked files takes 3 seconds. It depends on how many apps you have opened. The more handles and process the slower it gets.
Yes, I have been thinking about something like a helper function library for dopus using python, that could be used by many different scripts, specifically a fuzzy search engine, or more relevant to this topic, perhaps something like batch file handling/processing for columns etc. The modules would need to be pretty general/robust, but its just an idea at this point.
Even if it was possible to populate all columns at once, you are going to run Python interpreter on every folder opening.
I think a better way would be to implement it as a C++ COM class and add to DOpusScriptingExtensions for example.
The same issue with MediaInfo and ExifTool. Without C++ COM, you have to execute it from scratch on every folder opening.
I agree that would be ideal but I wonder if there are other less painful ways of getting decent speed like a persistent helper process + good caching.
Apparently, there is a WinHttp.WinHttpRequest COM class that you can use from JScript to do HTTP requests. Thus, you can run a local Python web server that will expose the functionality that you need.
True.In fact, I hadn't used any similar programs until you brought the topic up. Then I saw that LockHunter offered something like that and I became curious to replicate it.
You should use common sense to decide which process to do that to, IMO. The same applies when terminating processes.
For example, for a media file, if you unlock it, it doesn't crash the player; it simply stops playing it.
But does it really affect meaningful files? I mean, there are a lot of system files listed there that I don't think you'll ever need to interact with.
The script I made also cannot list files opened by programs opened as SYSTEM, even with elevated permissions; perhaps that's why you would need to run a service, which could. But in actual, real-life use, honestly I'm not so sure this makes a big difference.
It's quite fast too, taking about a second. Obviously, this will depend on how many and which processes the machine has running. In my case, it usually gives you about 5000 files on average.
I'd definitely like to test whatever you go out with. If installs a service, it would probably be best if it were separate from DOpusScriptingExtensions.
I might also upload my own new version, depending on whether there's enough interest. I'd like to see how it works on other machines. You need to have a working python installation accessible via PATH.
It's not just the process that had the lock that you might cause problems with, either. Especially if the handle you're closing isn't read-only.
If the handle value is recycled (so now it represents a completely different file) and the process that still has the "unlocked" handle then goes on to use it, it will be reading or writing a random file now.
How is more supposed to open the file? I mean, when you use it, you can see that the file is indeed locked (you can't delete it).
LockHunter can't see the lock! It tells you the file isn't locked.
My script sees it, but can't unlock it because it doesn't report any handles. Process Explorer also doesn't show any handles for the process related to the file, but it lists it in the DLLs tab.
You should use common sense to decide which process to do that to, IMO. The same applies when terminating processes.
Yes, of course. That is why you need to see what app uses this file first before you decide to terminate it.
But does it really affect meaningful files? I mean, there are a lot of system files listed there that I don't think you'll ever need to interact with.
WinApi works like this:
You call a method "Give me all handles in the system."
You iterate through handles and try to get information about this handle:
Is it a file? <-- BUG: This action can hang if this handle is a pipe (basically, it means it hangs always because some process always uses pipes). Thus, you are stuck. There is no way you can unstuck. Your thread is dead at this point. File Locksmith kills the process and continues the iteration in a new process. But this makes iteration very slow, and the code becomes quite complex. You also can't do it in a long-running process because killing a thread has unpredictable collateral damage.
What file path does it have?
If it is a handle from an elevated process, you fail to get information about the handle and can continue iteration.
LockHunter can't see the lock! It tells you the file isn't locked.
If the file is shown in the DLL tab, more somehow opens it as a module, and LockHunter doesn't look through the modules of processes. I assume it memory maps it instead of opening it. Thus, Windows considers it a module, not an open file.
I checked with AI, it says the DLL tab is actually DLLs and memory-mapped files.
Yes, I also think that. A lot of people depend on DOpusScriptingExtensions. I'm afraid of introducing a breaking change in a new version. I will create a new project.
So in your opinion, those types of programs shouldn't have that feature as it can cause issues?
Perhaps only offer terminate processes in a "nicer" way?
Or simply be for information-only?
I'm asking to see which features to follow with this new script.
Same as PolarGoose. I wouldn't personally put in an unlocker, but it's your choice. They are very occasionally the right tool for a job, but there's usually a better one and I think people are too used to using the worst method without thinking.
For example, if the aim is to move, rename or delete a file that is locked, and you can't close the process locking it, then you can tell Windows to perform that operation on the next reboot. (MoveFileEx is the API at the Win32 level. There are some command-line tools that wrap it which may be useful for calling from a script.)
That solves the problem if it isn't too frequent, and if it is happening frequently (and with things you can't close or restart) then the real problem to solve is probably elsewhere.
OTOH, being able to find out what is locking a file is really useful!
I'm in a abusive relationships with C++. It causes me a lot of pain but I still continue using it. I waited whole night for GitHub to build my code, and when I woke up I found that the build job was terminated because it couldn't build Google RPC library within 6 hours. What? Oh and if your repository name is too long, C++ will refuse to build too, because it doesn't like long path on Windows (that is true). What is it, 1980s?
How bad is GitHub? The author of the Zig language found this issue: safe_sleep.sh rarely hangs indefinitely
It reminds me of a phrase: "There is someone much less competent than you living your dream". Someone at Microsoft wrote this code. Next time you doubt if you are a good SW dev, remember this.
It is actually worse. Not only some person from Microsoft wrote it. Another person from Microsoft reviewed it. And several other people from Microsoft looked at the pull request that fixed this issue and rejected it.
