Windows Confidential
Forcing Handles Closed
Raymond Chen
Have you ever wanted to delete a file but couldn't because some program had the file open? The correct thing to do in this situation would be to convince the program that has the file open to close it—you could, for instance, ask the person editing the document to close it. But sometimes you get impatient and use some utility that can force file handles closed. And then you've traded temporary relief for long-term data corruption.
Forcing a handle closed is equivalent to reaching into a program and freeing some memory. The program thinks the handle (or memory) is still valid and will continue to use it. But since the handle is really free, it will be reused for something else.
Say you are in high school and you see a book in locker number 305 that you want. You force the locker open and take the book. Now the locker is open and somebody else comes by and puts his own book bag into it. Meanwhile, the original owner of locker 305 returns from lunch, grabs the book bag, and takes it to class.
Now you have two problems: The original user of locker 305 has the wrong book bag and is in danger of taking notes in somebody else's notebook. The second person, meanwhile, just had his book bag stolen.
If you're extremely lucky, the next thing the program will do is close the handle you forced to close earlier. The program gets an invalid handle error, but since there's nothing it can do, the program just proceeds as if the handle closed successfully. But this is rarely the case.
More likely, the program will eventually come back to the handle and try to use it, getting invalid handle errors. Even more likely is that the handle you forced closed will be used for something else. And that's when things start to go completely berserk. Suppose the handle were reused as another file. Now the program thinks it's reading and writing from the first file, but in fact it's operating on the second file. The result is data corruption in the first file (data that should have been written wasn't) and in the second file (data intended for the first file was written to the second file).
When the program closes the first file, it also closes the recycled handle. You've created a cascade error: You forced the first file closed, and the result is that some second handle was forced closed.
This data corruption can continue indefinitely, as each component that requests a handle has that handle inadvertently corrupted by the previous component. When the previous component finally closes what it thinks is a valid handle, it actually closes the handle used by the current component, which now proceeds to corrupt the next component that requests a handle.
Suppose a search index service has a file open for indexing but has gotten stuck temporarily and you want to delete the file, so you (unwisely) force the handle closed. The search index service opens its log file in order to record some information, and the handle to the deleted file is recycled as the handle to the log file. The stuck operation finally completes, and the search index service finally gets around to closing that handle it had open, but it ends up unwittingly closing the log file handle.
The search index service opens another file, say a configuration file for writing so it can update some persistent state. The handle for the log file gets recycled as the handle for the configuration file. The search index service wants to log some information, so it writes to its log file. Unfortunately, the log file handle was closed and the handle reused for its configuration file. The logged information goes into the configuration file, corrupting it.
Meanwhile, another handle you forced closed was reused as a mutex handle, which is used to help prevent data from being corrupted. When the original file handle is closed, the mutex handle is closed and the protections against data corruption are lost. The longer the service runs, the more corrupted its indexes become. Eventually, somebody notices the index is returning incorrect results. And when you try to restart the service, it fails because its configuration files have been corrupted.
You report the problem to the company that makes the search index service and they determine that the index has been corrupted, the log file has mysteriously stopped logging, and the configuration file was overwritten with garbage. Some poor technician is assigned the hopeless task of figuring out why the service corrupts its indexes and configuration files, unaware that the source of the corruption is that you forced a handle closed.
Raymond Chen's Web site, "The Old New Thing," and identically titled book (Addison-Wesley, 2007) deal with Windows history, Win32 programming, and misreading newspaper headlines.
