Synchronization / copying; image conversion tool bug

Sort of a grab bag of issues. No complaints, though. I've been using Directory Opus for twenty years, and right away it was something I couldn't work without. When I needed a synchronization tool, and image conversion, I checked to see if Directory Opus already had them, since it's always been good at covering such needs.

  1. Synchronization seems a bit slow. I have a directory that I frequently back up, about 6GB, ~35K files. The number of files that change at a given time is small. Updating an existing copy is nearly instant using copy with the options to skip identical files and replace nonidentical ones. Doing the same thing using the synchronization tool takes considerably longer, due to the file comparison -- actually, I've never hung around to see how much longer. For really large numbers of files it's something you'd start and take a long walk while the file comparison cranked away. In practice, I only use synchronization when there are files in the destination not in the source. Tools like BeyondCompare are much faster.

  2. The copy operation has useful options for running unattended. You can't set them until you've initiated the copy, however, and if you tick the box that brings up the options window, the copy operation plows ahead regardless, even though you've just told it that you mean to modify its behavior. In other words, as soon as you tick that box, the copy operation should pause. There are obviously ways around this, of course, such as explicitly pausing it first, but that shouldn't be necessary.

  3. Things like progress indicators are never going to work quite right. However, what you see on a copy operation is actively divorced from reality. First off, just showing the number of files copied is only part of the story, and probably the least useful one -- you really want to know how much storage has been copied. That would ideally be covered by the time estimate, which takes into account how fast the copying has been going. However, none of this takes into account the fact that you can modify the behavior of the copy operation to skip existing files, and that you might want to overwrite nonidentical files with the same name. For me, at least, this is in fact the most common case, and the number of files that need to be copied is typically much smaller than the total number of files in the source. Accordingly the progress indicators are useful only in that they tell you how far along you are in the list of source files. As for the time estimate, I've seen it telling me that there are twenty minutes left just before the operation completed.

I don't know whether there's a cheap way to fix that. Ideally you'd want to run through the files before the copy to see how many files will actually need to be copied, instead of just counting the number of files in the source. In the current approach, though, where the options dialog is available only after the copy is started, that would mean introducing a lag after the options had been specified. My guess is that wouldn't be too bad, judging by how fast copying using filtering operations seems to be,but I might be wrong. Ideally for large numbers of files I'd use the synchronization tool, I suppose, but it's just too slow to use unless you need to do more than a simple copy. (And it would be nice if it gave you an ETA, instead of just time elapsed, since it's in a position to make a reasonable estimate, but that's another matter.)

  1. The image conversion tool is useful and works very well. It would be nice if there were some version of it that had some sort of interface, though, since what you end up doing is using the search tool to find image files in subdirectories. That might introduce a complication, I guess, since it's also good to have the simpler usage that currently supports.

In any case, Directory Opus crashes intermittently when I use the image conversion tool. Some cases involved large numbers of files (say a few thousand), so I tried limiting usage to a hundred or two, but it still crashed. (The sort of crash that means business, too -- it not only brought down the lister but killed Directory Opus entirely.) I don't think it's a matter of number of files, though there might be a limit. I was using it with the search tool, as mentioned earlier, but that usually works. If it's crashing because it stumbled over some I/O error, for example, it should be possible to bulltetproof it.

  1. Sometimes Directory Opus will hang. In some cases this is associated with operations on large numbers of files; the lister will freeze and stop responding. It can also be very sluggish after large file operations, but still responsive. My impression has been that it looks like a memory leak. That's purely a guess, though.

General:


  1. I think we have some performance improvements to sync in the pipeline, although doing massive syncs is not really our focus; there are dedicated tools for that kind of job which you can integrate into Opus and which may be worth looking at if you need that type of thing often. Beyond Compare is a good and inexpensive one which we use ourselves, but there are lots of other options, including some free ones.

  2. You can set the options ahead of time via button/hotkey/command arguments.

  3. The progress dialogs can't predict the future. They're good for the normal case, and vast majority of operations, where every file is being copied/moved. In other situations, the totals won't be as good as lots of the files will end up being skipped. But they don't know that until it happens, and working it out in advance would just make things take longer (and could still end up being wrong, since things can change during the copy).

  4. Not sure what you mean there. The image conversion tool does have a UI. Maybe start a new thread with examples of what you have in mind.

  5. For crashes: Automatic crash logs (for bug reports)
    For hangs: Manually generating process snapshots

If you want to follow-up in more detail, please start a separate thread for each thing.