12.3.3 beta occasionally crashes when opening a new window

I get the occasional crash when opening a new window with 12.3.3.

In %temp%\Dopus.Minidumps, there are no dump files with recent time stamps. Last one is from Nov 2016 and it is zero bytes. But I had it crash today and yesterday.

What data can I provide in order to trace down the cause of these crashes?

DOpus just crashed while I was using it and it did create a minidump file this time. Attached.

Previously it was dying upon opening a new window. This time it was while I was working in it. I don't know exactly what I was doing when it crashed.
dopus.20170118.114021.zip (40.1 KB)

Looks like something is corrupting memory. It could be Opus itself but it's usually a shell extension installed by something else. Unfortunately, with this kind of crash the dump usually only shows what fell over the mess, not what created the mess.

Crash, exit or high CPU when viewing certain directories has suggestsions. In particular, the part on Shell Extensions is worth following.

The dump shows these extensions still loaded at the time of the crash, but it's always possible the one that caused the problem was unloaded before things actually crashed, so these are just the first ones to try if you want to avoid disabling all of them at the first stage:

  • Dropbox (DropboxExt64.8.0.dll)
  • MSOffice (msoshext.dll)
  • Autodesk (several DLLs)
  • SpiderOakOne (shell_extension.dll, and lots of Python DLLs)
  • TortoiseSVN

The time it crashed while I was running it, the SpiderOak ONE login page popped up. (I was previously not logged into SOO and I do not save my login to it.) Maybe it was SpiderOak ONE. I'll check when I get some time for this.

28 March 2017 Update: It's related to TortoiseSVN, not SpiderOak ONE.

Using Shell Extensions Manager (shexview.exe), I narrowed this down to TortoiseSVN's shell extensions. (It has several and I didn't reduce it further than the entire set.)

I'm using TortoiseSVN 1.9.5.27581 (64 bit) in Windows 10 with Directory Opus 12.4 build 6288. But I also had this problem in 12.3 betas.

Other than finding out which TortoiseSVN extension in its set is the culprit, I don't know how to narrow this further. But I don't want to spend time on that if it's not useful.

I have Opus opening a lot of local and network folders on startup. Ten in the left lister; 9 in the right lister. I could figure out if this crash is related to one of those 19 folders, but I don't really want to screw up my carefully set up default folder tabs. Would it be useful for me to reduce this?

I'm pretty sure I started getting this Opus startup crash when I upgraded to Opus 12 from Opus 11. Around that time, I watched a "what's new video" and so started using the Status column to display TortoiseSVN status icons and disabling TortoiseSVN's file icon overlays and I also started auto-opening about 19 network and folder tabs by default.

I disabled the TSVN stuff I found in preferences (see pic below of disabled TSVN settings that didn't solve the crash).

I rebooted after any settings changes or after disabling any shell extensions. Religiously.

Frequency of the crash: After a reboot, my first run of Opus often crashes. If the first run doesn't crash, I close the dual-lister window with its [x] button and re-run it. By about the fourth run this way, it will crash. A few times it took 11 runs or so.

Once it crashes twice in one session of Windows, the crash becomes hard to reproduce until I reboot.

What do you suggest, Leo?

We use TortoiseSVN a lot ourselves and have not run into any problems lately.

Of course, there may be a problem that is only triggered with certain combinations of actions or files. But, especially with what looks like a memory corruption issue, it's possible TortoiseSVN is not the real cause, but it being loaded moves other modules around such that things are lined up in the right way for the crash to happen.

Do you see the same crash if you disable most shell extensions except TortoiseSVN?

If you have the Windows debugging tools installed (specifically, gflags.exe), I can suggest some tests you can do which may locate the problem more exactly. You can switch into a mode where Windows does vigorous testing of the memory heaps after each operation, and will detect problems (usually) the moment they happen, instead of the moment something slips over the banana left by something else a few moments later. The process runs slower, but it means you can generate a crash dump which may point to the exact cause.

The debugging tools are an optional install, but if you're using TortoiseSVN for Windows development then you may already have them. If so, here are the commands, in case they are useful.

They need to be run from an Administrator command prompt.

  • Turn on PageHeap debugging, the next time dopus.exe is launched:

    "C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\gflags.exe" -p /enable dopus.exe /full /protect
    

    After turning it on, fully exiting Opus (via File / Exit Directory Opus), then re-launch it and see if you can trigger the crash. If you can, collect the crash dump as before.

    Opus will run much slower while it is on.

  • Turn off PageHeap debugging and return to normal, the next time dopus.exe is launched:

    "C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\gflags.exe" -p /disable dopus.exe
    

    As with turning it on, fully exit Opus, then re-launch it. Speed should return to normal.

Using shexview.exe, I disabled all extensions except those from Microsoft (and of course rebooted). The crash was no longer reproducible. I then re-enabled TortoiseSVN's group, rebooted, and was able to reproduce the crash starting Opus. I then disabled TortoiseSVN's group, rebooted, and was no longer able to reproduce the crash. Lucky for me, the TortoiseSVN impact seems pretty simple — that is, it doesn't require other shell extensions or weird combinations of factors in order to reproduce it.

Leo,

I followed your steps and reproduced the crash. But no minidump was created in "%temp%\DOpus.Minidumps". Note: I ran Opus via my normal shortcut, not from within my Administrator command prompt. Is that alright?

PageHeap was definitely on:

C:\docs>"C:\Program Files (x86)\Windows Kits\10\Debuggers\x64\gflags.exe" -p /enable dopus.exe /full /protect
path: SOFTWARE\Microsoft\Windows NT\CurrentVersion\Image File Execution Options
dopus.exe: page heap enabled

Thanks for trying PageHeap.

Running via the normal shortcut was the right thing to do, confirmed.

If no minidump is created when PageHeap is on then it's a dead end, unless you have Visual Studio or another debugger installed and can use that when the crash happens (via the "debug" option in the standard Windows crash dialog, if a debugger is installed). If you do have them, then clicking debug and looking at the stack trace, and which DLLs it mentions, should tell us which DLL(s) are involved in the memory corruption. It may confirm that Tortoise SVN is causing it, or may point to additional components to investigate.

But if you don't have that available, I think your other tests have shown it probably is Tortoise SVN, as much as it surprises me (since we use it ourselves), so leaving it disabled may be your best bet.

PS: Don't forget to disable PageHeap and restart Opus when done debugging, as having PageHeap on will slow things down a great deal.

On crashing when opening a new lister today, Opus (12.5) finally created a minidump file. Yay! Does this help you determine the cause, Leo?

Thanks!dopus.20170516.095540.zip (38.6 KB)

Please try the new 12.5.1 beta which should be in the banner at the top of the page. That will almost certainly fix the crash you're seeing.

(Of course, if it doesn't, please let us know and we'll investigate further.)

Thanks, Leo. I upgraded to 12.5.1 and rebooted twice. The occasional crash is still present. No apparent change.

To reproduce it every time, I can just switch to flat view in a lister.

Note: Opus hasn't been creating minidump files for these crashes. Yesterday's minidump-generating crash perhaps had a different cause than my abiding crash.

Thanks for trying 12.5.1. Looks like what you're seeing was unrelated to the issue others were seeing that we fixed. Sorry for wasting time there.

The crash dump from earlier shows another memory corruption, so it's consistent with what you were seeing before, and I'm guessing it's just semi-random which piece of code trips over the corrupted memory heap after whatever is causing the corruption has created the problem. That would explain why the crash dump is generated some times and not others.

For what it's worth, the code in the crash dump is to do with loading the saved Manual Sort order for a folder. But unless you get the same problem when entering the same folder, the problem probably isn't in that code; it's just the thing happened to trip over the memory heap corruption.

Does disabling Tortoise SVN's shell extension still stop the problem happening for you now as before?

I wonder if there is a particular sequence of actions, or configuration or similar, in Opus or TSVN which triggers the problem. It's still not something we've seen ourselves or had other reports of, and we use TSVN with Opus all the time, but we may not being doing the thing that triggers the issue.

It's also strange that PageHeap doesn't make the problem show itself, as it should make the process crash and generate a minidump the moment the corruption happens. But that might be because PageHeap slows things down so much that the timing of events changes, and the problem may only be triggered if things happen as just the right (or wrong) times.

Alright, I think this crash is now solved. I can't give full credit to 12.5.1 as I think I actually had two different crashes that I thought were the same.

Thanks for your help, Leo. You can stop reading here, if you like. I'm going to detail what I did today in case either of my two crashes come back.

I'm running Opus 12.5.1.

Previously, I would get crashes as follows:

1: Every once in a while opening a new Opus window. Would crash a couple times a day.
2: Every single time I press my [Flat] button, regardless of what directory is active when I do it -- local or network and I tried various directories. The button does this: Set FLATVIEW=On,Grouped. 100% reliable crash.

Prior to today, I had assumed both crashes had the same (unknown) cause but were linked to TortoiseSVN. But when I isolated crash (1) to TortoiseSVN, I suspect that I hadn't yet run into crash (2). I believe that I later discovered crash (2) and just assumed it was the same problem as crash (1), just reproducible every time.

Now that I upgraded to 12.5.1 from 12.5.0, I think crash (1) is gone. My earlier comment said it still existed after my 12.5.1 upgrade five hours ago, but I can't reproduce it now anyway. I can only reproduce crash (2) now. Okay, so what's the cause of crash (2)? This gets slightly weird.

I uninstalled TortoiseSVN and TortoisGit today, then rebooted. I could still reproduce crash (2) afterwards. More evidence that crash (2) is not crash (1), as if I needed more evidence.

So maybe Opus 12.5.1 fixed crash (1). I'm gonna say that's probably the case. And maybe crash (1) didn't produce a minidump until 12.5.0. Okay. Crash (1) does seem to be gone. Good riddance.

Now we have crash (2). TortoiseSVN and TortoiseGit are uninstalled at this point. So I checked other shell extensions using the wonderful shexview.exe. All listed shell extensions were enabled at this point. Zero disabled. I now disabled every shell extension except those from Microsoft. Didn't reboot, just tried to reproduce crash (2) and couldn't reproduce it. Rebooted, still couldn't reproduce it. Re-enabled every disabled shell extension 50% at a time (enable half, try to reproduce the crash, repeat on the remaining disabled extensions). Can't reproduce crash (2). Tried a reboot once I had every shell extension re-enabled. Can't reproduce crash (2) again. So it seems that disabling every non-Microsoft shell extension (they were all enabled before I started debugging this today) fixed crash (2). And re-enabling them didn't (or hasn't yet) re-introduced crash (2).

I reinstalled TortoiseSVN and TortoiseGit (I'm lazily omitting version numbers here, though I do know which versions I _un_installed) and then rebooted (though I wasn't prompted to reboot). Still can't reproduce crash (2). That's it. That's my report. Kinda weird. I hope neither crash recurs and I'm very glad to be able to use [Flat] again.

If you read this far, thanks again for all your help, Leo.

P.S. Somewhere in those steps for isolating crash (2), I wanted to know if 12.5.0 would still produce crash (1), so I downgraded from Opus 12.5.1 to 12.5.0. Not certain where in my steps I did this downgrade, but it was early on. With 12.5.0, it crashed on first run after reboot after the 12.5.1 to 12.5.0 install (therefore presumably crash [1]) and created a minidump. I decided crash (1) was indeed fixed in 12.5.1 based on this and upgraded back to 12.5.1, wherein I have not yet reproduced a crash on run so I assume crash (1) is truly fixed.

Opus is still not crashing today, nine days later. Yay!

1 Like