I'm back with some more info.
I managed to slow down dopus to a crawl with mere 40 tabs.
Saved all listers to a layout
Closed all listers and tabs except for one
Suddenly DOpus became fast and responsive.
Reopened saved lister layout
Slow and lagging again.
Closed all tabs pointing to network paths
Saved all remaining tabs pointing to local disks as a lister layout.
Closed all remaining tabs except one.
DOpus is now fast and responsive.
Reopened saved local tabs layout again.
DOpus starts lagging.
Closing all tabs until the following remain:
2 Split listers - 2 local tabs each.
1 lister - 1 tab.
All pointing to folders on C drive (SSD).
Disabled all Script Addins
Restarting dopus does not seem to fix anything.
Seems to be some notification overload or a threading bottleneck.
Tab count is killing it.
ProcMon is showing dopus accessing filesystems and folders that are NOT OPEN as tabs.
CreateFile, CloseFile, QueryFile.
Even when restarted with minimal tabs.
Those files are not being changed so there's no change notification going on them triggerring dopus.
Full system reboot fixes it for a while. Everything is going extremely fast.
The only thing left that comes to mind is remote desktop hooking into apps to make them slower for some reason and not unloading those hooks properly when back in front of the PC.
Eventually I'll try downgrading to 12.6 and 12.5x next to see if there's any difference.
That might be normal if it's reacting to filesystem change events and looking at the folders to see if any junctions point to them (which could mean folders you do have open need updating). Or it could be something else.
Is it possible to send us a PML log file from Process Monitor, and tell us which events to look at? The stack trace information it includes will let us see which code is causing the access (and also if it is part of Opus or something else).
After using the computer for a few days away from work I think this thing is related to remote desktop somehow. I suspect that RDP does not unload the hooks(?) it uses properly when reconnecting to a dropped remote session and/or resuming the session locally. Thus the behavior deteriorates more and more with each session reconnect until the next reboot. Somehow dopus seems to be more affected by this compared to other programs. I'm using RDP again today so we'll see how it goes.
This may be unrelated but I've seen lower directory opus responsiveness ever since the Intel Spectre/Meltdown patches. Nothing that can be done about that though, it's just something we have to live with.
I reverted back to 12.7, it is still going fast even though it has been on for a day and 21 hours. When I was on the beta (12.7.1), I could see signs of slow down after a day and a half. I will keep monitoring it.
I haven't experienced the slowdown recently. I suppose it's because of lower dopus usage on my part these days and as such I can understand how hard it would be to reproduce the heavy duty stuff I've put it through before that.
The only things that changed since then are:
I've upgrading my nVidia video driver - seems an unlikely cause considering I only skipped one driver version so far.
I keep only one lister with 10-15 tabs on each side, occasionally opening another for some temporary work
This seems to be enough for preventing dopus from going haywire.
About the slowdown:
The slowdown was occurring in between moving around hundreds of gigabytes of data among both local and remote disks and using dopus heavily for days, sometimes weeks.
I must also stress that the slowdown wasn't caused by a shortage of resources during file operations and persisted even after all of those were finished.
Listers and tabs were accumulating and eventually I was having more than 3 listers open with more than 20 tabs in total for each on both sides.
Procmon didn't show anything out of the ordinary
They happened both in 12.7 and 12.7.1 and as such could be unrelated to GQJ17's problem.
Just as other users reported, I've noticed severe slowdowns manifesting visually on the UI since the Intel Spectre/Meltdown patches, especially when SVN operations are involved. I've noticed that heavy disk usage causes mouse freeze/lag which was previously really rare.
If the mouse pointer is lagging, that's a very low-level issue, outside of anything that could involve Opus (other than that perhaps Opus is using the disk or drawing on the screen, which is tipping the system over the limit; but to be so close to the limit would mean something is wrong at a low level, i.e. drivers/hardware).
Windows 10 has a habit of replacing good drivers with old or incorrect ones, which is worth checking. Motherboard/chipset/storage drivers would be my guess here, as well as the usual GPU and mouse ones.
Guess I spoke too soon but at least I have some definitive results this time. The problem looks to be your plain old file-system notification flood. It seems dopus can't handle huge quantities of those efficiently and as such everything slows down while it's being hammered.
Debugging them doesn't show any dropped events and it looks like dopus is slowing itself down by giving too high priority to them and catching them all.
While browsing within a folder or even a drive - which has many events generated for it - is expected to become at least somewhat slower, this does not excuse all other tabs and listers, representing other drives, as well as the entire GUI to become sluggish and less responsive because of them.
Shutting down all programs doing disk access makes dopus go back to normal even without restarting it but the UI animations still feel a bit choppy.
This seems to be an especially severe problem for Windows 10 as it constantly does some stupid thing in the background generating notifications left and right adding to all those triggered by the user. And thats not even taking in the flood from those sloppy programmed crapware drivers and helper programs from all the hardware manufacturers.
I also noticed dopus is acting on events triggered for paths for all open tabs in all listers all of the time and when those tabs become too many it fails acting on those notifications either randomly or sometimes all notifications stop working completely. Only handling notifications for both active tabs (left and right) for every lister at the same time and only hooking up other tabs when they become the active ones seems like a more efficient approach in this situation. It's either that or background tab hibernation like some web browsers do it.
Which version of Opus are you using? Recent updates have some improvements to the change notification code for that kind of thing. I think we have some more on the way as well.
At the same time, there will always be a non-zero cost to processing filesystem changes, and the cost will always increase the more tabs you have open, so having 60+ tabs sitting around will always have some performance impact if there's a flood of change events. But it'll be better with recent updates.
We can't really "hibernate" tabs, unless you want a situation where either they completely miss changes that happened while they hibernated, or they start using huge amounts of memory buffering up all the changes while they hibernate and then use lots of CPU and delay you the moment you activate them. Neither seems that good.
If you aren't using a tab for a long time, it's best to close it rather than let them pile up. If you're using tabs as bookmarks, you might want to consider making some toolbar buttons to go to commonly used folders and use those instead of tabs (or as well as tabs).
Edit: I guess a hibernated tab could re-read the whole folder when you activate it. That might be worth doing.
Latest beta as always - 22.214.171.12493
And indeed I meant just rereading the folder instead of monitoring it all the time while it isn't "on top".
Also as I mentioned earlier last time this happened there was only a single lister with 10 tabs on one side and 15 on the other so none of that 60+ stuff.
Also the weirdest thing is that I don't remember experiencing this before 12.6.9. Could be my memory. Could be windows' fault too.
Using the new beta and it does seem to handle notifications nicely.
On the exact same day I installed it however - the slowdown happened again and at my wits end, I found this article.
Ever since the so-called Fall Creators Update of Windows 10 (version 1709), Adobe Lightroom Classic CC has been suffering from what I believe to be the GDI GetPixel() bug
The symptom is that UI performance degrades gradually as Windows remains up, and only a reboot of Windows restores correct performance. Switching modes and tools (in Programs) results in very slow screen repaint after a few days of Windows uptime.
For anyone having this issue (probably everyone on Windows 10), this update seems to fix it:
After manually installing the mentioned KB4058258 as winupdate refused to offer it to me by itself - so far all is good.
I'm positive this same bug was the one affecting dopus as like I mentioned before - the toolbars were gradually painted, lister redrawing was slow and erratic, focusing - unfocusing, and even filtering preferences made the results appear sequentially with visible lag between them and not instantly like on a fresh restart.
As it turns out the problem was on a lot larger scale than DOpus and I feel bad for making conclusions.
Fall creators update really did a number on us - until this update - hopefully.
To check if you have the update installed:
Open RUN (WIN+R)
Type winver, hit enter
Look at the version string - it should read Version 1709 (OS Build 16299.214) or higher
If it's lower check windows update or install the KB update above.
If it's equal or higher and you're still experiencing the slowdown - then it seems we're not in the clear yet.
Interesting. My PC auto-installed that two days ago, and while I didn't have any issues in Opus, it appears to have fixed a problem I had in another app where its toolbar took about 30 seconds to draw.
It probably depended on how you have your toolbar set to look (e.g. background images).
Chalk it up to yet another Windows 10 bug in the end. At least Microsoft fixed this one.
Never thought I had images on them until now - I noticed the toolbars using Standard toolbar image was checked. Looks like that is the default for new toolbars - or at least was at some point in time. Turned them off and they look exactly the same with or without it
In my case I noticed exteremely long startup times (10+ seconds) of a Video Splitter app which this update fixed so here's hoping for a long uptime
In dopus' case it wasn't the startup - at first it runs flawlessly and gradually gets worse with longer system up-time so most people might never notice any slowdown at all.
They probably weren't using a background image if it didn't look like it. Standard toolbar image refers to one of the image slots under Preferences / Display / Images, but if no image is defined then it doesn't mean much. It lets you define the image in one place and get it on all your toolbars without having to edit each one individually.
I guess we can only speculate about what triggered the Windows bug and why some people saw it at different times. For me, one app did it all the time on one PC, and sometimes on another. Perhaps it's triggered by an interaction between apps.
I'm glad Microsoft have fixed it, whatever it was.