If you set Preferences / Miscellaneous / Advanced: no_external_change_notify = True and click OK (then tell Opus to restart when prompted), does this still happen?
If that doesn't make a difference, what happens if you disable (clear the checkboxes of) everything under Preferences / Favorites and Recent / Label Assignments, then restart the program?
So far neither Jon nor myself have been able to reproduce what you're seeing, and I've been trying again today without luck, but I have a couple of ideas which each of those tests should check, to see if they're worth investigating in more detail.
That's really hard to say. At first I thought it was definitely better, when settings this to true. But this might have been a coincidence. It's different each time. Sometimes there's a massive tearing effect and sometimes it's only at times. I tried it now 3 times in a row with the flag enabled and disabled. I would say it's slightly better when this is set true but this could also be completely coincidence.
This doesn't seem to have any effect. The parsing is faster, because files doesn't need to be checked for their bit depth, so the period where Dopus is reading all the stuff is shorter in general. As a result, the tearing ends earlier, but I think that's just because the parsing will be finished earlier.
Yeah, I already guessed that this is not an easy one to reproduce.
What might help is, if you create a similar setup I have:
Auto-Start dopus with a configuration with several tabs opened.
Some of the tabs should contain large picture libraries. (Several thousand files in different formats)
Start dopus and activate the tab, while the folders content is still read.
All other settings I've tried so far doesn't seem to have an reproducable impact on this. The only setting that elimates the problem for sure is when I remove the >{picdepth} tag from the TilesMode setup in the file types configuration. When I remove this tag, everything looks perfectly. No tearing, no redraw, no double and triple updates. But that tag is so darn useful;)
Just let me know if you want my configuration or if I can support you with any other info on this.
Just to clarify, the bug we're looking for from the video is where a file keeps changing what it displays, as if the information is invalidated and recalculated endlessly.
It's normal for the information for a file to appear in stages and move around slightly as the information is calculated. (In particular, when the status icons are calculated, that causes the other information to move up slightly, even if no icons are visible in the end, as the icon height is larger than the font height.) That part is OK/normal, but the information should eventually settle down and remain static once all the different parts are calculated.
Calculating bit depth is usually very fast, so if the filter labels are taking a long time to calculate but only based on that, it's potentially interesting. It looks like there are more filter labels involved, though, e.g. one which says files were changed today can be seen in the video.
I've send you my config via mail, so you can check all the attached conditions for further investigations.
I think we're talking about the difference between using the {picdepth} tag or not? Otherwise I would be confused:) When I use this tag, I get all the strange effects you can see in the video. If not, these effects are not there. Normal file updating is so fast and hardly visible, I wouldn't mention it at all.
If you're interested I can make a video without using the {picdepth} tag, but then you will see ... well basically nothing:) There is a very small 'flickering' when items are updated, but this is a one time event and as you've said that's probably the normal file refresh and I wouldn't complain because of that. But it's really a difference like like day and night.
The refreshing doesn't last endlessly, it's while all directories are read and thumbnails are created. (I have the thumbnail cache disabled, so thumbnails are generated each time I start Opus) This usually takes 2-3 minutes after dopus was started on my machine, because several thousand files have to be parsed and the tearing happens during that phase. After that point, it also happens when something was changed in a directory or when I scroll through large directories.
I've double checked both modes again and discovered some more fine differences, that might be relevant or not, but I want to mention them though:
When I quit and restart dopus and wait until everything was read and cached (the mentioned 2-3 minutes, when there is no more update indicator visible in the breadcrumb bar): When I now activate a preloaded tab for the first time, in "non {picdepth} mode" everything is rock solid and stable, you can see no updates and no tearing. When I activate a preloaded tab for the first time in "{picdepth} mode", there is often some kind of initial refresh operation running. This implies the same effects you can see in the video, although they stop usually after 2-3 seconds.
When scrolling through directories in "{picdepth} mode", the same tearing effects happen, although they usually stop after a few seconds. This also doesn't occur in "non {picdepth} mode".
In "{picdepth} mode" sometimes tabs that doesn't even contain thumbnail-files (excel files, or stuff like that) are refreshed again over and over during the preload phase, although there contents have already been updated.
I've also checked the time dopus needs to parse the data with and without using the >{picdepth} tag. It's more or less the same time, so this was a false alert, please ignore this.
This is really not easy to describe, because the effects vary each time. Sometimes it's really annoying, sometimes it's only a little bit ugly:) But I've double checked that this is definitely a huge difference to the normale file update you've mentioned.
I hope that helps. Let me know if you have more questions or if you need anything else.
Both videos have been created after Dopus was freshly started and left idle for 3 minutes, so all data could be parsed.
Although I'm not exactly doing the same stuff, you should clearly see the difference.
Especially when initially activating a pre loaded tab, in the Disabled version you see no refreshes at all, while the Enabled version is full of refreshes and updates, although the frequency differs.
I've spent now countless days to improve the situation, but unfortunately couldn't ressolve this by myself. What I've tried so far:
Disabling windows defender
Uninstalling the corel thumbnail preview
Uninstalling other tools where I assumed a connection
Disabling windows file indexing
Disabling security and virus software, even switching to different virus software
Updating drivers
Playing around with dozens of windows option I thought they could be relevant in any way
Playing around with hundreds of Dopus option I thought they could be relevant
None of that ressolved the issue, but at least by now I have some additional insights:
The tearing just comes from the fact, that when no bit depth is available, Dopus skips the entire line in the tile. This way, the tile display switches between 3 and 4 lines very often which seems a bit like tearing. In fact it isn't and you probably already noted this:)
To reduce the busy redraw behaviour, I adjusted the tile display, so that 4 lines are always displayed, regardless if bitdepth is available or not. This change has improved the situation and I could face the core of the problem: That bitdepth is set and cleared multiple times within a single refresh operation. Sometimes up to 3-5 times. I don't think this is intended behaviour?
After recognizing that, I tried your suggestions again and this time it became clear that setting no_external_change_notify to TRUE ressolves the issue of multiple updates within one refresh operation. I didn't realize that at first, because it still was updated once and due to my previous setting this still created an effect similar to tearing as descrived above.
So, setting no_external_change_notify to TRUE ressolves the issue with multiple updates, yes. But it has a huge downside, I don't get any other automatic file updates any more:/
Is there anything I can do to ressolve this without loosing auto update for all my files?
If no_external_change_notify stops it from happening then it's likely a property handler or thumbnail extractor are modifying the files (or at least their timestamps, or opening them for writing and then closing them) when asked to extract information from them. That's bad, since it means reading the files causes them to appear to have changed, which causes things to re-read them, and you get an endless cycle.
ShellExView can tell you all the shell extensions which are installed, which will include the property handlers and thumbnail extractors. Disabling any that look related to the file formats in question may be worth a try, to see if a particular component is causing the problem. If it is, it should be easy for the authors to fix, and they probably just don't know their code is doing it at the moment. (Although they'll need to be willing as well. One of Microsoft's own shell extensions for emails has this exact same problem and people have been complaining to them about it for years with no fix.)
As adviced, I started disabling all icon and image relating app and system stuff with the ShellExView tool. I was trying to do this one by one and testing at the same time, but ended up by disabling everything, including all MS windows and shell32.dll extensions, although the tool showed some warnings and didn't recommend this.
It was definitely disabled, the right click context menu was almost empty, beside some standard Dopus entries.
However, it didn't help at all. Even with all shell extensions disabled, this still happens. Do you have any other idea?
Are you sure this isn't a dopus issue? Could you reproduce the issue by using the config I've send you via mail?
You could use Process Monitor to see if the files are being modified, which would confirm that theory (it's always possible it's something else). If they are being modified, the Process Monitor log will record a stack dump which we should be able to see to see which process and DLL is causing it.
As adviced I used the ProcessMonitor to track down the issue even further. Whoa, that's a lot of info this tool is generating
I already filtered by file activity and only the time range when the refresh was active, but it's a million events for just a few seconds though.
From what I see it's mainly dopus that is writing in the files or tries to create (overwrite?) the files that are parsed. But I guess it makes more sense if you take a look at that log.
Should I send this in native ProcessMonitor format? Via forum or email?
The log file shows change events seem to be coming in for some files in the folder, but didn't give any solid indication of what was triggering them.
A couple of things to try:
Disable the Movie plugin. (Preferences / Viewer / Plugins.) That's the only thing I can see opening files for writing, but it's not actually writing for them. (It's just checking if anything else has them open for writing, and closes them immediately. Shouldn't trigger notifications, and wouldn't explain why they're happening for non-video files. Still, worth ruling out its involvement.)
Uninstall TSVN, as you suggested in your email. I'd be surprised if it is involved, but it is accessing the folder at the same time, and it is something that can trigger shell change notification events, so it's not completely impossible. Worth ruling out, at least. Be sure to reboot after uninstalling it, as it won't be fully unloaded until then.
Enabling the Debugging steps near the bottom of here and seeing what they indicate may prove useful. Knowing more about what the change events are may help understand where they are coming from.
The only other thing I noticed from the log is that the folder is below C:\ but some of the low-level function calls look related to network drives. It's possible that is normal and just something I've never noticed before (maybe Windows routes everything through a generic function, for certain operations). On the other hand, if there's anything unusual like a junction or symlink that is pointing the folder or a sub-folder to another drive or a network drive, then that might be an important detail.
FWIW, I tried again with your config but still can't get it to happen here. The information in the tiles moves up and down once when the extra info is populated, which is normal, but it does not endlessly/repeatedly refresh for any of the files.
Thank you so much for your detailed help, much appreciated!
I already tried disabling all viewer plugins in earlier tests, cause me too thought it could be related in the refresh issue. Unfortunately, it didn't help.
I tried it now. Uninstalled TSVN completely and rebooted. No difference, same issue.
I've created a logfile. Because logs are very dopus related you'll probably see much more than I do. Hopefully there's something there that helps. I've send it to you by mail.
Regarding junctions and links: I wasn't aware of any junctions or links in this folder, but you never know. So I used the Dopus find file function and searched for filetype: "Junction and Links". No hits.
I don't have any network drives nor shared folders on this machine. However it's a delevopers machine and there are many Sony tools installed. They're mainly for supporting the connected PlayStation DevKits, but now you mention it, there is one tool that installed a low level file driver. This allows browsing memory cards like any other directory. I think it's based on the dokan file system library.
I couldn't see that this is involved in any previous logs, but if you think it might be worth to give it a try, I'll uninstall this library too, just to make sure that this isn't the cause: dokan-dev.github.io/
Besides VS 2013 is installed, which is also some kind of software octopus:) I think installing this suite alone installs up to 15-20 additional projects and components like SQL-servers, .net stuff and so on. But still, it shouldn't cause such issues and I think you guys are working with VS too.
Thanks anyway for taking the time to check and for all your help. Hopefully you'll see something helpful in the log.
Nothing from the quick look I had a while ago. It's still in my queue for a more detailed look, but fixing the forum has taken up most of my recent time.
Whatever is going on looks like something unusual triggered by something external to Opus which keeps generating events telling it the files have changed. Looking at any tools & shell extensions involved in the file formats being changed might be worth doing while waiting for me to get back to it, if it's causing problems.
Could you please explain what steps are required for that? I'm not sure of what you're talking exactly. Thanks.
In the meantime, I tried a different approach, but not sure if it makes sense, so please let me know, if it's nonsense
Just took the logfile from DebugView and deleted all lines that are not a file/pathname. Then I sorted all lines of the file. In the last step, I was looking for duplicate lines.
The outcome was, that there are 73 duplicate lines from ~2300 lines. Could this be involved in the multi refresh triggering? Or am I completely wrong?
You could use ShellExView and disable anything with Type =
Column Handler
Icon Handler
Icon Overlay Handler
MetaData
Property Handler
Thumbnail
Thumbnail handler
Uninstalling software that's installed and related to the file types having problems is also worth a try. Especially anything that has shell extensions (in the ShellExView list) or which monitors folders to maintain a media library or a search index/cache.
Similar with asset management or source control software, which might scan files/folders and can generate change events to signal their their source control icons need updating.
From the log it looks like the problem is happening on C:, so not likely to involve a NAS or other type of network drive (which sometimes have issues with spurious change events triggered by reading files), but if anything exotic is involved, such as junctions/symlinks that point to files/folders on other drives than C then those are worth investigating/mentioning.
From another look at the Process Monitor logs, and filtering dopus.exe out to see what else is going on (since the problem seems to be another program generating "this file has changed" events for lots of files), a couple of things look worth looking at:
TSVNCache.exe from TortoiseSVN is scanning some of the files. It can also generate events to tell programs to update their icons. Maybe something is going wrong and causing it to repeatedly generate events. Try uninstalling it to see if the problem remains with it no longer there.
A service is running which accesses C:\ProgramData\BullGuard\LogData\BsMainLog.db a lot. Looks related to an antivirus product. It might be wroth a try to uninstall it and see if it's involved.
The Windows Search Indexer is doing its usual stuff. Turning off indexing might be wroth a try, in case it is the trigger. (If it is, it's likely in conjunction with something else that has installed an extension, but first just see if it's involved at all.)
None is definitely involved, but they are worth ruling out.
Be sure to reboot after disabling or uninstalling anything as the DLLs involved tend to stay around until the reboot.
I'm aware you have to handle hundreds of issues here, so you can't keep track of everything, but we already went through all the topics you have suggested. (Feel free to check earlier posts in this thread)
Especially SVN, antivirus software and windows search index service has been disabled with no change. (Reboots have also be performed)
Is the hint with the duplicate lines from the DebugView log of any use for you?
[quote="Skeeve, post:18, topic:24033"]
As adviced, I started disabling all icon and image relating app and system stuff with the ShellExView tool. I was trying to do this one by one and testing at the same time, but ended up by disabling everything, including all MS windows and shell32.dll extensions, although the tool showed some warnings and didn't recommend this.
It was definitely disabled, the right click context menu was almost empty, beside some standard Dopus entries.
However, it didn't help at all. Even with all shell extensions disabled, this still happens.[/quote]
