Opus 12.6 - File list not updating for local/network/ftp folders

Leo,

Thanks for the quick reply! I had rebooted the computer (Windows Server 2008 R2) after the Opus upgrade but I will do that again tonight. I recall seeing this in earlier versions and I think I even recall that we did some debugging on it as the article you suggested sounds very familiar as I read it. Unfortunately, I have no recollection of what the ultimate resolution was. Hopefully, the reboot will sort it out.

edit...

I should add that the settings referenced in the article were already set as suggested. The problem is effectively new for us with Opus 12 as it was pretty much, if not entirely, non-existent with Opus 11 on the same server. We pay a great deal of attention to what's happening to the files during our normal work so if it was happening in Opus 11 I'd have been hearing about it.

Again, thanks for the quick reply. I have a server reboot scheduled for 3 am tonight and hopefully it will sort it out.

Jason

The reboot had no effect on the problem, unfortunately.

Here's some events which I triggered myself and which show up in DebugView but which Opus didn't show until I refreshed the directory.

Just so it's not lost, I'll mention again that the problem is intermittent.

00189649	9:48:25 AM	[10856] [7584] dopus: Change modified on R:\sastemp	
00201695	9:48:36 AM	[10856] [7584] dopus: Change added    on R:\sastemp\_TD28508_XENAPP01_	
00201698	9:48:36 AM	[10856] [7584] dopus: Change modified on R:\sastemp	
00202190	9:48:36 AM	[10856] [7584] dopus: Change added    on R:\sastemp\_TD28508_XENAPP01_\SASMONO.FOT	
00202191	9:48:36 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00202194	9:48:36 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_\SASMONO.FOT	
00202199	9:48:36 AM	[10856] [7584] dopus: Change added    on R:\sastemp\_TD28508_XENAPP01_\SASMONOB.FOT	
00202200	9:48:36 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00202203	9:48:36 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_\SASMONOB.FOT	
00203237	9:48:37 AM	[10856] [7584] dopus: Change added    on R:\sastemp\_TD28508_XENAPP01_\TEMPXXXXMRJ.MRJ28508	
00203238	9:48:37 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00203239	9:48:37 AM	[10856] [7584] dopus: Change removed  on R:\sastemp\_TD28508_XENAPP01_\TEMPXXXXMRJ.MRJ28508	
00203244	9:48:37 AM	[10856] [7584] dopus: Change added    on R:\sastemp\_TD28508_XENAPP01_\TEMPXXXXMRJ.MRJ28508	
00203247	9:48:37 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00203250	9:48:37 AM	[10856] [7584] dopus: Change removed  on R:\sastemp\_TD28508_XENAPP01_\TEMPXXXXMRJ.MRJ28508	
00203253	9:48:37 AM	[10856] [7584] dopus: Change added    on R:\sastemp\_TD28508_XENAPP01_\TEMPXXXXMRJ.MRJ28508	
00203256	9:48:37 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00203258	9:48:37 AM	[10856] [7584] dopus: Change removed  on R:\sastemp\_TD28508_XENAPP01_\TEMPXXXXMRJ.MRJ28508	
00212245	9:48:45 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00212249	9:48:45 AM	[10856] [7584] dopus: Change added    on R:\sastemp\_TD28508_XENAPP01_\TEMPXXXXMRJ.MRJ28508	
00212252	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00212255	9:48:46 AM	[10856] [7584] dopus: Change removed  on R:\sastemp\_TD28508_XENAPP01_\TEMPXXXXMRJ.MRJ28508	
00212256	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00212259	9:48:46 AM	[10856] [7584] dopus: Change added    on R:\sastemp\_TD28508_XENAPP01_\dd1.sas7bdat.lck	
00212262	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00212265	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_\dd1.sas7bdat.lck	
00212268	9:48:46 AM	[10856] [7584] dopus: Change ren_old  on R:\sastemp\_TD28508_XENAPP01_\dd1.sas7bdat.lck	
00212269	9:48:46 AM	[10856] [7584] dopus: Change ren_new  on R:\sastemp\_TD28508_XENAPP01_\dd1.sas7bdat	
00212272	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00212273	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_\dd1.sas7bdat	
00212279	9:48:46 AM	[10856] [7584] dopus: Change added    on R:\sastemp\_TD28508_XENAPP01_\TEMPXXXXMRJ.MRJ28508	
00212282	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00212284	9:48:46 AM	[10856] [7584] dopus: Change removed  on R:\sastemp\_TD28508_XENAPP01_\TEMPXXXXMRJ.MRJ28508	
00212581	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00212582	9:48:46 AM	[10856] [7584] dopus: Change added    on R:\sastemp\_TD28508_XENAPP01_\dd2.sas7bdat.lck	
00212583	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00212584	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_\dd2.sas7bdat.lck	
00212585	9:48:46 AM	[10856] [7584] dopus: Change ren_old  on R:\sastemp\_TD28508_XENAPP01_\dd2.sas7bdat.lck	
00212586	9:48:46 AM	[10856] [7584] dopus: Change ren_new  on R:\sastemp\_TD28508_XENAPP01_\dd2.sas7bdat	
00212587	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_	
00212588	9:48:46 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD28508_XENAPP01_\dd2.sas7bdat	
00219636	9:48:52 AM	[10856] [7584] dopus: Change modified on R:\sastemp\_TD22032_XENAPP01_\sastmp-000000009.sas7butl	

I also had my DebugView session because of problems with folder updates. This is quite a painful thing since.. whenever it creeped into DO or my computer or whatever. Explorer.exe does not seem to be affected, so I think remote servers or something are not the reason for this, for me at least.

Yes, Windows Explorer has never, to my knowledge, been affected by this issue. Opus 11 seemed to be fine, too. It's very unfortunate to be experiencing this on our XenApp server where everyone uses Opus all day every day. I'm not seeing it happen on desktops and laptops that I've put Opus 12 on so I wasn't expecting it to be an issue when I upgraded the XenApp server last weekend.

What I was really very surprised at is that it's happening without any apparent regard to the type of device the file changes are happening on. I've seen it on local drives, network drives and even FTP connections.

If this investigation doesn't turn up a solution, I'll probably need to revert back to Opus 11. :frowning_face:

To confirm, when the log was generated, you were looking at R:\sastemp\_TD28508_XENAPP01_ and Opus didn't show the SASMONO.FOT file being created, but then a file with that name appeared after an F5?

While the problem is happening, if you check in Task Manager, is dopus.exe seeing high CPU usage?

It might be worth doing a manually generated dump for the dopus.exe process, during a time when the problem is happening, and emailing that to us. If we look at that it could reveal that the change-processing code is getting stuck (or taking a long time) somewhere, which could be the reason the events are not reaching the file displays.

Leo, thanks again for your prompt attention to this issue.

Yes, all the files including the one you mentioned specifically were not shown in the folder containing them (which I was looking at (in Flat View mode, technically) at the time until I pressed F5 to refresh the list. At that point they showed up immediately.

There is no apparent excess CPU usage of Opus or anything else on the server when this is happening.

A couple of dump files for the Opus.exe process that was running in my session have been sent.

Many thanks for making the dumps.

From looking at them, the dopus.exe process seems to be completely idle, with every thread waiting for something to happen, and none stuck anywhere.

That means it isn't a backlog of events that is stopping new ones from being processed, so we can rule that out.

Assuming no_external_change_notify has not been turned on, I am at a loss as to what might be happening that could cause the problem in all folders.

It is possible that Opus sees the change events (as per the debug log) but does not recognise the folders it is displays as being the same as the ones the events are about. If it was only happening in certain folders or drives then there are things that might be involved, but it's very odd for it to affect everything at once. The only thing I can think of that might cause that is if all the folders were navigated to in a weird way. (e.g. By prefixing the paths with \\?\, but I don't think that would do it as Opus strips that out. Something else along those lines might be a possiblity, and some programs do things like that, if all the tabs were initially opened by another program. But it's unlikely if you've navigated to places by hand.)

Another possibility is that something is blocking some of Opus's window messages which it sends between threads. An antivirus/security tool could do that in theory. If we were talking about messages sent between processes then this would be a likely possibility to look at, but it's pretty rare for things to block messages between threads in the same process.

Another factor could be UAC elevation (running the whole dopus.exe process elevated, instead of using the UAC support built into Opus to elevate individual windows or actions). Using Run As to run Opus as a different user is similar. Both could potentially confuse things by blocking some events or by having drive letters which mean different things in different user contexts, or folders which are accessible in one context but not in another. This would be unusual, but is another possibility. (Essentially, if you are launching dopus.exe with a different account or integrity level to the desktop itself then it can cause issues.)

If none of that applies to your setup then the best thing might be for us to create a version that outputs more verbose debugging, so we can see where the notification events are going and why they aren't reaching or being matched to the windows displaying those folders.

(By the way, it is best to email dumps, especially ones created in Task Manager, as they can include everything that the dopus.exe process has in memory, including files from your computer that it had open. The instructions on making the dump go into a bit more detail, but that's why I removed them from the post above, after downloading them.)

To add to what Leo said - are you sure there aren't any filters active that could be hiding the files?

Change notifications on FTP sites use a totally different mechanism to real file systems, so the fact that you find both aren't working seems very strange to me.

1 Like

Yes, I agree. It's very strange to me that it's happening with ftp as well as local and network files.

I can't think of any way to create a filter which would prevent a deleted file from disappearing from the file list or a renamed file from appearing with the new name. So I think a filter isn't the cause of this.

I don't think any of Leo's possibilities apply, either. The AV product was not changed and was causing no problems with DO 11. The noexternalnotify option isnt enabled, either. We're not running DO as another user or as an admin. We're accessing the paths using network and local drive letters (c:, h:,...).

I guess that leaves a special build.

Thanks for helping with this.

Jason

I think yes, folder updates for me do not work whenever the %G placeholder in the titlebar resolves to {xxxxx-...-yyyy} volume ids instead of the actual UNC path the folder is on. Refreshing the folder fixes that volume id and suddenly the correct path is visible, stepping some folders down and up might also get rid of the wrong path/volume id in the title. But another refresh can bring it back anytime and then the folder updates will not be recognized. That's what I analyzed so far. I did a thread on that recently, but still wanted to throw that detail in here, while you were assuming similar.

+1 o)

Is this with an unusual setup of UNC paths pointing to junctions pointing to mount points pointing to junctions pointing to... etc.? I seem to remember someone here had something like that but not the exact details. That kind of thing will definitely confuse change notification at some point, as the redirects may not be followed to an infinite depth, and may slow things down as well.

(Basic junctions and mount points should work, of course.)

Should I still be anticipating a custom debug build for this issue? I haven't yet heard anything that explains the reason this is happening for me with Opus 12.

The plan is to make one, yes. It will take a little while.

ok, great. thanks. I'll do my best to hang on with Opus 12 until then.

In the event I end up needing to revert back to Opus 11 (due to a user revolt, basically) is there anything special that needs to be done or do I simply uninstall Opus 12 and reinstall Opus 11.

Please understand I have 30+ people using this particular installation of Opus and they are NOT Opus experts. If their Opus profiles will be damaged by a downgrade I need to know that and what I can do to mitigate the problem. I can't find out after the downgrade that I have 30 people who now have the default Opus configuration rather than their normal carefully constructed one.

I do have Zips of their Opus 11 ...\Roaming\GPSoftware\Directory Opus and ...\Local\GPSoftware\Directory Opus folders. I can also have them create a native Opus backup before the downgrade to Opus 11.

Thanks,

Jason

I just observed this error for the second time since Opus 12 was installed. Perhaps it's related to our problem?

image

Was a Crash Dump created?

Ya, sorry...That would have been a good idea but I was in a rush at the time and didn't think to create one. I'll do that if it happens again. Sorry.

Did you read that in a rush too? Your answer does not fit the question. He asked if a dump was created automatically (you have to check the dump folder), not if you manually created one (which you say you did not).

If yes, don't worry, happens, but go find that dump.

Thanks for the pointer

I don't see that it's obvious from his actual text that he was asking about an automated dump having been made. However, I do see that the page he linked to mentions an automated dump being made and one was in fact made. I've emailed it to crashdumps@gpsoft.com.au

Thanks, again