Prevent looping symlinks/junctions/hard links in search

Hi!
This is about symptoms I've seen before: Searching in %USERPROFILE% sometimes results in search getting bugged down in what seems like just an endless loop of nested references for Application Data:


I don't remember what the circumstances were when I first saw this a long time ago, but in this case I'm searching through a mounted .vhdx system disk image.

I'm trying to exclude this by using location no match "*\Application Data\Application Data\*", but to no avail.

I think after quite a while, search may finally fall out of the loop, presumably when some limit is reached, but I'd like to avoid this all together.

What can I do?
Possible bug report/feature request about recognizing a self-referencing nested link?

It should detect that kind of loop, and does here.

(Those junctions usually block direct access as well; are you running with UAC turned off or similar? It should detect the loops either way, but the way the junctions are permissioned by default makes loops even less likely to occur.)

That also isn't the best way to restrict search depth or exclude a folder. See How to filter items by location or sub-folder

You mean, using the "subfolder" clause instead?
Yes, that was the forst thing I tried.

Yes, I have UAC disabled.
I just turned it to "always notify" (highest level), it still happens.
It also happens with my actual system disk as well, not just the mounted image, it turns out.

"Subfolder" is the only thing (other than the automatic loop detection, and permissions blocking access) which will prevent folders being entered. The other clauses, including "Location" from your screenshot, will filter results out but won't prevent recursion into folders.

Just to confirm, even after a reboot? And Opus wasn't running as Admin, correct? (It would say "Administrator" in the titlebar if it was elevated under UAC. Make sure that isn't there when testing with UAC enabled.)

Are you sure it's really looping and not just taking a long time?

Which version of Opus are you using?

Ah, I see

Oh of course! That old chestnut :man_facepalming:
I was in Administrator mode. That definitely has an impact; restarting it without elevation seems to have made it skip this loop.

Would there be legitimate cases where an ever increasing nesting of "Searching in: H:\Users\Ben\AppDate\Local\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data\Application Data"... would appear?
No, I guess I can't be sure it's actually a loop/circular reference, it just... looks like it to the naked eye ^^

Is it normal/accepted that the loop detection fails in Administrator mode?
If so, then I guess we've found the culprit ^^

Oh, I thought it was 12.24, but because I'm in the middle of working with/between older system images, I must've restored an older version, I'm on 12.23 again for the moment.

It shouldn't fail, but maybe something's not working there. I've made a note to take a look next week and see if I can reproduce what you're seeing with UAC turned off.

Thanks for checking! I don't think we changed how anything related works between 12.23 and 12.24. I'll post again once I've had a look in more detail, since it seems something is wrong from what you're seeing.

I can't reproduce this, even with UAC disabled.

Opus checks for recursive junctions, and doing a search of C:\, or %USERPROFILE%, and below completes normally without getting stuck in a loop.

I enabled some debugging code we have for this and it's working as I'd expect:

It might be worth looking at your filesystem to see if those deeply nested folders really exist. Something may have actually created them. (e.g. When copying the profile from an old machine to another or restoring a backup or similar, a tool that wasn't aware of junctions may have created the nested folders for real until a limit like path length was reached.)

Currently on a fresh Windows installation, the safeguards work even with Admin mode and UAC disabled, as you described :+1:
Testing again on the old system image (just mounted as a virtual hard drive), it gets into those loops again.
I had taken a look, but all I saw was a junction (and I don't know enough about how that whole mess is supposed to work to spot any inconsistency)

Now I've had a closer look:
The mounted file system's junctions (currently assigned H:) are still pointing to C: :man_facepalming:
I thought maybe they were defined relative to the root drive, but either they're defined relative to the system drive (thus creating problems when they're mounted to a different system), or hard-coded to a drive letter (probably unlikely).

The junction in question for my system drive C as reported by NTFSLinksView:

Name              : Application Data
Full Path         : C:\Users\Ben\AppData\Local\Application Data
Type              : Junction
Target Path       : C:\Users\Ben\AppData\Local
Created Time      : 26/06/2021 06:05:16

...and the same junction for the old virtual HD mounted as H: on my current system:

Name              : Application Data
Full Path         : H:\Users\Ben\AppData\Local\Application Data
Type              : Junction
Target Path       : C:\Users\Ben\AppData\Local
Created Time      : 25/05/2021 12:23:47

Could this interfere with the recursion detection somehow?

Wait, I think there might be some weirdness happening between some combinations of the variables here.
I'm collecting data.

In the mean time: Can I enable a verbose debugging log for search?

That still shouldn't result in a loop. It'd just search the folder on C as well as the one on H.

There's isn't a way to enable the same debugging I was using; it has to be compiled into the exe file.