DO12 - Serious memory leak after prolonged operation - beta12.15

Here's a screenshot of process hacker's dopus stats approx 27hrs since restarting it the last time.

You can see the amount data transferred as well. Before the restart it's private memory reached ~7.6 GB in less than two days but I didn't make a screenshot as I thought it would be a one time thing.

Just for comparison DO11 was idling around less than a gig after weeks of uptime and heavy usage.

Simply refreshing the same folder with 30 items inside multiple times I can see the memory usage grow 5mb on each refresh and they're not being released.

Just to be sure I removed all columns except filename and it is still growing at the exact same rate.

Disabling 7 used wildcard label filters made it stop growing on refresh.
ReEnabling them made it start growing again.
Attached colorgroups and foldercolors data files.
foldercolors.rar (614 Bytes)
colorgroups.rar (614 Bytes)

Please follow How to find components causing memory leaks and post a screenshot of the Call Stack window (step 9) if it is able to locate where the leak seems to be coming from.

Another thing to check: Close all toolbars and see if it still happens. (If that fixes it, I have a feeling this may be something we've already fixed internally, but the VMMap stack trace could still be useful as confirmation.)

As an aside, the data transferred (I/O) values are likely just going to indicate the amount of file data you have copied using Opus. If you copy 70GB of data, it's going to show up in that column, and doesn't indicate anything wrong (unless you haven't copied 70GB of data or close to it, or done anything else that would cause that amount of writing to disk).

I never implied the disk usage was a factor. I only used it to demonstrate normal usage of dopus.

I did the VMMap traces with the latest vmmap version. It ran out of memory and crashed the first time around so I had to reduce the working sample of dopus' leak by hiting refresh on a dir and watching private memory grow from 60 to 150 MB

Dopus at that moment in time

Trace for dopus

Call stack for Top Item

Call stack for Second top item

State of VMMap at that point

I see no suspicious dll. Everything belongs to microsoft's and has valid image signatures.

Another test: Single lister with one tab after fresh start - no toolbars.

Increase happens with defined and enabled label filters (Alt+R).
I determined the grow in memory is directly proportional to file count in the current lister, the label filter complexity and label count ie. the more files there are in the folder, the more steps there are in each filter and the more filters are enabled - the faster memory increases on each refresh.

The increase does not happen when only using wildcard filters (Alt+C) but they are useless to me as they apply the parent label to all children elements inside as it seems they match on the whole path instead on just the filename.

I have to note that the leak happens regardless if we have matches on those filters as long as they're enabled.

Do you get the same leak with the label filters disabled, and the Description column enabled instead?

The stack trace suggests the leak is due to a video file; if your label filters are querying file metadata, it may be that you have an invalid video file that the AVI library is having trouble parsing.

You could also try disabling the Movie plugin in Preferences -> Viewer -> Plugins and see if it has any effect.

Jon, you're right. Disabling the Movie plugin did seemingly fix it for now. What's strange is that in the folders i tested there were no media files whatsoever so it's strange that it was even loaded.

Now I can't help but wonder what the relation is to Label filters. My filters only match on filenames so unless dopus reads the whole metadata before being asked what is needed for the filter I don't see how the two things can relate in this way.

On a side note I only use potplayer for media playback and have no 3rd party video filters or codec packs installed. Windows is a fresh 10 anniversary install (not upgrade).

Looks like I spoke too soon. Anyway I think I found the culprit.
I was testing with 30 empty folders some of them containing matches.

Removing the additional matches on type (match FILE or FOLDER) in the label filters made them work and not leak memory any more. For the exact filters I used previously see my 3rd post.

Type should not require reading metadata or headers tho. No description columns or anything of that sort in the lister either.

Some plugins check file types by inspecting the files, so it's not out of the question that a type check could cause the Movie plugin to open some files.

Metadata can also be obtained for a lot of things you might not think of, e.g. the status bar if it is showing Duration.

The underlying issue may be with a video codec that is leaking memory, or it could be something in the movie plugin itself.

Saying this, the movie plugin would not be involved at all if you are literally only refreshing a folder containing nothing but 30 other empty folders. The movie plugin would only be used on files. So something else seems likely to be going on which is not yet understood.

If you do that test with empty folders, and without opening any other Opus windows, do you get the same results in VMMap?

Test folder:

Stack trace (@100mb):

Label settings:

When I disable the Godlike label memory usage stops growing on refresh.
Re-enabling it makes it all start again.
Growth rate is 1 ~ 1.5mb per refresh.

Statusbar has only - files / folders / hidden files / bytes and the default stuff - no metadata fields

You're right, there was a leak involving Type clauses when filtering. We've tracked it down and will have it fixed in the next update.

Many thanks for the great details/reporting and also for testing the earlier theories.

You're quite welcome Leo. I'm glad we managed to single it out and that it's getting fixed. Keep up all the great work.