High memory and CPU usage during VS2010 unit tests

Could you post a screenshot of your Opus setup, making all the file display columns visible if possible.

BTW, did you try with the folder tree turned off before, or was it still on when you had just one window pointing at an empty dir?

All tests done today were with Opus looking roughly as it does in the screenshot. I haven't tried the 'minimal' approach again since last night.


Did you ever try with the folder trees turned off?

Presumably the unit tests & related files are all on the local drive(s), not the network drives?

Yes, last night. I had a single-view lister pointing at an empty folder. The problem still occurred.

Yup, everything is in the folder shown on the left, which is a local harddrive.

Have you tried uninstalling Tortoise SVN to test whether or not it is involved?

No, not yet. I'll try that now and report back.

No, no change - I'm still getting the problem with Tortoise removed. I don't have any other shell-extensions installed that I'm aware of.

You can use ShellExView to see which other shell extensions you have and if disabling any helps, although it would be unusual for many shell extensions to have an effect if change-notification was turned off in Opus, unless they're something that monitors the folder/network by themselves.

There's nothing of note there - just a selection of context-menu extensions and thumbnail-generators.

The only really interesting one is probably Avast. Are you aware of any interactions between that and Opus?

I'm not aware of any problems between Opus and Avast, but it's possible with any antivirus tool.

Right :slight_smile: I'll lock my system down and turn Avast off and see what that does.

Incidentally, I mentioned earlier in the thread that I thought the problem might only occur once per session; that is, if I kill and restart Opus and continue with my unit-tests, I won't see any further issues. This seems to be borne out by today's testing - after my report four posts ago, I have restarted Opus (but not restarted my PC) and run more unit-tests and so far Opus' memory usage is sitting around 20MB, which is normal for my system. I will continue testing for a while before turning Avast off.

Nope, disabling Avast makes no difference either :frowning:

I have noticed something else, though: the increase in memory-usage does not occur all in one go. Opus seems to be grabbing memory in approximately 2.2GB chunks. Each time it grabs, it runs one CPU core up to 100% until it's allocated about 1.1GB, stays at 100% for a minute or so, allocates another 1.1GB, and then the CPU returns to normal and Opus responds normally. Only when it reaches the physical RAM limit (16GB) does it never recover; this is usually the point at which I've spotted the problem, since of course the entire system slows drastically due to memory-starvation and I start looking for the culprit :slight_smile:

At present, Opus is sitting on around 2.2GB. I run another set of unit-tests and Opus runs the CPU up to 100%, grabs 1.1GB, spins the CPU for a minute (during which Opus is completely unresponsive), eats another 1.1GB and finally returns. It now has roughly 4.5GB (including the memory it was using prior to the problem starting up). If I do a few more unit-test runs then Opus will grab all the remaining free memory and sit there chewing CPU cycles until I kill it via TaskManager.

As Opus is still responding, I've just quit it; all 4.5GB has been freed up as expected. On restart it's using around 25MB, as per normal. Further unit-test runs have so far failed to trigger the problem, so it seems clear that this only occurs after the initial start of Opus on bootup.

It seems like a very unusual problem...

#1 Maybe this guide to tracking down memory leaks will help.

#2 If you set Opus not to run on startup, then launch it later on after everything has finished booting up, do you still see the problem?

#3 A third thing to try: Does disabling Explorer Replacement make any difference?

(Thanks for trying all these different things, and sorry we're having to try wild guesses to work out what's going on.)

[quote="leo"]It seems like a very unusual problem...

#1 Maybe this guide to tracking down memory leaks will help.
[/quote]

Yes, I need to try that out soon.

My very thoughts :slight_smile: I shall be trying those next...

[quote]
(Thanks for trying all these different things, and sorry we're having to try wild guesses to work out what's going on.)[/quote]

No problem :slight_smile: I do this stuff for a living, I know how difficult it can be to diagnose a problem by remote!

Still no joy :frowning: Disabling start-on-bootup and Explorer-replacement makes no difference.

Is there any way we could try to re-create what you're seeing (or something similar) locally?

e.g. An example project we can load into VS2010 and run the tests on. (Doesn't have to be the real thing, of course. A made-up test project will do, as long as it still triggers the problem for you so that we can try the same on our machines.)

I can certainly let you have the project - it'll be open-sourced once complete anyway - and that's probably the best way to ensure that we're running the same thing. It's a good few hundred MB, though - are you Ok with downloading that much? :slight_smile:

Looking at the output from VMMap, the allocations are occurring in blocks of 16192kB. The only dlls involved are Windows ones.

Excellent, that seems like the best way forward. If we can reproduce it then we can look at it much more directly and save you having to try more of my increasingly crazy ideas. :slight_smile:

(Of course, it's possible that the behaviour is tied to a 3rd tool/driver/mystery-object only on your machine, but hopefully not.)

A few hundred MB is fine. Drop me a PM with the details and I'll grab the file. (If you need somewhere to store it, we can probably set up a temporary FTP login or something.)

Righty, that's great, thanks! :slight_smile: I'll get everything packaged up and uploaded to my site, and let you know how to get at it later this evening.

Something has just shown up; I haven't confirmed this yet, but: it appears that the problem will always occur once per session - even if I have exited and restarted Opus before running any unit-tests. Once it's happened and I've restarted Opus, everything will be fine until the next reboot of the system. Up until now it had appeared that it only happened on the first run-up for Opus, but now it seems this is not so.