Disabling the Microsoft entries via ShellExView would do something similar and might be worth a try in that case.
If it isn't folders with exe files in then you can probably ignore that part.
Other than those things, I'm not sure why reading directories would be slower in one program than the other. The folder listing ultimately comes from Windows.
What sort of speeds do you see in other software? For example, the dir command from a command prompt, on the same drive/path?
There is a fairly exhaustive list of things that may affect speed here: General slowdown or instability investigation steps