It depends... Opus hooks into so many 3rd party components that it's very difficult to say how much memory it should be using as it will vary by system.
One example of 3rd party components which can allocate memory within dopus.exe is shell extensions, such as for file context menus. I encountered one which had such a bad bug that right-clicking certain files caused it to allocate a gig of memory. (More details.) Video codecs and other viewer-related things are another example and can be used even when you don't display files since they can be called upon to extract metadata (e.g. video duration or dimensions).
Configuration can also change how much memory is used. For example, if you have Opus set to use lots of large background images or thousands of different icon files then those all take up a bit of memory, though usually only when windows are open.
Fragmentation can cause memory usage to go up over time as well, and different versions of Windows can reclaim or use memory differently... 64-bit processes also tend to use a lot more memory than 32-bit ones (possibly because all the pointers are twice as large).
If you were on 32-bit Windows then I'd say 51MB was quite high but on 64-bit it doesn't sound high enough to worry about. If you are worried, though, then many of the steps in the FAQ on general slowdown or instability investigation steps are worth a try. Many of those steps involve disabling 3rd party code (such as shell extensions and other plugins) which Opus can call.