Go to Preferences / Viewer / Plugins, select the ActiveX + Preview + Office + Web plugin, click Configure, then in the Preview Handlers section at the top of the list, enable Windows Media Player.
That will change the viewer used for those files to one which is click-to-play.
Back in Preferences, you may also want to configure the Movie plugin and turn off its Automatically play movies option, if it is enabled and handling some of the movie types. Or disable the Movie plugin entirely if you want to use the Windows Media Player preview handler for movies as well.
Using the Windows Media Player preview handler should give you the same behavior as you see in File Explorer's viewer pane.
Edit -- Newer and more detailed answer: