Refresh to remember position

Try turning this on:

Preferences / File Displays / Folder Expansion / Remember expanded folders when refreshing

As well as keeping expanded folders expanded, that also causes the selections and scroll position to be preserved when refreshing.