I found options to set a path field to left, right, focus etc., but not for what to use that.
Can you have one big that works for both panes?
A path field added to a normal toolbar will do that without any extra arguments.
(Ones in the File Display Toolbar(s) always act on the file display they are above, but ones in normal toolbars act on whichever side is currently active.)
How do you remove the default path field then? Can you remove the field of one of the panes and leave the other?
You can edit those toolbars the same as any others, if you want to remove the path field from them but keep the other parts.
Or, if you want to remove the whole file display toolbar, that can be done via Preferences / File Displays / Border.
I see. You always keep a minimal border though? And you can't have the path just for the destination as far as I can see.
There's always some kind of border if you're in dual-display mode. It's optional in single-display mode.