There are several ways you can set up the columns so they never push others off out of view:
You could also make a button or hotkey which can be clicked to resize the column to a specific size, if you want to keep the behavior as-is the rest of the time.