When viewing folders, back only 2 levels?

I've noticed when I click through multiple folder levels, say 5 deep, that when I click the back button, it only goes "back" 2 levels, then does nothing. Is there a setting for this, I'd like to be able to back to the original folder if possible.

Preferences / Favorites and Recent / Recent List -> Lister History. Sounds like it has been set to 2.

Hey Thanks! I KNEW there had to be a setting for this as well.