As per Subject. This already happens on other edit controls (eg. path bar), but can you make it happen for the New Folder edit box, and any other edit controls where it doesn't already happen (eg. Rename dialog: Old/new name), Script editor, where appropriate (eg. not on Filter edit/drop-down).
I think the path bar does it because it's an auto-complete/drop-down control, where the up and down keys move you between other matches (or effectively insert the current string over the top of itself if there are no other matches).
It's not a standard behaviour in normal edit controls, where you typically have to use Home and End to go to the start or end of a line.