If there are pic files with names of h8 and h80 typing h8 selects h80 but doesn't select h8 also doesn't work in thumbs mode,these files are in random order not sorted by type or ext and manual sort is enabled.fayt firstchar repeat is set as true.
I'd expect it to either select h8 or select the one which is first in the list the way you have it sorted, depending on settings.
You say it's in "random order"; is h80 before h8 in the list?
Typing "h8" will match "h80" if it comes first in the list, and then you can press F3 to jump to the next match.
In addition to what Jon says above, you can also turn on Preferences / File Displays / FAYT and Filter Bar Options / Prioritize shorter filenames.