The Grouping tab in the folder format dialog seems to be showing a different list of groups than the main grouping button.
Button groups
Folder Format groups
looks very much like the column header menu
That's the list of columns you can group by; the grouping schemes (if any) are in the separate control underneath the column list.