Add labels to get the labels:
Go DRIVEBUTTONS=fixed,labelsThe Go command help page lists various other things you can add to DRIVEBUTTONS which you may also find useful.
The icons displayed are whatever icons the drives have. There are a few ways you can change a drive's icon. I personally do it via the registry setting made for doing just that. HDD Letter Icons shows how.
If you don't want to change the drives' icons, you could also make individual buttons for some or all of the drives and assign any icons you want to those buttons.