It turns out Photoshop writes not just RGBA into the preview bitmap but all of the alpha channels.
On the plus side, it's easy to make this particular example work.
On the down side, this has revealed a problem with interpreting channels beyond RGB in PSD files. There is no way to know that the 4th channel in the file is actually opacity data. If it is some other kind of alpha channel, the results will be wrong when it's interpreted as opacity.
You can see this already if you put a solid background layer behind your test image, so it has no non-opaque pixels, and re-save it, while keeping the alpha channel. You then get a file with the 3 RGB channels + 1 extra channel that isn't opacity (in this case it it's inverted opacity, but it could be any 8bpp data). That channel is interpreted as opacity, since it's 4th in the list. Since it's inverted opacity in this case, we end up with the whole foreground of the image cut out, and the only thing visible is the background forming a silhouette around where the foreground used to be.
We'll probably add a psd_assume_alpha option similar to the tiff_assume_alpha one for people whose workflows involve images with no opacity channel and some other kind of alpha channel. That will have to be in 12.2 though.