Coincidentally I saw this for the first time yesterday, while I was debugging a separate OneDrive problem. I actually don't think it's anything the new beta will help with unfortunately. It seems to be a bug in OneDrive itself - sometimes, for some reason, the OneDrive DLL starts returning the wrong status to us for all files.
Obviously Explorer uses a different mechanism to query for file status but unless we can find out what that is there doesn't seem to be anything we can do about this in the short term.
I had it happen twice; the first time it resolved itself after a reboot, the second time I was able to fix it by a complete uninstall of OneDrive (including deleting the folder, as you did, but also deleting the OneDrive data directory in %LocalAppData%).
I followed the instructions here, for what it's worth: