From this statement, I assume you are in Windows 10, because honestly in 2025, there are no situation where having a .jpg with metadata changes that is not reliable. Even Windows 11 is rotating .jpg images using orientation metadata.
Do you have a concrete example with tools that are up to date in 2025, where it is not reliable?
I am not convinced with this approach: this is the same as recommending to some one to use only plain text files instead of .docx files because Word doc are not always reliable. Basically that's true, but it should not fully stop you to use .docx files when you need them.
The file size change is not just moderate, it can be massive (Rotate images using linked viewer - #9 by Kris) so we can have strong doubts about the lossless behavior.