Backup Error 5: Access is denied - Reinstall did not help

The fix for what this thread is about was only released into 10.5.1.1 beta (and the subsequent 10.5.1.2 beta) so it won't be in the earlier 10.5.1.0 stable release.