UTF-8 broke during FTP file transfer (not SSH), now forcing ASCII mode

Log

File name was "08 - millstones - ジタンのテーマ~ガーネットのテーマ~ビビのテーマ~エーコのテーマ~クイナのテーマ~フライヤのテーマ~サラマンダーのテーマ~スタイナーのテーマ~ローズ・オブ・メイ.flac"

I had the entire folder uploading, and at the end of transferring the file, it had an error on the next file saying "Could not create the directory or file". And the file that had been uploaded had a corrupt filename (track 08).

I then found out that DOpus had changed to enforcing ASCII mode, instead of UTF-8 mode, and all my albums that had Japanese characters before were now being shown in ascii characters. (Through SSH they are named correctly though)

Seeing the logs, it reverted to a standard character set, and I cannot find a way to fix it back to UTF-8.

Is this a glitch? Is it possibly because the file name/folder path was too long for the server and Directory Opus flunked out?

I think that error is coming from the server. It may not be able to represent that filename on its side for some reason (length, total length including path, or particular characters, possibly).

Closing the connection and opening a new one should get you back into UTF-8 mode, unless the server itself is refusing to turn it back on. But transferring the same file again may trigger the same response from the server again.

A server can enforce that? Yeah, I tried closing DOpus down entirely (exiting from the tray) and it still forces ASCII even without attempting to transfer files.

The server is a virtualized SSH shared server, so I find it odd that this could happen.

Well, it was too long of a filename, which is really odd. It should not be... It seems to be forcing ASCII in Filezilla too, but Filezilla has a "Force UTF-8" mode. Is there a way you could add that option? I tried turning the checkbox into a checkmark in the Misc tab, but it didn't help. Also, perhaps add a way to check if a filename is too long before uploading like Filezilla does?

I don't know if we have any way to know if a filename is too long for the server. That's something the server is doing, and AFAIK the protocol has no way to query length limits. (It will probably also depend on the total length of the paths on the remote system, which is not always knowable as the root FTP folder is usually below several other folders in the real filesystem.)

I'm not sure what the best way to handle a server unilaterally choosing to switch back to ASCII mode is, either. That seems like the server is incorrect to me. A long name should not make a server switch character encodings when the client and server have already agreed what to use, and when there is no way to know it has switched encodings other than to read the English text next to an error code (which will be different on different servers, and cannot be reliably parsed for information, beyond what the error code is, which is not specific to "I just randomly changed character encodings due to an error that wasn't really to do with the encoding anyway".)

IMO you should complain to the server authors that they are switching encoding because of path length limits, which aren't a good reason to switch encoding.

I have been in contact with them, but a force UTF-8 feature would be nice and is possible.

https://wiki.filezilla-project.org/Character_Encoding

Filezilla wiki explains details on how, and clients can enforce UTF-8 instead of attempting to autodetect the encoding.

So actually the real problem is DOPUS's FTP system needs to detect when filenames are too long for the remote filesystem. Windows 10 Anniversary added the default registry tweak to allow longer filenames than 255 bytes, normally the limit of NTFS, and while Filezilla catches this before attempting to upload, DOPus still attempted to upload the file and it broke. It created a rogue file in the home directory of my server (not where I uploaded) and I could not delete it from within DOPus. I had to SSH in and remove it manually.

UTF-8 characters use more bytes than ASCII characters, so the Japanese file I uploaded exceeded the filename limit by far.

Might you be able to look into this? I know you said there isn't a way, but if Filezilla is able to detect it pre-upload (shows a message in the log) there must be a way...

AFAIK there is no way to know how long the remote system allows paths to be, nor how long the path leading up to the FTP root folder is, which you would need to know as well.

Why is the server switching encoding because of a path length? That seems to be the problem here.

I admit, I have no understanding of how FTP servers work. I apologize if I sounded irrational.