Id3 tag data not displaying in details view

Hi Jon and Steje,

here is a screenshot:

and here is one of the files as seen in the screenshot:

http://www.devo.com/Avengers-10-The%20American%20in%20Me.mp3

-Z

Hmm... kind of strange. At first glance, it seemed like Opus was not able to process the v2.4 ID3 tag data of this file. But on further analysis:

a) An apparent difference between Opus and Windows Explorer is that Opus initially tries to show the ID3v2 data while Explorer tries ID3v1 first...
b) For the file in question, it seems like Windows Explorer shows only the v1.1 ID3 data properly. If this info is removed from the file, Explorer chokes on the remaining v2.4 data the same as Opus is apparently doing on your system (but...)
c) Oddly enough, while Explorer choked once the v1.1 data was removed from this file, Opus shows the remaing v2.4 tag data CORRECTLY.

On my system, the values are not BLANK... instead I see a series of question marks. The 'Artist' field for example shows ??? ????????, which is the correct number of characters of the actual value. Maybe this has something to do with running Unicode vs. Ansi Opus versions?

Anyhow, maybe the 'encoding' or 'padding' of some of the original tag data is whacking Opus out? After writing the v1 data BACK to the file, Opus is still working just fine. However, the structure of both the re-written v1.1 and v2.4 tag data is indeed different from the original state of the file. It seems like the rewritten data is padded with null/block characters or something between the v1 tag fields, whereas the original v1 data apparently has just spaces between each field until the very end of the tag data where there is a null/block character... depending on how Opus tries to read tag data - maybe this is causing a string function to blow chunks or something?

Side note- I tried the app that wrote the original file tags... 'MusicBrainz Tagger'. Maybe it isn't 'supposed' to be a normal tag editor or something (it tries to make tagging easy by referencing an online tag database), but if it is I found it to be an un-intuitive POS. I wonder if others having problems with Opus recognizing MP3 tags have files tagged with this app...

Well, never mind about the null characters... that was just in the rewritten v1.1 tag.

It's something in the way the v2.4 data is encoded, every tag editor I try (TagIt, Mp3Tag, TGF) they all re-write the v2 data in a slightly different way than how this original file is encoded... and the only way that Opus can actually read is Ansi. Opus cannot read the UTF-8, UTF-16 (BE), or UTF-16 (LE) encoded v2 data. And the original file info from your sample file seems to be yet ANOTHER sort of encoding...

Ok I've worked out the problem. Whatever program you used to write the ID3v2 tags has written the strings as Unicode but in "big-endian" byte order. While this is legal as far as Unicode goes it's quite uncommon to find this in a Windows program, because all Intel CPUs are natively "little-endian".

(as a brief explanation, "endianness" determines the order the component bytes of a larger data unit (16 or 32 bits) appear. On Intel CPUs, the least-significant byte comes first. This means that the string "Hello" in Unicode (which uses 16 bit values) would be represented as "H.e.l.l.o." on an Intel CPU (where the . represents a 0 byte). On a big-endian CPU, eg Motorola, the same string in Unicode would be represented as ".H.e.l.l.o". See en.wikipedia.org/wiki/Endianness for more info).

Ok, so the actual problem is that Opus doesn't support big-endian Unicode strings in ID3v2 tags. We'll fix this in the next version. In the mean time, maybe try using an ID3 tagger that writes the tags as little-endian (which almost ALL Windows-based taggers would do. Maybe the program you are using was ported from another system? It's very odd to see big-endian used like this under Windows, as they would have had to go out of their way to do it like that).

[quote="jon"]
Ok, so the actual problem is that Opus doesn't support big-endian Unicode strings in ID3v2 tags. We'll fix this in the next version. In the mean time, maybe try using an ID3 tagger that writes the tags as little-endian (which almost ALL Windows-based taggers would do. Maybe the program you are using was ported from another system? It's very odd to see big-endian used like this under Windows, as they would have had to go out of their way to do it like that).[/quote]

Its very cool that you figured it out!!! Thanks. :smiley:

The program I am using now is Tag & Rename (http://www.softpointer.com/tr.htm), I think one of the best taggers out there, as it handles not just ID3 tags, but Vorbis, Ape, WMA, and MP4 tags as well. This is particuallirly import for people, like me, that have alot of FLACs, because although its technically allowed to add ID3 tags to FLACs, the correct thing to do is use Vorbis tags.

Now that you mention it, I also have UTF-16 tags as well. The reason for this is that my MP3 player (a Rio Karma) handles non Latin-1 characters this way.

I look forward to the next OP8 release that handles some more ID3 tag options!

-Z

I just came here to ask about this. Foorbar 2000 0.9 now seems to write tags in "big-endian" as my mp3s are displaying with the question marks as well.

weird that they would change the way it writes the tags. oh well.

Glad you're going to fix this rather than tell me to not use foobar :slight_smile:

I've not upgraded to Foobar2000 0.9 yet but in 0.8 there's an option to write non-Unicode tags to MP3 files. (See first screenshot).

Also beware that, at least in Foobar2000 0.8, the default configuration is to write APEv2 tags to MP3 files, not ID3v2. (See second screenshot, drop-down at the top.)

(I have no idea why APEv2 tags are the default for MP3 since almost nothing supports APEv2 tags on MP3 files and it can cause problems ranging from lack of tags to corrupt music playback. From reading about APEv2 on Wikipedia it seems to offer absolutely no benefits over ID3v2 so it's very strange that it's the default in Foobar2000.)

If you're using Foobar2000 to batch-convert music into MP3 format from something else, it's also worth checking the Diskwriter component's configuration, since this can override the tag format as well. Select Components -> Diskwriter then in the Output presets box click Edit to see what tags the preset uses. Each preset can be different.

(Click screenshots for full-size versions.)



[quote="nudel"]I've not upgraded to Foobar2000 0.9 yet but in 0.8 there's an option to write non-Unicode tags to MP3 files. (See first screenshot).

Also beware that, at least in Foobar2000 0.8, the default configuration is to write APEv2 tags to MP3 files, not ID3v2. (See second screenshot, drop-down at the top.)

(I have no idea why APEv2 tags are the default for MP3 since almost nothing supports APEv2 tags on MP3 files and it can cause problems ranging from lack of tags to corrupt music playback. From reading about APEv2 on Wikipedia it seems to offer absolutely no benefits over ID3v2 so it's very strange that it's the default in Foobar2000.)
[/quote]

0.9 doesn't have options for tagging apart from what sort of tags (id3v1, id3v2 or ape). id3v2 is now native and not via a plugin, though given the lack of options I'd prefer for the plugin to be ported. So there is no choice for the encoding type. I was looking on the foobar forums and one of the developers seemed to be suggesting that they were using little endian encoding with id3v2.3 /me shrugs...

here's an mp3 with id3v1 and v2 tags written by foobar in case anyone is interested in looking at it or testing it...

mexicankitsch.biz/16-Blur-Lot105.mp3

I'm pretty sure they used ape tags originally as they are at the end of the file instead of the beginning, so re-writing the tags was quick rather than having to rewrite the entire file. Now foobar seems to use padding on the id3v2 tags so that isn't an issue anymore.

Well... besides taking the word of the foobar developer at face value, I don't think there's any reason to believe foobar is writing in Big Endian ayway. While I haven't yet looked at v0.9 in depth, I think what is causing the problem is less an issue with just Big Endian format so much as Unicode tags - period.

The tests I ran last night was with the file linked to by Zn0rt. I used the app ID3-TagIT with the following encoding and results:

ANSI = worked ok in Opus
UTF-16 (BE) = no good
UTF-16 (LE) = no good
UTF-8 = no good

I also have no reason to doubt Jon's findings about Big Endian... the file Zn0rt referred us to seemed to have been tagged by that MusicBrainz tag utility which seems to be trying to serve as a 'Cross Platform' tag database app. I'd rather suspect it of serving out BE tag data over foobar...

So like Nudel has suggested... foobar is likely just writing a Unicode tag and nothing more. If it can't be configured to write in ANSI, then it's something that will have to wait til Jon has a chance to make a change in Opus to support.

Note- yeah, I guess the 'benefit' of APE tags is that it's at the end of the file and would allow for faster tagging. Padding an ID3v2 tag at the beginning of a file might help certain tag 'updates' go faster, but NOT if no tag exists there in there first place :slight_smile:.

Yup, from the foobar faq...

*  Q: ID3v2 tags added/modified by foobar2000 are not read from application X or portable Y. What can I do?
* A: foobar2000 writes ID3v2.4 tags encoded as UTF-8 (specified in late 2000). Please ask the vendor of X or Y to support this revision of the ID3v2 standard.

UTF-16 is not the same as "Unicode". It is an encoding system that can represent 16 (or 32) bit character values, but it is not the same as straight 16 bit Unicode encoding, which is what Opus does (as little-endian anyway) support.

I guess we may as well add UTF-8/16 support as well as BE Unicode support while we're at it...

[quote="jon"]UTF-16 is not the same as "Unicode". It is an encoding system that can represent 16 (or 32) bit character values, but it is not the same as straight 16 bit Unicode encoding, which is what Opus does (as little-endian anyway) support.

I guess we may as well add UTF-8/16 support as well as BE Unicode support while we're at it...[/quote]

I guess you should. Otherwise I'll set Nudel's fleshlight on you. (uncleaned)

Woopsy - right you are Jon - sorry.

If I'm reading the change list correctly, it looks like an update just came out which fixes the problem with some types of Unicode tags:

[Directory Opus 8.2.2.1)

"Added full support for UTF-8, UTF-16LE and UTF-16BE encoded tags in MP3 files."

[quote="nudel"]If I'm reading the change list correctly, it looks like an update just came out which fixes the problem with some types of Unicode tags:

[Directory Opus 8.2.2.1)

"Added full support for UTF-8, UTF-16LE and UTF-16BE encoded tags in MP3 files."[/quote]

This seems to be working in the file list but the viewer plugin hasn't been updated. Not that I care, tbh, but someone else might :slight_smile:

Edit: actually there is some really weird stuff happening here. If I keep refreshing the lister (i.e. pressing F5 repeatedly) the fields keep changing and adding extra characters on the end.

Like in this screenshot...

I am on Windows 2000 SP4 if this helps at all.

Hmmm... sovereignty68 seems to have experienced a similar issue with some of his mp3 tags, but so far has not responded (that I've seen) with a posting of a sample 'problem' file. Could you help out with that ledge? The utility I had used (ID3-TagIT) to find out what worked/didn't work is mysteriously offline of a sudden... so I can't test at the moment to try and reprodude it here without hunting down a tagger that let's me manipulate the encoding. At least - as easily as the app above allowed...

I've been able to repeat this here after doing some retagging with foobar2000 - investigating now...

This file is among those that are causing the problems, but I don't think it is the file as the extra characters keep changing.

Anyway here is a file that it happens with.

mexicankitsch.biz/GivingHerAway.mp3

Ok, this is fixed now. A new version is available for download from the GPSoft website.

Note that we did not bump the product version number as this is such a minor change - it is still 8.2.2. However the file version number of dopus.exe itself has been bumped.

[quote="jon"]Ok, this is fixed now. A new version is available for download from the GPSoft website.

Note that we did not bump the product version number as this is such a minor change - it is still 8.2.2. However the file version number of dopus.exe itself has been bumped.[/quote]

This is all working at home. I'll confirm it at work, where I noticed the problem, tomorrow.