Invalid color mapping for CMYK Jpeg Images with DOpus 13.5.2 (Beta)

Continuing the discussion from Directory Opus 13.5.2 (Beta):

I thought this and some of the other changes mentioned in the release-notes of this beta may does fix the color mapping of 32-bits JPEG CMYK Images, since at least for me DOpus never shows there images "correctly" (at least, compared to how Browsers does show them)

So... i went and installed the Beta, and found out these kind of images are now completely broken with invalid colors -- following is a sample picked from Wikipedia, to make sure the problem is not with my own images.

241723102932

(Source: File:Channel digital image CMYK color.jpg - Wikipedia)

I also tried to specify an ICC profile in use_color_management (i.e. the same one i use to create/convert my images), but found that is not working, upon selecting it on the file picker such advanced option remains unchanged (i would expect the filename selected shown there, but it is not)

Please let me know if you do need further information to help on troubleshooting / fixing this.

PS: Might there be some drawback / side-effect /problem from downgrading to the previous Beta i was using, or rather should wait for a new beta to be released?

It looks like the fix for this problem broke your images, sigh. CMYK in JPEGs seems to be a complete mess.

Give this version a try, it seems to work with both images: https://cdn.gpsoft.com.au/files/Opus13/Beta/DOpusInstall-13.5.2a.Beta.exe

1 Like

That fixed it on my end.

Thanks Jon for promptly trying to revolve it, and it did it...for the Wikipedia image, but my own images are still wrong i'm afraid.

Here's a sample...

python-pillow-cmyk.7z (302.9 KB)

I can give you the Python code i do use to create them, if it does help further, wondering if the issue is on my side...but, browsers does show it correctly.

Many thanks. We have a fix coming for that as well.

1 Like

The root of the mess is an error which was recently fixed by the JPEG library, but apparently not by Adobe.

The error is NOT in Opus and it can't be fixed there!
Any application using the JPEG library will show the same result.
The error can only be fixed at the place where the bogus write happens.

Following is an elaborate explanation which I gave in response to a corresponding bug report.

There has been old criticism of the JPEG format that it would not specify the colorspace properly.
In some prior version this issue has been fixed for the 3-component colorspaces (YCC/RGB) in file jdapimin.c, function default_decompress_parms():

  switch (cinfo->num_components) {
  ...

  case 3:
    ...

    /* For robust detection of standard colorspaces
     * regardless of the presence of special markers,
     * check component IDs from SOF marker first.
     */
    ...

After this fix I also received false color reports.
I changed the component IDs in the transmitted file to the correct values by a binary (hexadecimal) editor and then the file displayd correctly.
What has happened? Previously JPEG files with RGB colorspace were primarily recognized by the presence of a custom Adobe marker. RGB JPEG files generated by Adobe software use the component identifiers 1,2,3 which are reserved by JFIF (JPEG File Interchange Format) conventions for the YCbCr colorspace. This Adobe method is therefore wrong. I suspect that Adobe programmers used some "black box" JPEG module from third parties and thus could not access the SOF component IDs, and their "solution" was to introduce the custom marker.
IJG libjpeg software, on the other hand, has always written RGB JPEG files with special SOF component IDs ('R', 'G', 'B') in file jcparam.c function jpeg_set_colorspace(), beside emitting the custom Adobe marker, so that the files are marked correctly and also work with Adobe software.
Note: we do NOT want to depend on custom Adobe markers for proper standard colorspace detection, that's why the bogus Adobe files are no longer supported.

The JPEG library version 9e simply applied the same fix to the 4-component colorspaces (YCCK/CMYK) - the current version is JPEG 9f.
So you get different colors with wrong SOF component IDs (files generated by bogus Adobe software, for example).
In your case the image was created by apparently bogus "Intel JPEG Library" according to the COM (Comment) marker in the file, and I have attached the modified version with corrected component IDs in the SOF (Start Of Frame) and SOS (Start Of Scan) markers.

This image displays the same colors with any libjpeg version, and such kind of images are generated by any libjpeg version when selecting CMYK JPEG output.

Regards
Guido
JPEG developer

See also Colors of CMYK pictures not correctly showed in DO 13 image viewer

3 Likes

Thanks Leo!

Guido, first of all let me say is quite awesome finding the libjpeg author around here as another DOpus user, thanks for taking the time to reply with such a detailed response, much appreciated.

Actually, i already saw a couple of your earlier posts fixing the bogus images for other users, and upon comparing the file(s) saw you added the CMYK markers.

Then went to verify my images to check if the problem was the same, and despite this rudimentary method using a hex viewer i think mine do contain both as expected (?)

Your hint about the COM marker gives a clue, although let me say i used ExifTool to remove all metadata before uploading my sample...which may or may not is the reason for that comment marker you saw.

I use Python and Pillow to convert the image(s) between colorspaces, which uses the lcms library, not sure from what of these components the issue comes from, if there's a way i can automatically fix these images with some python function/helper i will be more than please to integrate it, instead of relying on workarounds being applied by DOpus.

Btw, you mentioned you've attached a fix for my image but i don't see it here, missed? :slight_smile:

Hello diegocr

It seamed that my previous posts were not enough to understand the issue sufficiently, so I picked an elaborate response from my email archive to another reporter (got quite a few "bug reports" regarding this topic).

So the mentioned example with the COM marker is not related to your image, it was another case.

The other examples here in the previous posts were Adobe generated files and I had to use JPEGsnoop first to find the position (offsets) of the SOF and SOS markers, because there was plenty of Adobe marker stuff to skip. With these offsets I could go to the Hex editor and edit.

So yes, it is good if all marker baggage is stripped off, then you can easily find the essential markers in the beginning as in your case.

I didn't check your image because you told it comes from Python.
I'm not familiar with Python, but I had reports regarding this issue from Python users.
So I learned that Python has integrated libjpeg and can even use different libjpeg versions.
The problem reported was reading externally generated files with newer versions, which is clear as described. There were no problems reading Python generated JPEG files.

I have now checked your file and it looks perfectly OK to me.
As mentioned, any libjpeg version, old and new, should write correct CMYK files.
I have still Opus 13.5.1 running, because I hesitate to install 13.5.2 which they say "fixes" something in this regard, but as said I see nothing that could be fixed here, and it is OK as it should be. Again: There is nothing to fix here. Any "fix" here could only make it worse. Only Adobe can fix it.

Here is your file with djpeg:

F:\>djpeg -v 00031-3170241409.jpg nul
Independent JPEG Group's DJPEG, version 9f  14-Jan-2024
Copyright (C) 2024, Thomas G. Lane, Guido Vollbeding
Start of Image
Adobe APP14 marker: version 100, flags 0x0000 0x0000, transform 0
Define Quantization Table 0  precision 0
Define Quantization Table 1  precision 0
Start Of Frame 0xc0: width=576, height=768, components=4
    Component 67: 1hx1v q=0
    Component 77: 1hx1v q=1
    Component 89: 1hx1v q=1
    Component 75: 1hx1v q=1
Define Huffman Table 0x00
Define Huffman Table 0x10
Start Of Scan: 4 components
    Component 67: dc=0 ac=0
    Component 77: dc=0 ac=0
    Component 89: dc=0 ac=0
    Component 75: dc=0 ac=0
  Ss=0, Se=63, Ah=0, Al=0
PPM output must be grayscale or RGB

You see there is still the custom Adobe marker. It is unnecessary here but may make Adobe happy.
This was my concern: If some transcoding process loses the marker, you will still have all essential information for proper image reconstruction in the file. The SOF marker is essential for image reconstruction anyway so this is a better place for colorspace identification.

Here is the corresponding source code in file jdapimin.c, function default_decompress_parms() with the hex values of the component identifiers:

  case 4:
    cid0 = cinfo->comp_info[0].component_id;
    cid1 = cinfo->comp_info[1].component_id;
    cid2 = cinfo->comp_info[2].component_id;
    cid3 = cinfo->comp_info[3].component_id;

    /* For robust detection of standard colorspaces
     * regardless of the presence of special markers,
     * check component IDs from SOF marker first.
     */
    if      (cid0 == 0x01 && cid1 == 0x02 && cid2 == 0x03 && cid3 == 0x04)
      cinfo->jpeg_color_space = JCS_YCCK;
    else if (cid0 == 0x43 && cid1 == 0x4D && cid2 == 0x59 && cid3 == 0x4B)
      cinfo->jpeg_color_space = JCS_CMYK;   /* ASCII 'C', 'M', 'Y', 'K' */
    else if (cinfo->saw_Adobe_marker) {

Regards
Guido
JPEG developer

1 Like

Recent changes in Opus have all been around color profiles (via lcms) rather than libjpeg specifically.

Problem in this thread was with CMYK images without an embedded color profile (we have a default CMYK profile but it wasn't being used due to changes in the previous version).

1 Like

Thanks everybody, all is fine now with 13.5.3 :partying_face:

2 Likes