Convert JPG to resized/quality changed JPG error

Help & Support
1

Convert JPG to resized/quality changed JPG error

I see several similar issues reported in the past, but none exactly the same as far as I can tell. Not sure if this is a problem with DOpus or with Windows, or both. I’ve been doing numerous mass JPEG image conversion operations recently with settings in Image Conversion dialog as follows:

Convert to: JPEG
Quality: 75
Resize: 5000 x 5000
Add filename suffix
Preserve aspect ratio

I’m generally converting all images in a folder this way. Number of images ranges from maybe a dozen to 100 per folder. All folders have basically the same nested depth in the volume they’re on, which is 8 deep beyond root. Complete path length is roughly 160, give or take. Average filesize of unconverted JPEGs is 1.5–2.5 MB. All JPEGs have an X or Y pixel dimension of 5000–7500. What happens is that sometimes I get the following error message:

“An error occurred converting ‘X.jpeg’: The operation could not be completed. A retry should be performed. (1237)”

Retrying yields the same error, repeatedly.

For most of these folders, I do not get the error. For some, it occurs for a handful of images but not the rest; for others, it occurs for ALL images. The really weird thing (to me, anyway) is, I can move all offending images to a temporary subfolder (9 deep beyond root), or to the parent folder, and then complete all conversions from there without a hitch (so far).

Using Windows 10 x64, fully upated, and currently DOpus v12.14, but it has also happened for me using the past few versions. I think I started performing these mass conversions maybe 4–5 months ago.

2

Hi!

Do your paths contain spaces? Sometimes it causes trouble. I always name my folders like:
C:\Pics\Car_pics\ (with underscore) instead of C:\Pics\Car pics\ (with space in the name)
My idea is mainly from an excel/macro/vb point of view, if this is irrevelant please ignore my comment.

3

Spaces shouldn't matter.

Total path length might, as some APIs or libraries are limited to 260 chars.

How long are the paths? Is length the difference between your working and non-working folders, or is there another difference? e.g. Network drives vs local, or a folder something else is monitoring for changes?

4

In answer to tortilladude — Yes, there are spaces in ALL of these folder paths, but most of them work.

Leo — From my original post:

All folders have basically the same nested depth in the volume they’re on, which is 8 deep beyond root. Complete path length is roughly 160, give or take.

Example path:

“D:\\Data1\ABCD E FGHIJKLMN OP\IMGS\EDMI\EX\ZYXWVUTSRQ ONMLKJIH\QWxleC1MeW5uL NvbSBOYW5jeSAtI dhcmRyb2JlIFBvc2l\2luZyAtIHg3NCAtI QyMjBweCAoMzAgSn uLCAyMDE4KS zNTE1NTgxMA\”

Moving images that couldn’t be converted in the location above to a temporary subfolder would result in:

“D:\\Data1\ABCD E FGHIJKLMN OP\IMGS\EDMI\EX\ZYXWVUTSRQ ONMLKJIH\QWxleC1MeW5uL NvbSBOYW5jeSAtI dhcmRyb2JlIFBvc2l\2luZyAtIHg3NCAtI QyMjBweCAoMzAgSn uLCAyMDE4KS zNTE1NTgxMA\Temp\”

Or, moving them to the parent folder:

“D:\\Data1\ABCD E FGHIJKLMN OP\IMGS\EDMI\EX\ZYXWVUTSRQ ONMLKJIH\QWxleC1MeW5uL NvbSBOYW5jeSAtI dhcmRyb2JlIFBvc2l\”

All folders are on a local drive. There are probably differences in length between working and non-working folders, since they all have different names, but since I can move images from a non-working folder to a folder named “Temp” within it (resulting in a slightly longer path) and convert them just fine from there, can it be relevant?

5

Agreed, it won't be length in that case.

My guess is something else is getting in the way of things. (It could even be Opus itself, but could also be something else monitoring the files.)

A Process Monitor log of what happens is probably the quickest way for us to diagnose the issue, if you're OK with making one.

6

I’ll keep that in mind the next time it rears its ugly head.

7

All right, I finally got around to looking at this issue with Microsoft Process Monitor. In this case, I was only attempting to convert two images, rather than a large number as I had when encountering this problem before. Conversion attempted was JPEG @ Quality 95, resized to 500 x 500 without aspect ratio preservation.

This file (531,519 bytes, 738 x 734 x 24) failed:

E:\Music\Beatles\Beatles - 2009 Remasters (Mono & Stereo) + Extras [FLAC]\Mono\Beatles - [1970-03-06] Mono Masters\Artwork\Singles & EPs\[1968-08-30 Apple R5722] Sleeve+Label A (Hey Jude).png

…while this file (254,702 bytes, 743 x 744 x 24) succeeded:

E:\Music\Beatles\Beatles - 2009 Remasters (Mono & Stereo) + Extras [FLAC]\Mono\Beatles - [1970-03-06] Mono Masters\Artwork\Singles & EPs\[1968-08-30 Apple R5722] Sleeve+Label B (Revolution).png

Full paths for the two files are 191 and 193 characters, respectively.

Process Monitor log file:

https://drive.google.com/open?id=1vdWKGj9ZR_UtzJDWZD1cNCHcTZJhGBAr

Links to 2019-07-05 20;53;13 - MAZE - Logfile.7z (7,858,588)

Contains 2019-07-05 20;53;13 - MAZE - Logfile.PML (147,211,956)

Please let me know when you no longer need the link to the file, so I can delete it.

1 Like
8

It looks like the EXIF library didn't like the metadata that is the source file.

Could you zip and share the PNG somewhere?

(Please zip it, rather than upload the PNG directly, as the forum and a lot of other things will modify PNG images to remove metadata unless they are zipped.)

9

[1968-08-30 Apple R5722] Sleeve Labels (Hey Jude Revolution).7z (626.8 KB)

Ok, but I didn’t add any metadata to the images, which I manually combined from separate images of the sleeves and labels that I found on the same web site, and DOpus apparently doesn’t even recognize them as containing any metadata (which, of course, doesn’t mean that there isn’t any there). Here you go…

Uploaded: [1968-08-30 Apple R5722] Sleeve+Labels (Hey Jude+Revolution).7z (641,821)

Contents:

[1968-08-30 Apple R5722] Sleeve+Labels (Hey Jude+Revolution)\
[1968-08-30 Apple R5722] Sleeve+Label A (Hey Jude).png (531,519) [738 x 734 x 24]
[1968-08-30 Apple R5722] Sleeve+Label B (Revolution).png (254,702) [743 x 744 x 24]

1 Like
10

Thanks. With that file I can reproduce the problem. It wouldn't matter where the file was -- it would always fail -- so it may not be the same issue you were seeing before.

That PNG file has 150KB of metadata added to it by Photoshop (possibly a record of editing history). This is far more than a normal image would have ("typically around 2KB"). The image converter in Opus aims to preserve metadata when converting from PNG to JPG. But the JPG standard only allows and only allows at most 64KB per section. XMP has methods for splitting metadata across multiple sections, but they are not supported by a lot of readers and writers, and the XMP library we are using fails in that situation.

This is a good summary of what's happening: java - Inserting large amount of XMP data into jpg using multiple packets? - Stack Overflow

Also:

Initially, Adobe didn't expect the XMP data length would exceed the limit of one JPEG segment (about 64K) and their XMP specification stated the XMP data must fit into one. Later when they found a single JPEG APP1 segment is not large enough to hold the XMP data, they changed their specification to allow for multiple APP1 segments for the whole XMP data.

From: java - Reading JPG file's XMP metadata - Stack Overflow

(The two Stack Overflow links are about languages/libraries we don't use, but the info about JPG/XMP still applies.)

We might be able to improve the XMP library we're using to handle this, but I haven't looked at that in detail yet.

If you want to avoid the problem in the mean time (and also reduce a lot of wasted space in the images), stripping the cover art of metadata should fix things. One quick way to do that is to convert to BMP, then back to PNG, or to JPG. There are also tools you can use to (batch) remove it directly without changing formats.

11

Something doesn’t sound right. I used the following source JPEGS (the largest of which is only 117K, and it was used for the PNG that ultimately converted successfully) to create the PNGs:

[1968-08-30 Apple R5722] Label A (Hey Jude).jpg (109,335) [794 x 819 x 24]
[1968-08-30 Apple R5722] Sleeve A (Hey Jude+Revolution).jpg (33,755) [738 x 734 x 24]

…put together to make [1968-08-30 Apple R5722] Sleeve+Label A (Hey Jude).png (531,519) [738 x 734 x 24]

[1968-08-30 Apple R5722] Label B (Revolution).jpg (120,242) [799 x 808 x 24]
[1968-08-30 Apple R5722] Sleeve B (Hey Jude+Revolution).jpg (48,834) [743 x 744 x 24]

…put together to make [1968-08-30 Apple R5722] Sleeve+Label B (Revolution).png (254,702) [743 x 744 x 24]

All of the JPEGs above were obtained (with different filenames) from here:

http://thebeatles-collection.com/wordpress/2011/12/01/hey-jude-revolution-apple-r-5722-6/

I just used the latest version of ExifTool to extract all metadata from all four JPEGs. The largest file of extracted metadata is 1,514 bytes, and contains no history from any image editor.

I also extracted all metadata from the PNGs I created. The file [1968-08-30 Apple R5722] Sleeve+Label A (Hey Jude).png resulted in 123,110 bytes of extracted metadata (but that size includes the tag and group names, plus actual metadata, in JSON format). Obviously, 123,110 bytes is a lot, and XMP:History contains the lion’s share of it (121,969 to be exact), but you’re saying you found 150K, or that’s just a rounded-up ballpark figure?

Anyway, it looks like that metadata had to have been inserted by my own image editor, Corel PaintShop Pro, though I don’t know why similar data wouldn’t then also exist in the other PNG, whose extracted metadata only amounts to 997 bytes in JSON format. I’ve now used ExifTool to delete the XMP:History data from the “Hey Jude” PNG, and successfully converted it to a 500 x 500 JPEG using DOpus.

Back to searching for an example of the original issue…

1 Like
12

That's what's causing the problem. 65KB is the limit.

13

Apologies for the mega bump, but I just ran into this problem today. I created the PNGs in Paint.net, so I'm a little surprised that it added so much metadata. As an experiment I took the original JPEG straight off the phone and converted that to another JPEG. The resulting file was slightly smaller and the conversion completed okay, so apparently Paint.net added the metadata.

Is there any chance this will be fixed? Even just a checkbox to allow discarding excessive metadata would be useful.

Edit: And while I'm here, some more controls over the JPEG output, such as the chroma subsampling.

1 Like
14

Seems like something that should be fixed in Paint.Net. Why is it writing excessive amounts of metadata into compressed formats intended for web use?

1 Like
15

In fact Paint.net did cut the amount of metadata down considerably, from 155k in the original JPEG to about 73k in the PNG.

However, it shouldn't have to remove the metadata that is still relevant to the PNG. That isn't a bug. It contains useful information about the location where the photo was taken, the camera settings, ICC colour profile and so forth. The removal of the colour profile in particular will result in the colours changing, i.e. being wrong. You can't just apply the colour profile and save the result, that way you lose information that is needed to correctly display and print it.

Of note is the fact that Opus handles this situation in JPEG files, but not in PNG files. While JPEG files do have a limit of 64kB per segment, supporting multiple metadata segments is required if you don't want to be discarding the user's metadata. In any case, PNG has no such limitation, the maximum chunk size being 2TB.

PNG isn't only for web use, it's a useful lossless format for all kinds of work. Although a lot of imaging software used TIFF for historical reasons, PNG is more capable and produces smaller files. As an experiment I made a TIFF version, and the file grew from 13.6MB to 18.1MB, and Paint.net stripped some of the metadata though, and Opus was able to convert it. I can't copy all the metadata into the TIFF version even with exiftool, which appears to be a limitation of TIFF files - a great example of why people use PNG for photo work.

16

The samples from earlier in the thread convert OK in Opus today, as far as I can tell.

If you have files that don't convert, please zip and upload some examples.

17

PXL_20221213_074424549 64px.zip (62.0 KB)

Attached. I was able to resize the image down to tiny proportions and still see the error, which is handy as the original is 18MB. Thanks.

18

We might be able to make that file convert by stripping out the huge chunk of metadata. I think that would be the only way. I've made a note to look at it once we have some time. Too busy with Opus 13 right now.

An ICC profile and GPS coordinates do not take up >64KB of data. Something excessive is being saved to the metadata there.

If you're OK with removing the metadata entirely, conversion to BMP and then JPG will work.

19

Thanks, that's a work-around at least. I'll investigate the metadata when I have time, see what else is in there.

Actually now I think about it, a button to strip metadata would be handy. There is a Windows version of exiftool so I'll give it a try.

20

I did some investigation. It's all camera related metadata. Details of every part of the optical and digital pathways. Curves for various corrections, some serial numbers and other binary stuff.

I'd like to keep it all sometimes, and other times I want to strip it. An option would be nice.