Comparing JPEG, WebP and HEIC images after bit flips

28 Feb 2024

In the last few months, I have thought a little about long-term archiving files, the effects of random bit errors and how it interacts with compression. So today, I looked a little at the different image formats.

I am not an expert in modern image compression, so I ran a totally unscientific experiment for fun. I wrote a terrible a Python script that uses ImageMagick to convert an image into different formats and flip either a single or four random bits. This is repeated nine times and the images are converted to PNG and composed into a grid.

I look at JPEG, WebP and HEIC (Apple’s format based on H265). My assumption is that JPEG uses much older and less efficient compression algorithms that leave some redundancy in the file and might degrade less, while WebP and HEIC seem kind of similar to me.

1 bit flipped

Click images to enlarge.

JPEG

Eight of the nine images look fine, while in the middle image, something important seems to be destroyed. Maybe something is in the file header.

WebP

Here the horrible blocky artifacts start from some line and everything after is broken, but retains the basic color palette.

Two images look fine, and two others have major artifacts (to me, comparable to old photographs with major water damage), but if the motif was important to me, they seem kind of okay.

The other five images are basically destroyed. You might recognize them when you know the image, but they seem mostly useless to me.

HEIC

This one confuses me. The artifacts seem only to wander down and to the right, and they are incredible horrible to look at. Blocky geometric shapes in strange colors.

Again, two images look fine. For the other images, I am at a loss how to categorize them. There are more perfectly good pixels than in the WebP images, but the artifacts are horrible. Some might be okay as mementos if you blur and fade out the color of the artifacts. Five might be salvageable in this way.

One is a complete loss except for the bird head, but even that might be good in some cases.

4 bits flipped

Click images to enlarge.

JPEG

Four images are fine. The other ones have a line where the color shift or the motif is shifted a little. They are not great, but totally acceptable. This time, there was no header damage.

WebP

Three images are kind of okay with some major artifacts; five are for bird-head enthusiasts only, and the last one is a complete loss.

HEIC

This is a horror show.

BMP with 1000 bit flips

Now, for fun, let’s look at BMP as an uncompressed file format, but let’s introduce 1000 bit errors instead of four.

This looks perfectly fine. There are obviously some pixel errors, but they don’t destroy the image. I think it would be trivial to find the worst of them and simply replace them with an average of the surrounding pixels.

Of course, at this bit-flip rate, we were lucky not to destroy the header fields. Even if this happens, we might be able to fix the header with a text editor and calculator, because BMP is a really simple format.

Restrictions

Especially the comparison with the bitmap is kind of unfair. The uncompressed file is orders of magnitude bigger and thus will have a much higher rate of bit error.

If I had written less horrible Python code, I might have corrected the flip rate for different file sizes.

What did we learn at the end?

Use redundant storage with checksums, so that there are no bit flips.

JPEG is kind of an okayish compromise. It is still much more efficient to store five copies of every JPEG instead of a BMP.

Why are the artifacts so different between WebP and HEIC? This might be a question for an expert in image compression, instead of some dubious experiment.