Vips images rendering offset vs ImageMagick

[fitnycdigitalinitiatives]

Hello,

I've run into a funny issue I wonder if you might have some insight into. We were recently processing a new batch of images when I noticed some of tiles rendered in OpenSeadragon viewer were misaligned. You can see at the bottom of this image how the border is offset.

Once you begin to zoom in, it realigns. I did some troubleshooting tested created the same image using ImageMagick rather than VIPS. In that case the image align correctly initially:

Do you think there might be a bug in VIPs or is this possibly an issue with serverless-iiif?

I will send you the images directly.

Thanks,

Joe Anderson

[mbklein]

That's an interesting find. I'd like to look into it further. Would you be able to share both the ImageMagick-derived TIFF and the VIPS-derived TIFF? You can either attach them to this issue, or if you prefer, you can send them to my gmail address (same username as github).

[fitnycdigitalinitiatives]

Just sent them to you.

[mbklein]

What is the size of the div you're embedding OSD in? I'm seeing some artifactual differences in my rendering, but the closer I can get to the actual OSD options/sizes/framing you're using the better.

Ignore that question. I have more data. Following up soon.

[mbklein]

The main issue seems to be that ImageMagick isn't generating a proper pyramidal TIFF – it's creating a single frame image that the IIIF processor is slicing up as needed. There's not a huge performance hit since the image isn't that big, but it's also smoothing out some rounding issues. I'll keep digging and see if I can be more specific.

[fitnycdigitalinitiatives]

Hmm, that's interesting. And it makes sense because this batch of images were initially digitized back in the day at a lower resolution (max 4200 px) than what we do now, which is probably why I noticed the issue. Because at a lower resolution the rounding makes a bigger difference. But odd that ImageMagick isn't actually making a pyramid!

Additional thing I noticed: I upgraded our OpenSeadragon to the latest version, and with that the misalignment is less pronounced but still there on the bottom right:

[mbklein]

What you're seeing with the latest OpenSeadragon is exactly what I've been seeing looking into this, so that's good.

This is a rounding issue brought on by tiling math. Below is the same image, with red borders drawn at the edges of the IIIF tiles OpenSeadragon is requesting.

For efficiency, iiif-processor figures out the scaling factor of the image first, figures out which page within the pyramid is the next highest resolution, and uses that as its starting point. In this case, we're right on the cusp – the tiles on the left side are working with a scale target of 756x1050, which is exactly the size of pyramid page 2. The scaling calculation for the tiles on the right is rounding the scale target to 757x1051, which is one pixel too large in either direction to use page 2, so those tiles are being generated based on page 1. Since each request is fully self-contained, they can't coordinate and use the same page.

I'm going to have to figure out a solution that won't cause other cascading rounding errors. It's possible that adding a couple pixels' worth of fudge factor to the paging decision will fix it without any noticeable degradation.

[fitnycdigitalinitiatives]

That's quite interesting. So would you essentially add a buffer that says that if it's within like say 10px of the target size, just jump down to the larger page?

[mbklein]

I think a 2px buffer would suffice, because this particular rounding error should only happen 1 pixel on either side of a page size boundary, as in the example above.

[mbklein]

As for the ImageMagick issue, I have learned in the course of researching this issue that ImageMagick does not automatically create all the downsampled levels when creating a pyramidal tiff as I previously thought it did. I'll be updating the docs very soon, and also adding instructions for GraphicsMagick (which uses a slightly different command).

[fitnycdigitalinitiatives]

Great, let me know when the update is merged and I can test it out.

Thanks for your help. It's funny that ImageMagick isn't actually making the ptiff's because when I was initially debugging and saw the ImageMagick image didn't have the misalignment, I had this sinking feeling that I was going to have to rederive all our images with ImageMagick instead of Vips.