How do I use ImageSharp with large orthomosaics?

[aevitas]

Hi,

I've been a long time user of ImageSharp for all my imaging needs, so when I started a recent project using satellite imagery and drone footage, I once again came back to this library.

The images I'm working with are large geographical TIFF images, also known as orthomosaics. They can be stitched together from numerous other images, and as a result can become quite large (some of our current images are north of 1.5GB).

I'd like to use ImageSharp to transform, transcode and "tile" these images into smaller images, but when I attempt to load one of these images, I get the following exception:

Unhandled exception. SixLabors.ImageSharp.InvalidImageContentException: Cannot decode image. Failed to allocate buffers for possibly degenerate dimensions: 0x0.
 ---> SixLabors.ImageSharp.Memory.InvalidMemoryOperationException: Attempted to allocate a buffer of negative length=-227135488.
   at SixLabors.ImageSharp.Memory.InvalidMemoryOperationException.ThrowNegativeAllocationException(Int64 length)
   at SixLabors.ImageSharp.Memory.UniformUnmanagedMemoryPoolMemoryAllocator.Allocate[T](Int32 length, AllocationOptions options)

Debugging this, I can see the following number of bytes being allocated every time:

1. 62424
2. 15606
3. 15606
4. 262144
5. -227135488

Which I assume is four metadata allocations, followed by an allocation for the actual (overflowed size) image data.

Given that the TIFF I'm trying to load is 740MB, I fail to see why it would need to allocate over 2.14GB of memory to process the image. Other libraries such as GDAL are perfectly fine loading these images in an instant, but don't offer the image manipulation I need.

Is this a limitation with ImageSharp specifically? Is there something I'm doing wrong and is there a way to use these images with ImageSharp?

I've explored making my own fork of ImageSharp and replacing the allocation with just flat direct allocations as I'd be running this from a command line so the heap would never be long-lived, but it seemed like a non-trivial modification so I'd like to make sure there aren't any more sensible approaches.

Can provide sample files if needed.

[JimBobSquarePants]

Hi @aevitas

Are you using the latest version v3.1.6? That negative number looks to me like we are suffering from some sort of arithmetic overflow issue. Were there additional lines in the stack trace?

[aevitas]

Yes, I'm using ImageSharp 3.1.6. I think it's overflowing somewhere in DecodeTilesChunky<SixLabors.ImageSharp.PixelFormats.Rgba32>:

UniformUnmanagedMemoryPoolMemoryAllocator.Allocate<byte>() UniformUnmanagedMemoryPoolMemoryAllocator.cs:line 86
TiffDecoderCore.DecodeTilesChunky<SixLabors.ImageSharp.PixelFormats.Rgba32>() at /_/src/ImageSharp/Formats/Tiff/TiffDecoderCore.cs:line 642
TiffDecoderCore.DecodeImageWithTiles<SixLabors.ImageSharp.PixelFormats.Rgba32>()
TiffDecoderCore.DecodeFrame<SixLabors.ImageSharp.PixelFormats.Rgba32>()
TiffDecoderCore.Decode<SixLabors.ImageSharp.PixelFormats.Rgba32>()
ImageDecoderCore.Decode<SixLabors.ImageSharp.PixelFormats.Rgba32>()
TiffDecoder.Decode<SixLabors.ImageSharp.PixelFormats.Rgba32>()
TiffDecoder.Decode()
ImageDecoder.<>c__DisplayClass1_0.<Decode>b__0()
ImageDecoder.<WithSeekableStream>g__PerformActionAndResetPosition|11_0<SixLabors.ImageSharp.Image>()
ImageDecoder.WithSeekableStream<SixLabors.ImageSharp.Image>()
ImageDecoder.Decode()
Image.Decode()
Image.<>c__DisplayClass80_0.<Load>b__0()
Image.WithSeekableStream<SixLabors.ImageSharp.Image>()
Image.Load()
Image.Load()
Image.Load()
Program.<Main>$()

If needed I can provide a fairly minimal application to reproduce with some of the actual TIFF data I'm using.

[antonfirsov]

That would help us a lot!

[aevitas]

That would help us a lot!

This code will throw the exception on the Image.Load call:
Archive.zip

When used with this orthomosaic

Let me know if you need me to convert this discussion into an issue.

[aevitas]

I've been diving a little deeper into this, and I've found that uncompressedPixelBuffer in DecodeTilesChunky overflows. It requires 4067831808 bytes to allocate the full image, which seems impractical.

I'm not that well-versed in the nitty gritty of the TIF file format, but would it perhaps be possible to determine tiles using specific regions of the image so that I'd never have to allocate the full image data, but only specific sections?

Is it even possible to load images this large since considering the maximum length of an Array is int.MaxValue?

[JimBobSquarePants]

I'm no expert in the Tiff format (I didn't write that code) but it looks to me like the method can be refactored to decode each row of tiles. That would reduce the allocation requirements significantly.

[JimBobSquarePants]

Yep. It's possible to remove that allocation completely and decode each tile row directly to the image.