.NET GDI+ image size - file codec limitations

Question

Is there a limit on the size of image that can be encoded using the image file codecs available from .NET?

I'm trying to encode images > 4GB in size, but it simply does not work (or does not work properly i.e. writes out an unreadable file) with .bmp, .jpg, .png or the .tif encoders.

When I lower the image size to < 2GB it does work with the .jpg but not the .bmp, .tif or .png.

My next attempt would be to try libtiff because I know tiff files are meant for large images.

What is a good file format for large images? or am I just hitting the file format limitations?

(All of this is being done on a 64 bit operating system (WinXP 64) w/ 8 GB of RAM and compiled using x64 architecture.)

Random r = new Random((int)DateTime.Now.Ticks);

int width = 64000;
int height = 64000;
int stride = (width % 4) > 0 ? width + (width % 4) : width;
UIntPtr dataSize = new UIntPtr((ulong)stride * (ulong)height);
IntPtr p = Program.VirtualAlloc(IntPtr.Zero, dataSize, Program.AllocationType.COMMIT | Program.AllocationType.RESERVE, Program.MemoryProtection.READWRITE);

Bitmap bmp = new Bitmap(width, height, stride, PixelFormat.Format8bppIndexed, p);
BitmapData bd = bmp.LockBits(new Rectangle(0, 0, bmp.Width, bmp.Height), ImageLockMode.ReadWrite, bmp.PixelFormat);

ColorPalette cp = bmp.Palette;
for (int i = 0; i < cp.Entries.Length; i++)
{
  cp.Entries[i] = Color.FromArgb(i, i, i);
}
bmp.Palette = cp;

unsafe
{
  for (int y = 0; y < bd.Height; y++)
  {
    byte* row = (byte*)bd.Scan0.ToPointer() + (y * bd.Stride);
    for (int x = 0; x < bd.Width; x++)
    {
      *(row + x) = (byte)r.Next(256);
    }
  }
}

bmp.UnlockBits(bd);
bmp.Save(@"c:\test.jpg", ImageFormat.Jpeg);
bmp.Dispose();

Program.VirtualFree(p, UIntPtr.Zero, 0x8000);

I have also tried using a pinned GC memory region, but this is limited to < 2GB.

Random r = new Random((int)DateTime.Now.Ticks);

int bytesPerPixel = 4;
int width = 4000;
int height = 4000;         
int padding = 4 - ((width * bytesPerPixel) % 4);
padding = (padding == 4 ? 0 : padding);
int stride = (width * bytesPerPixel) + padding;
UInt32[] pixels = new UInt32[width * height];
GCHandle gchPixels = GCHandle.Alloc(pixels, GCHandleType.Pinned);
using (Bitmap bmp = new Bitmap(width, height, stride, PixelFormat.Format32bppPArgb, gchPixels.AddrOfPinnedObject()))
{
    for (int y = 0; y < height; y++)
    {
        int row = (y * width);
        for (int x = 0; x < width; x++)
        {
            pixels[row + x] = (uint)r.Next();
        }
    }

    bmp.Save(@"c:\test.jpg", ImageFormat.Jpeg);
}
gchPixels.Free();

Answer 1

The problem you are facing is very likely the following limitation in the .NET Framework: the maximum object size allowed in the GC Heap is 2GB, even on 64-bit OS / builds.

Some references: link, link, link.

Depending on your needs, with a relatively simple (uncompressed) format like TIFF it might be doable to generate your image in parts and merge the parts afterwards. Just an idea..

Answer 2

From the TIFF specification Section 2: TIFF Structure at at http://partners.adobe.com/public/developer/en/tiff/TIFF6.pdf:

TIFF is an image file format. In this document, a file is defined to be a sequence of 8-bit bytes, where the bytes are numbered from 0 to N. The largest possible TIFF file is 2**32 bytes in length.

Unfortunately many TIFF writers and readers use signed integers in implementation and reduce the practical size to 2**31. This is compounded by the frequent attempt by drivers to keep the entire image in memory as mentioned by Cristophe.

You are likely to find similar limitations in the other image formats created in the '80s or earlier. Multi gigabyte file size limits were not considered a problem when disks were only in the tens of megabytes.

Answer 3

So your goal is to encode them into jpeg? I am sure there would be a way to pull in only the section you are encoding, then dispose and go on to the next section. If you open the file raw and and parse it yourself, I am sure there are open source jpeg encoders out there to help with creating the sections. This should allow you to encode files extremely large, but your will always have OS and filesystem hurdles to overcome.