Python PIL “IOError: image file truncated” with big images

Question

I think this problem is not Zope-related. Nonetheless I'll explain what I'm trying to do:

I'm using a PUT_factory in Zope to upload images to the ZODB per FTP. The uploaded image is saved as a Zope Image inside a newly created container object. This works fine, but I want to resize the image if it exceeds a certain size (width and height). So I'm using the thumbnail function of PIL to resize them i.e. to 200x200. This works fine as long as the uploaded images are relatively small. I didn't check out the exact limit, but 976x1296px is still ok.

With bigger pictures I get:

Module PIL.Image, line 1559, in thumbnail
Module PIL.ImageFile, line 201, in load
IOError: image file is truncated (nn bytes not processed).

I tested a lot of jpegs from my camera. I don't think they are all truncated.

Here is my code:

if img and img.meta_type == 'Image':
  pilImg = PIL.Image.open( StringIO(str(img.data)) )
elif imgData:
  pilImg = PIL.Image.open( StringIO(imgData) )

pilImg.thumbnail((width, height), PIL.Image.ANTIALIAS)

As I'm using a PUT_factory, I don't have a file object, I'm using either the raw data from the factory or a previously created (Zope) Image object.

I've heard that PIL handles image data differently when a certain size is exceeded, but I don't know how to adjust my code. Or is it related to PIL's lazy loading?

Answer 1

I'm a little late to reply here, but I ran into a similar problem and I wanted to share my solution. First, here's a pretty typical stack trace for this problem:

Traceback (most recent call last):
  ...
  File ..., line 2064, in ...
    im.thumbnail(DEFAULT_THUMBNAIL_SIZE, Image.ANTIALIAS)
  File "/Library/Python/2.7/site-packages/PIL/Image.py", line 1572, in thumbnail
    self.load()
  File "/Library/Python/2.7/site-packages/PIL/ImageFile.py", line 220, in load
    raise IOError("image file is truncated (%d bytes not processed)" % len(b))
IOError: image file is truncated (57 bytes not processed)

If we look around line 220 (in your case line 201—perhaps you are running a slightly different version), we see that PIL is reading in blocks of the file and that it expects that the blocks are going to be of a certain size. It turns out that you can ask PIL to be tolerant of files that are truncated (missing some file from the block) by changing a setting.

Somewhere before your code block, simply add the following:

from PIL import ImageFile
ImageFile.LOAD_TRUNCATED_IMAGES = True

...and you should be good.

EDIT: It looks like this helps for the version of PIL bundled with Pillow ("pip install pillow"), but may not work for default installations of PIL

Answer 2

Best thing is that you can:

if img and img.meta_type == 'Image':
    pilImg = PIL.Image.open( StringIO(str(img.data)) )
elif imgData:
    pilImg = PIL.Image.open( StringIO(imgData) )

try:
    pilImg.load()
except IOError:
    pass # You can always log it to logger

pilImg.thumbnail((width, height), PIL.Image.ANTIALIAS)

As dumb as it seems - it will work like a miracle. If your image has missing data, it will be filled with gray (check the bottom of your image).

Note: usage of camel case in Python is discouraged and is used only in class names.

Answer 3

This might not be a PIL issue. It might be related to your HTTP Server setting. HTTP servers put a limit on the size of the entity body that will be accepted.

For eg, in Apache FCGI, the option FcgidMaxRequestLen determines the maximum size of file that can be uploaded.

Check that for your server - it might be the one that is limiting the upload size.

Answer 4

I had to change the tds version to 7.2 to prevent this from happening. Also works with tds version 8.0, however I had some other issues with 8.0.