Efficiently generating thumbnails with ImageMagick and convert

Question

I\'m looking to efficiently generate various sized thumbnails with ImageMagick\'s convert utility in Python. Some of my image files are quite large (~15MB JPGs).

One

Answer 1

15MB JPGs are really large. I would first resize it to reasonable size (say 2500x2500) with fastest "-sample" parameter and this smaller image then resize to different thumbnails.

You can make intelligent decision based on image size and choose the right way of resize.

I would recommend to focus on thumbnail quality instead of conversion speed so please look at different filters (-filter), sharping (-unsharp) and recommended downsampling methods

Answer 2

I'm thumbnailing ~50MB JPG files. The one option which made the biggest difference (~5x speedup) was "-define jpeg:size 128x128" before the input filename. The example given here:

http://www.imagemagick.org/Usage/formats/#jpg_read

...made a huge difference:

convert -define jpeg:size=128x128 jpeg_large.jpg -thumbnail 64x64  jpeg_thumbnail.jpg

-define jpeg:size allows ImageMagick to read in only as much data as it needs from disk, which greatly reduces load time for very large images.

As the linked page suggests, use a jpeg:size= of twice your final thumbnail size to avoid aliasing.

The -thumbnail option, described here:

http://www.imagemagick.org/Usage/resize/#thumbnail

...samples and strips the image, further speeding up the process.

Answer 3

I tried timing vipsthumbnail against @KurtPfeifle's excellent answer. I ran this with a 10k x 10k pixel RGB JPG image (about 15MB):

convert                           \
  /data/john/pics/wtc.jpg         \
 -quality 80                      \
 -colorspace rgb                  \
 +profile '*'                     \
 -filter Lanczos                  \
 -write mpr:copy-of-huge-original \
 +delete                          \
  mpr:copy-of-huge-original -resize '1024x768'  -write sample-1024x768.jpg +delete \
  mpr:copy-of-huge-original -resize '800x600'   -write sample-800x600.jpg  +delete \
  mpr:copy-of-huge-original -resize '400x300'   -write sample-400x300.jpg  +delete \
  mpr:copy-of-huge-original -resize '200x150'   -write sample-200x150.jpg  +delete \
  mpr:copy-of-huge-original -resize '163x163!>' -write sample-163x163.jpg  x.jpg

I found I needed the extra x.jpg at the end, I'm not sure why. On this machine (E5-1650 3.2 GHz, IM 6.8.9-9) I see:

$ time ./m.sh 
real    0m6.560s
user    0m31.908s
sys     0m0.264s
peak RES 711MB

This is (I think) the equivalent with vipsthumbnail:

img=/data/john/pics/wtc.jpg
icc=/usr/share/color/argyll/ref/sRGB.icm

for size in 1024x768 800x600 400x300 200x150 163x163; do
  vipsthumbnail $img --size $size --eprofile $icc -o vips-sample-$size.jpg[Q=80]
done

vipsthumbnail defaults to Lanczos3. Timing it with vips-8.4.4 I see:

$ time ./n.sh
real    0m2.376s
user    0m2.412s
sys     0m0.108s
peak RES 70MB

So a useful speedup, and a large drop in memory use.

Because memory use is low, you can run many vipsthumbnail in parallel without killing your server. If I change the script to be:

img=/data/john/pics/wtc.jpg
icc=/usr/share/color/argyll/ref/sRGB.icm

parallel vipsthumbnail $img \
  --size {} --eprofile $icc -o vips-sample-{}.jpg[Q=80] ::: \
  1024x768 800x600 400x300 200x150 163x163

I now see:

$ time ./n.sh 
real    0m0.593s
user    0m1.960s
sys     0m0.100s
peak RES 280MB

More than 10x faster than ImageMagick.

Answer 4

For my point of view, and after testing, resizing 1024x768 to 800x600 is bad for rescaling algorithm. The next resize are more easier, because of multiple of integer (2).

So, for quality reason, I personnaly thing, this is better :

convert sample_image.jpg -resize 1024x768  sample_image-1024x768.jpg
convert sample_image.jpg -resize 800x600   sample_image-800x600.jpg
convert sample_image-800x600.jpg   -resize 400x300   sample_image-400x300.jpg
convert sample_image-400x300.jpg   -resize 200x150   sample_image-200x150.jpg

Answer 5

ImageMagick has a few tricks up its sleeves which help you to optimize for speed when you want to process large images and when you want to create different output from the same original:

1. Make use of ImageMagick's mpr:{name} feature, which makes it temporarily save the input image into a named memory program register, from which you can later (while processing) read the data much faster than you could do from harddisk.

2. Do all resize operations in one single process writing out the different output sizes you require.

And the even better news is you can combine both these into one single command.

So you do not need to run multiple processes with all their context-switching overhead -- do it all in one go.

The following example also crops two separate areas from the original image and creates re-sized thumbnails from them, just to show how many different operations IM can do in one commandline. It also, of course outputs the sizes you requested. (You'll need, of course, a really large-dimensioned input image for the cropping parameters to work).

convert                           \
  huge-original.jpg               \
 -quality 80                      \
 -colorspace rgb                  \
 +profile '*'                     \
 -filter Lanczos                  \
 -write mpr:copy-of-huge-original \
 +delete                          \
  mpr:copy-of-huge-original -crop '3000x2000+0+480'   -resize '200x125!>' -write thumb1-extract.jpg +delete \
  mpr:copy-of-huge-original -crop '2000x1500+280+220' -resize '75x75!>'   -write thumb2-extract.jpg +delete \
  mpr:copy-of-huge-original -resize '1024x768'  -write sample-1024x768.jpg +delete \
  mpr:copy-of-huge-original -resize '800x600'   -write sample-800x600.jpg  +delete \
  mpr:copy-of-huge-original -resize '400x300'   -write sample-400x300.jpg  +delete \
  mpr:copy-of-huge-original -resize '200x150'   -write sample-200x150.jpg  +delete \
  mpr:copy-of-huge-original -resize '163x163!>' -write sample-163x163.jpg

---

Update

I only now saw the question asked by @JonathanOng: How to stream the output to <stdout>?

Assuming, you want the format going to stdout also be JPEG, you can try this:

convert                           \
  huge-original.jpg               \
 -quality 80                      \
 -colorspace rgb                  \
 +profile '*'                     \
 -filter Lanczos                  \
 +write mpr:copy-of-huge-original \
  mpr:copy-of-huge-original -crop '3000x2000+0+480'   -resize '200x125!>' +write thumb1-extract.jpg \
  mpr:copy-of-huge-original -crop '2000x1500+280+220' -resize '75x75!>'   +write thumb2-extract.jpg \
  mpr:copy-of-huge-original -resize '1024x768'  +write jpeg:- \
  mpr:copy-of-huge-original -resize '800x600'   +write jpeg:- \
  mpr:copy-of-huge-original -resize '400x300'   +write jpeg:- \
  mpr:copy-of-huge-original -resize '200x150'   +write jpeg:- \
  mpr:copy-of-huge-original -resize '163x163!>' +write jpeg:-

This way each variant will go to stdout. How you deal with this stream of consecutive images then, is up to you...

Note, instead of writing -write filename +delete you can use +write filename. It amounts to the same effect.