How can I install GraphicsMagick or ImageMagick on AWS Lambda?

Question

I am using the gm package for Node.js along with the default ImageMagick installation that is available on AWS Lambda.

const gm = require('gm').subClass({ imageMagick: true });

For some reason, the resize functionality fails for certain images.

I created an EC2 instance with Amazon Linux AMI (ami-hvm-2016.03.3.x86_64-gp2). I installed the (old) 6.x version of ImageMagick that is available from yum. When I run my script with that install on the EC2 instance, it reproduces the failure I see when the code runs on Lambda, confirming it is something with this version of IM that is causing the failure.

If I install GrpahicsMagick with sudo yum install GraphicsMagick. This allows my script to perform the resizes without error.

const gm = require('gm').subClass({ imageMagick: false });

However, I'm not sure how to bundle this in my deploy with serverless. If I install GraphicsMagick to the same folder as the script with sudo yum --installroot=/var/task install GraphicsMagick, and run my script using this require statement instead:

const gm = require('gm').subClass({ imageMagick: false, appPath: './usr/bin/' });

The resizes work when I run my script on the EC2 instance. But when I deploy with serverless, and the script runs in Lambda, the executable appears to be broken. gm fails with the following error on a call to gm(buffer).size(/*...*/).

could not get the image size: ERR: {"code":"EPIPE","errno":"EPIPE","syscall":"write"}

How can I build a version of ImageMagick or GraphicsMagick that can be deployed with serverless?

Answer 1

I spun up the latest aws linux and ran the commands below.

yum -y install gcc-c++ libpng-devel libjpeg-devel libtiff-devel wget
wget https://downloads.sourceforge.net/project/graphicsmagick/graphicsmagick/1.3.26/GraphicsMagick-1.3.26.tar.gz
tar zxvf GraphicsMagick-1.3.26.tar.gz
cd GraphicsMagick-1.3.26
./configure --prefix=/var/task/graphicsmagick --enable-shared=no --enable-static=yes
make
sudo make install
tar zcvf ~/graphicsmagick.tgz /var/task/graphicsmagick/

I scp the dir down into my local and threw it in the package to be zipped and deployed. My layout is similar to the aws repo code linked, but modified for serverless.

Lambda code:

// graphicsmagick dir is at the root of my project
const BIN_PATH = process.env['LAMBDA_TASK_ROOT'] + "/graphicsmagick/bin/";
const Gm = require('gm').subClass({ appPath: BIN_PATH });

// below is inside the handler
process.env['PATH'] = process.env['PATH'] + ':' + BIN_PATH;

serverless.yml

package:
  artifact: /path/to/function.zip

I use the artifact and build my own zip. If you run into the issue below I suggest you do that. https://github.com/serverless/serverless/issues/3215

# -y to keep the symlinks and thus reduce the size from 266M to 73M
cd lambda && zip -FS -q -r -y ../dist/function.zip *

Ideas grabbed from:

https://gist.github.com/bensie/56f51bc33d4a55e2fc9a

https://github.com/awslabs/serverless-image-resizing

Edit: Might want to also check out lambda layers. May only need to do this kind of thing once.

Answer 2

If you want to tackle image resizing, you may also take a look at the serverless sharp image library which uses Sharp, a high performance Node.js library for image resizing which is about 3x - 5x faster compared to GM/IM. You didn't provide enough informations to say that it fits your use case requirements but I just wanted to mention it since this library already saved me a lot of AWS Lambda costs so far.

By the way: I am not related to this project (but licences are MIT/Apache License 2.0 anyway).

Answer 3

All dependencies can be packed and uploaded as a part of your AWS Lambda function

You can mostly use any package you want from AWS Lambda, if you can fit it within the allowed size limits and upload the zip file. Take a look at the AWS Lambda Deployment Limits section

Also, here's an example of how to package dependencies (for python code) https://stackoverflow.com/a/36093281/358013

Answer 4

For node.js, you can use node-lambda, it simplifies packaging using a docker image :

node-lambda package -I lambci/lambda:build-nodejs6.10 -A . -x '*.lock *.zip'

The -I argument will launch a docker image and launch npm i in your project so it will compile the binary node_modules against the right architecture.

Answer 5

If you install Docker on your local device and add this command into your package.json.

"dockerbuild": "rm -rf node_modules/gm && docker run -v \"$PWD\":/var/task lambci/lambda:build-nodejs8.10 npm install"

Run npm run dockerbuild before you deploy your app.

You should change the node version based on the version of your lambda environment.