I have a short code that uses the multiprocessing
package and works fine on my local machine.
When I uploaded to AWS Lambda
and run there,
As far as I can tell, multiprocessing won't work on AWS Lambda because the execution environment/container is missing /dev/shm
- see https://forums.aws.amazon.com/thread.jspa?threadID=219962 (login may be required).
No word (that I can find) on if/when Amazon will change this. I also looked at other libraries e.g. https://pythonhosted.org/joblib/parallel.html will fallback to /tmp
(which we know DOES exist) if it can't find /dev/shm
, but that doesn't actually solve the problem.