Parallel Pip install

Question

Our Django project is getting huge. We have hundreds of apps and use a ton of 3rd party python packages, many of which need to have C compiled. Our deployments are taking a

Answer 1

I come across with a similar issue and I ended up with the below:

cat requirements.txt | sed -e '/^\s*#.*$/d' -e '/^\s*$/d' | xargs -n 1 python -m pip install

That will read line by line the requirements.txt and execute pip. I cannot find from where I got the answer properly, so apologies for that, but I found some justification below:

1. How sed works: https://howto.lintel.in/truncate-empty-lines-using-sed/
2. Another similar answer but with git: https://stackoverflow.com/a/46494462/7127519

Hope this help with alternatives. I posted this solution here https://stackoverflow.com/a/63534476/7127519, so maybe there is some help there.