How to add a Spacy model to a requirements.txt file?

别来无恙 提交于 2020-12-05 10:24:03

问题


I have an app that uses the Spacy model "en_core_web_sm". I have tested the app on my local machine and it works fine.

However when I deploy it to Heroku, it gives me this error:

"Can't find model 'en_core_web_sm'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory."

My requirements file contains spacy==2.2.4.

I have been doing some research on this error and found that the model needs to be downloaded separately using this command: python -m spacy download en_core_web_sm

I have been looking for ways to add the same to my requirements.txt file but haven't been able to find one that works!

I tried this as well - added the below to the requirements file:

-e git://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz#egg=en_core_web_sm==2.2.0

but it gave this error:

"Cloning git://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz to /app/.heroku/src/en-core-web-sm

Running command git clone -q git://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz /app/.heroku/src/en-core-web-sm fatal: remote error: explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz is not a valid repository name"

Is there a way to get this Spacy model to load from the requirements file? Or any other fix that is possible?

Thank you.


回答1:


Add this in your deployment step, if using docker add in Dockerfile

pip3 install https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.5/en_core_web_sm-2.2.5.tar.gz --user

EDIT

Add

spacy>=2.2.0,<3.0.0 https://github.com/explosion/spacy-models/releases/download/en_core_web_sm-2.2.0/en_core_web_sm-2.2.0.tar.gz#egg=en_core_web_sm

in requirements.txt

Spacy Doc Refer Downloading and requiring model dependencies section

For more detail on how to add github-source see this and follow YPCrumble answer




回答2:


Ok, so after some more Googling and hunting for a solution, I found this solution that worked:

I downloaded the tarball from the url that @tausif shared in his answer, to my local system.

Saved it in the directory which had my requirements.txt file.

Then I added this line to my requirements.txt file: ./en_core_web_sm-2.2.5.tar.gz

Proceeded with deploying to Heroku - it succeeded and the app works perfectly now.



来源:https://stackoverflow.com/questions/61702357/how-to-add-a-spacy-model-to-a-requirements-txt-file

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!