How to install PyTorch with pipenv and save it to Pipfile and Pipfile.lock?

问题

I’m currently using Pipenv to maintain the Python packages used in a specific project. Most of the downloads I’ve tried so far have worked as intended; that is, I enter pipenv install [package] and it installs the package into the virtual environment, then records the package information into both the Pipfile and Pipfile.lock.

However, I’m running into some problems installing PyTorch.

I’ve tried running pipenv install torch, but every time the locking step fails. Instead, I’ve tried forcing a download directly from the PyTorch website using

pipenv run pip install torch===1.6.0 torchvision===0.7.0 -f https://download.pytorch.org/whl/torch_stable.html

And it actually installs! If I run pipenv graph it displays both torch and torchvision with their dependencies. But one problem remains: neither torch nor torchvision are being saved into Pipfile and Pipfile.lock.

Any idea on how I can make this happen?

回答1:

When you use pipenv run pip install <package>, that skips the custom pipenv operations of updating the Pipfile and the Pipfile.lock. It is basically equivalent to doing a plain pip install <package> as if you did not have/use pipenv.

The only way to also update the Pipfiles is to use pipenv install.

Unfortunately, as I'm posting this, pipenv does not have an equivalent for pip's -f/--find-links option. One workaround is to manually find the correct torch wheel (.whl) links you need, which with pytorch, usually means looking for the correct link from https://download.pytorch.org/whl/torch_stable.html (I'll show a tip how to do this below).

Then, create/modify the Pipfile with the specific package versions and URLs to the wheels:

[[source]]
name = "pypi"
url = "https://pypi.org/simple"
verify_ssl = true

[requires]
python_version = "3.8"

[packages]
torch = {version = "==1.6.0", file = "https://download.pytorch.org/whl/cpu/torch-1.6.0-cp38-none-macosx_10_9_x86_64.whl"}
torchvision = {version = "==0.7.0", file = "https://download.pytorch.org/whl/torchvision-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl"}

Then just do normal pipenv install.

You can confirm the installation with pipenv install --verbose:

Collecting torch==1.6.0
  ...
  Looking up "https://download.pytorch.org/whl/cpu/torch-1.6.0-cp38-none-macosx_10_9_x86_64.whl" in the cache
  Current age based on date: 8
  Starting new HTTPS connection (1): download.pytorch.org:443
  https://download.pytorch.org:443 "GET /whl/cpu/torch-1.6.0-cp38-none-macosx_10_9_x86_64.whl HTTP/1.1" 304 0
  ...
  Added torch==1.6.0 from https://download.pytorch.org/whl/cpu/torch-1.6.0-cp38-none-macosx_10_9_x86_64.whl#egg=torch 
...
Successfully installed torch-1.6.0

Collecting torchvision==0.7.0
  ...
  Looking up "https://download.pytorch.org/whl/torchvision-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl" in the cache
  Current age based on date: 8
  Starting new HTTPS connection (1): download.pytorch.org:443
  https://download.pytorch.org:443 "GET /whl/torchvision-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl HTTP/1.1" 304 0
  ...
  Added torchvision==0.7.0 from https://download.pytorch.org/whl/torchvision-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl#egg=torchvision
...
Successfully installed torchvision-0.7.0

This also adds entries to Pipfile.lock:

"torch": {
    "file": "https://download.pytorch.org/whl/cpu/torch-1.6.0-cp38-none-macosx_10_9_x86_64.whl",
    "hashes": [
        ...
    ],
    "index": "pypi",
    "version": "==1.6.0"
},
"torchvision": {
    "file": "https://download.pytorch.org/whl/torchvision-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl",
    "hashes": [
        ...
    ],
    "index": "pypi",
    "version": "==0.7.0"
}

With that, you now have a Pipfile and Pipfile.lock that you can check-in/commit to version control and track/manage as you develop your application.

Instead of manually editing the Pipfile, you can also do it from the command line:

(temp) $ pipenv install --verbose "https://download.pytorch.org/whl/torchvision-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl"
Installing https://download.pytorch.org/whl/torchvision-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl...
...
Adding torchvision to Pipfile's [packages]...
✔ Installation Succeeded

That should also add an entry to the Pipfile:

[packages]
...
torchvision = {file = "https://download.pytorch.org/whl/torchvision-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl"}

Of course, this all depends on finding out which wheel you actually need. This can be done by first using a plain pip install <package> with the -f/--find-links option targeting the https://download.pytorch.org/whl/torch_stable.html URL, then checking which wheel it used.

First, let's get the correct .whl file with pip install

$ pipenv run pip install --verbose torchvision==0.7.0 -f https://download.pytorch.org/whl/torch_stable.html
Looking in links: https://download.pytorch.org/whl/torch_stable.html
...
Collecting torchvision==0.7.0
  Downloading torchvision-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl (387 kB)
...

Remove pip install-ed things from the virtual environment
```
$ pipenv clean
```
Repeat installation but using pipenv install
```
$ pipenv install --verbose "https://download.pytorch.org/whl/torchvision-0.7.0-cp38-cp38-macosx_10_9_x86_64.whl"
```
- Just combine "https://download.pytorch.org/whl/" + .whl filename from step 1

It might seem a bit backwards using pip install first then copying it over to pipenv, but the objective here is to let pipenv update the Pipfile and Pipfile.lock (to support deterministic builds) and to "document" your env for version control.

来源：https://stackoverflow.com/questions/63974588/how-to-install-pytorch-with-pipenv-and-save-it-to-pipfile-and-pipfile-lock

标签

python

windows

pytorch

pipenv

torchvision