Python's requests “Missing dependencies for SOCKS support” when using SOCKS5 from Terminal

Question

I'm trying to interact with an API from my Python 2.7 shell using a package that relies on Python's requests. Thing is the remote address is blocked by my network (university library).

So to speak to the API I do the following:

~$ ssh -D 8080 name@myserver.com

And then, in new terminal, in local computer:

~$ export http_proxy=socks5://127.0.0.1:8080 https_proxy=socks5://127.0.0.1:8080

Then I run the program in Python console but fails:

~$ python
>>> import myscript
>>> id = '1213'
>>> token = 'jd87jd9'
>>> connect(id,token)

File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 518, in post
    return self.request('POST', url, data=data, json=json, **kwargs)
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 475, in request
    resp = self.send(prep, **send_kwargs)
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/sessions.py", line 585, in send
    r = adapter.send(request, **kwargs)
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 370, in send
    conn = self.get_connection(request.url, proxies)
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 273, in get_connection
    proxy_manager = self.proxy_manager_for(proxy)
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 169, in proxy_manager_for
    **proxy_kwargs
  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 43, in SOCKSProxyManager
    raise InvalidSchema("Missing dependencies for SOCKS support.")
requests.exceptions.InvalidSchema: Missing dependencies for SOCKS support.

This excerpt is from the adapters.py requests module:

> try:
>     from .packages.urllib3.contrib.socks import SOCKSProxyManager except ImportError:
>     def SOCKSProxyManager(*args, **kwargs):
>         raise InvalidSchema("Missing dependencies for SOCKS support.")

Now problem seems to be originated in urllib3's SOCKSProxyManager.

So I read you can use SOCKSProxyManager with SOCKS5 if you have install PySocks or you do a pip install urllib3[socks]

Alas, I tried both PySocks and urllib3 with Socks without any success.

Any idea of another workaround?

EDIT:

I also tried pip install requests[socks] (that's requests 2.10.0 with Socks support) and I am getting this:

  File "/home/username/.virtualenvs/venv/local/lib/python2.7/site-packages/requests/adapters.py", line 467, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: SOCKSHTTPSConnectionPool(host='api-server.com', port=443): Max retries exceeded with url: /auth (Caused by NewConnectionError('<requests.packages.urllib3.contrib.socks.SOCKSHTTPSConnection object at 0x95c7ccc>: Failed to establish a new connection: SOCKS5 proxy server sent invalid data',))

Answer 1

I had the same issue with conda and requests 2.11 (I work in a Ubuntu VM behind a corporate proxy).

This issue helped me. I changed my environment variable all_proxy (which was originally set to a SOCK proxy socks://....) to the https version in my .bashrc file :

export all_proxy="https://<proxy>:<port>/"

and now it works.

Answer 2

This means that requests is using socks as a proxy and that socks is not installed.

Just run pip install pysocks

Answer 3

I added the requests[socks]>=2.10.0 to my requirements.txt, updated my https_proxy env variable, and came across the above error. I then tried a regular pip install requests[socks] after resetting the https_proxy env variable and PySocks was installed. I'm not sure why the pip install -Ur requirements.txt failed to install PySocks the first time.

After that, I was able to make a request in python using the socks proxy.

It looks like your socks server is not behaving. I would see if you, or your admin, could watch the logs and see what the machine is complaining about.

Answer 4

In Ubuntu you can run :
unset all_proxy && unset ALL_PROXY

Answer 5

I also stumbled upon this issue while doing a simple pip install -U pip, but information I found from your question helped me resolved my issue. I am on Mac OS X.

As you have pointed out, adapters.py from the requests package was trying to do this:

try:
    from .packages.urllib3.contrib.socks import SOCKSProxyManager
except ImportError:
    def SOCKSProxyManager(*args, **kwargs):
        raise InvalidSchema("Missing dependencies for SOCKS support.")

So it seems sensible to look for the place of definition of SOCKSProxyManager. It seems to be in a "contrib" module in urllib3 and not installed alongside urllib3 by default. The docstring of that module says:

This module contains provisional support for SOCKS proxies from within
urllib3. This module supports SOCKS4 (specifically the SOCKS4A variant) and
SOCKS5. To enable its functionality, either install PySocks or install this
module with the ``socks`` extra.

The instructions of the pip docs says this about setuptools extras:

6. Install a package with setuptools extras.

$ pip install SomePackage[PDF]
$ pip install git+https://git.repo/some_pkg.git#egg=SomePackage[PDF]
$ pip install SomePackage[PDF]==3.0
$ pip install -e .[PDF]==3.0  # editable project in current directory

So I followed the instructions and did:

$ pip install 'urllib3[socks]'

I then continued with pip install -U pip, which was what I was supposed to be doing, and now it works.

I wonder how many people got tricked-up by the square brackets, as Bash and other shells often treat it as a special character, which needs to be escaped for it to reach the invoked program (in this case, pip).

Answer 6

I got the same error just a few mins ago.Then re-installed request[socks] via pip. It seems that there was a missing part of socks which is win-inet_pton. Pip installed it and the problem is solved.

Answer 7

Just unset your all_proxy environment variable, and this should work. Also you can refer to this issue in github.

On Ubuntu you can use the following command unset all_proxy and restart terminal

Answer 8

My environment is Ubuntu 16.4 LTS and Python3.5.2 I use pip3 to install libs got the same issue. so I use the command unset ALL_PROXY to solve this problem and it works.

PS: use printenv | grep -i proxy to show the proxy info.