urllib and “SSL: CERTIFICATE_VERIFY_FAILED” Error

匿名 (未验证) 提交于 2019-12-03 01:29:01

问题:

I am getting this error

Exception in thread Thread-3: Traceback (most recent call last): File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 810, in        __bootstrap_inner self.run() File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/threading.py", line 763, in  run self.__target(*self.__args, **self.__kwargs) File "/Users/Matthew/Desktop/Skypebot 2.0/bot.py", line 271, in process info = urllib2.urlopen(req).read() File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 154, in urlopen return opener.open(url, data, timeout) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 431, in open response = self._open(req, data) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 449, in _open '_open', req) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 409, in _call_chain result = func(*args) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1240, in https_open context=self._context) File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib2.py", line 1197, in do_open raise URLError(err) URLError: 

This is the code that is causing this error:

if input.startswith("!web"):     input = input.replace("!web ", "")           url = "https://domainsearch.p.mashape.com/index.php?name=" + input     req = urllib2.Request(url, headers={ 'X-Mashape-Key': 'XXXXXXXXXXXXXXXXXXXX' })     info = urllib2.urlopen(req).read()     Message.Chat.SendMessage ("" + info) 

The API Im using requires me to use the https. How can I make it bypass the verification?

回答1:

If you just want to bypass verification, you can create a new SSLContext. By default newly created contexts use CERT_NONE.

Be careful with this as stated in section 17.3.7.2.1

When calling the SSLContext constructor directly, CERT_NONE is the default. Since it does not authenticate the other peer, it can be insecure, especially in client mode where most of time you would like to ensure the authenticity of the server you’re talking to. Therefore, when in client mode, it is highly recommended to use CERT_REQUIRED.

But if you just want it to work now for some other reason you can do the following, you'll have to import ssl as well:

input = input.replace("!web ", "")       url = "https://domainsearch.p.mashape.com/index.php?name=" + input req = urllib2.Request(url, headers={ 'X-Mashape-Key': 'XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' }) gcontext = ssl.SSLContext(ssl.PROTOCOL_TLSv1)  # Only for gangstars info = urllib2.urlopen(req, context=gcontext).read() Message.Chat.SendMessage ("" + info) 

This should get round your problem but you're not really solving any of the issues, but you won't see the [SSL: CERTIFICATE_VERIFY_FAILED] because you now aren't verifying the cert!

To add to the above, if you want to know more about why you are seeing these issues you will want to have a look at PEP 476.

This PEP proposes to enable verification of X509 certificate signatures, as well as hostname verification for Python's HTTP clients by default, subject to opt-out on a per-call basis. This change would be applied to Python 2.7, Python 3.4, and Python 3.5.

There is an advised opt out which isn't dissimilar to my advice above:

import ssl  # This restores the same behavior as before. context = ssl._create_unverified_context() urllib.urlopen("https://no-valid-cert", context=context) 

It also features a highly discouraged option via monkeypatching which you don't often see in python:

import ssl  ssl._create_default_https_context = ssl._create_unverified_context 

Which overrides the default function for context creation with the function to create an unverified context.

If you want to read a paper on why not validating certs is bad in software you can find it here!



回答2:

This isn't a solution to your specific problem, but I'm putting it here because this thread is the top Google result for "SSL: CERTIFICATE_VERIFY_FAILED", and it lead me on a wild goose chase.

If you have installed Python 3.6 on OSX and are getting the "SSL: CERTIFICATE_VERIFY_FAILED" error when trying to connect to an https:// site, it's probably because Python 3.6 on OSX has no certificates at all, and can't validate any SSL connections. This is a change for 3.6 on OSX, and requires a post-install step, which installs the certifi package of certificates. This is documented in the ReadMe, which you should find at /Applications/Python\ 3.6/ReadMe.rtf

The ReadMe will have you run this post-install script, which just installs certifi: /Applications/Python\ 3.6/Install\ Certificates.command

Release notes have some more info: https://www.python.org/downloads/release/python-360/



回答3:

On Windows, Python does not look at the system certificate, it uses its own located at ?\lib\site-packages\certifi\cacert.pem.

The solution to your problem:

  1. download the domain validation certificate as *.crt or *pem file
  2. open the file in editor and copy it's content to clipboard
  3. find your cacert.pem location: from requests.utils import DEFAULT_CA_BUNDLE_PATH; print(DEFAULT_CA_BUNDLE_PATH)
  4. edit the cacert.pem file and paste your domain validation certificate at the end of the file.
  5. Save the file and enjoy requests!


回答4:

You could try adding this to your environment variables:

PYTHONHTTPSVERIFY=0  


回答5:

Like I've written in a comment, this problem is probably related to this SO answer.

In short: there are multiple ways to verify the certificate. The verification used by OpenSSL is incompatible with the trusted root certificates you have on your system. OpenSSL is used by Python.

You could try to get the missing certificate for Verisign Class 3 Public Primary Certification Authority and then use the cafile option according to the Python documentation:

urllib2.urlopen(req, cafile="verisign.pem") 


回答6:

To expand on Craig Glennie's answer (sorry not enough reputation to comment):

in Python 3.6.1 on MacOs Sierra

Entering this in the bash terminal solved the problem:

pip install certifi /Applications/Python\ 3.6/Install\ Certificates.command 


回答7:

import requests requests.packages.urllib3.disable_warnings()  import ssl  try:     _create_unverified_https_context = ssl._create_unverified_context except AttributeError:     # Legacy Python that doesn't verify HTTPS certificates by default     pass else:     # Handle target environment that doesn't support HTTPS verification     ssl._create_default_https_context = _create_unverified_https_context 

Taken from here https://gist.github.com/michaelrice/a6794a017e349fc65d01



回答8:

For Python 3.4+ on Centos 6/7,Fedora, just install the trusted CA this way :

  1. Copy the CA.crt to /etc/pki/ca-trust/source/anchors/
  2. update-ca-trust force-enable
  3. update-ca-trust extract


回答9:

I was having a similar problem, though I was using urllib.request.urlopen in Python 3.4, 3.5, and 3.6. (This is a portion of the Python 3 equivalent of urllib2, per the note at the head of Python 2's urllib2 documentation page.)

My solution was to pip install certifi to install certifi, which has:

... a carefully curated collection of Root Certificates for validating the trustworthiness of SSL certificates while verifying the identity of TLS hosts.

Then, in my code where I previously just had:

import urllib.request as urlrq resp = urlrq.urlopen('https://foo.com/bar/baz.html') 

I revised it to:

import urllib.request as urlrq import certifi resp = urlrq.urlopen('https://foo.com/bar/baz.html', cafile=certifi.where()) 

If I read the urllib2.urlopen documentation correctly, it also has a cafile argument. So, urllib2.urlopen([...], certifi.where()) might work for Python 2.7 as well.



回答10:

Try

pip install --trusted-host pypi.python.org packagename

It worked for me.



回答11:

Like you, I am using python 2.7 on my old iMac (OS X 10.6.8), I met the problem too, using urllib2.urlopen :

urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] 

My programs were running fine without SSL certificate problems and suddently (after dowloading programs), they crashed with this SSL error.

The problem was the version of python used :

  1. No problem with https://www.python.org/downloads and python-2.7.9-macosx10.6.pkg

  2. problem with the one instaled by Homebrew tool : "brew install python", version located in /usr/local/bin.

A chapter, called Certificate verification and OpenSSL [CHANGED for Python 2.7.9], in /Applications/Python 2.7/ReadMe.rtf explains the problem with many details.

So, check, download and put in your PATH the right version of python.



回答12:

I hang my head in semi-shame, as I had the same issue, except that in my case, the URL I was hitting was valid, the certificate was valid. What wasn't valid was my connection out to the web. I had failed to add proxy details into the browser (IE in this case). This stopped the verification process from happening correctly.
Added in the proxy details and my python was then very happy .



回答13:

I need to add another answer because just like Craig Glennie, I went on a wild goose chase due to the many posts referring to this problem across the Web.

I am using MacPorts, and what I originally thought was a Python problem was in fact a MacPorts problem: it does not install a root certificate with its installation of openssl. The solution is to port install curl-ca-bundle, as mentioned in this blog post.



回答14:

Take a look at

/Applications/Python 3.6/Install Certificates.command

You can also go to Aplications ans click on Certificates.command



回答15:

I am surprised all these instruction didn't solved my problem. Nonetheless, the diagnostic is correct (BTW, I am using Mac and Python3.6.1). So, to summarize the correct part :

  • On Mac, Apple is dropping OpenSSL
  • Python now uses it own set of CA Root Certificate
  • Binary Python installation provided a script to install the CA Root certificate Python needs ("/Applications/Python 3.6/Install Certificates.command")
  • Read "/Applications/Python 3.6/ReadMe.rtf" for details

For me, the script doesn't work, and all those certifi and openssl installation failed to fix too. Maybe because I have multiple python 2 and 3 installations, as well as many virtualenv. At the end, I need to fix it by hand.

pip install certifi   # for your virtualenv mkdir -p /Library/Frameworks/Python.framework/Versions/3.6/etc/openssl cp -a /site-package/certifi/cacert.pem \   /Library/Frameworks/Python.framework/Versions/3.6/etc/openssl/cert.pem 

If that still fails you. Then re/install OpenSSL as well.

port install openssl 


回答16:

Python 2.7.12 (default, Jul 29 2016, 15:26:22) fixed the mentioned issue. This information might help someone else.



回答17:

installing steps for nltk (I had python3 (3.6.2) installed already in MAC OS X

sudo easy_install pip 

use ignore installed option to ignore uninstalling previous version of six, else, it gives an error while uninstalling and does not movie forward

sudo pip3 install -U nltk --ignore-installed six 

Check the installation of pip and python, use the '3' versions

which python python2 python3 which pip pip2 pip3 

Check if NLTK is installed

python3 import nltk nltk.__path__ ['/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/nltk'] 

Install SSL certificate prior to installing the examples book, else we will certificate error while installing the examples

/Applications/Python\ 3.6/Install\ Certificates.command python3 -m nltk.downloader book 

That completed the installation successfully of nltk and nltk_ata for book examples



回答18:

Installing PyOpenSSL using pip worked for me (without converting to PEM):

pip install PyOpenSSL 


回答19:

If your on vCenter 6, you should instead add your vCenter's vmware certificate authority cert to your OS's list of trusted CA's. To download your cert do the following

  1. Open your Web browser.
  2. Navigate to https://
  3. In the lower right-hand corner, click the Download Trusted Root CA link

On Fedora

  1. unzip and change the extension from .0 to .cer
  2. Copy it to the /etc/pki/ca-trust/source/anchors/
  3. run the update-ca-trust command.

Links:

  1. https://virtualizationreview.com/articles/2015/04/02/install-root-self-signed-certificate-vcenter-6.aspx?m=1
  2. http://forums.fedoraforum.org/showthread.php?t=293856


回答20:

ln -s /usr/local/share/certs/ca-root-nss.crt /etc/ssl/cert.pem

(FreeBSD 10.1)



易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!