Python - requests.exceptions.SSLError - dh key too small

Question

I\'m scraping some internal pages using Python and requests. I\'ve turned off SSL verifications and warnings.

requests.packages.urllib3.disable_warnings()
p         


        

Answer 1

this is not an extra answer just try to combine the solution code from question with extra information So others can copy it directly without extra try

It is not only a DH Key issues in server side, but also lots of different libraries are mismatched in python modules.

Code segment below is used to ignore those securitry issues because it may be not able be solved in server side. For example if it is internal legacy server, no one wants to update it.

Besides the hacked string for 'HIGH:!DH:!aNULL', urllib3 module can be imported to disable the warning if it has

import requests
import urllib3

requests.packages.urllib3.disable_warnings()
requests.packages.urllib3.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
try:
    requests.packages.urllib3.contrib.pyopenssl.util.ssl_.DEFAULT_CIPHERS += ':HIGH:!DH:!aNULL'
except AttributeError:
    # no pyopenssl support used / needed / available
    pass

page = requests.get(url, verify=False)

Answer 2

I had the same issue.

And it was fixed by commenting

CipherString = DEFAULT@SECLEVEL=2

line in /etc/ssl/openssl.cnf .

Answer 3

I will package my solution here. I had to modify the python SSL library, which was possible since I was running my code within a docker container, but it's something that probably you don't want to do.

1. Get the supported cipher of your server. In my case was a third party e-mail server, and I used script described list SSL/TLS cipher suite

check_supported_ciphers.sh

#!/usr/bin/env bash

# OpenSSL requires the port number.
SERVER=$1
DELAY=1
ciphers=$(openssl ciphers 'ALL:eNULL' | sed -e 's/:/ /g')

echo Obtaining cipher list from $(openssl version).

for cipher in ${ciphers[@]}
do
echo -n Testing $cipher...
result=$(echo -n | openssl s_client -cipher "$cipher" -connect $SERVER 2>&1)
if [[ "$result" =~ ":error:" ]] ; then
  error=$(echo -n $result | cut -d':' -f6)
  echo NO \($error\)
else
  if [[ "$result" =~ "Cipher is ${cipher}" || "$result" =~ "Cipher    :" ]] ; then
    echo YES
  else
    echo UNKNOWN RESPONSE
    echo $result
  fi
fi
sleep $DELAY
done

Give it permissions:

chmod +x check_supported_ciphers.sh

And execute it:

./check_supported_ciphers.sh myremoteserver.example.com | grep OK

After some seconds you will see an output similar to:

Testing AES128-SHA...YES (AES128-SHA_set_cipher_list)

So will use "AES128-SHA" as SSL cipher.

2. Force the error in your code:

   Traceback (most recent call last): File "my_custom_script.py", line 52, in imap = IMAP4_SSL(imap_host) File "/usr/lib/python2.7/imaplib.py", line 1169, in init IMAP4.init(self, host, port) File "/usr/lib/python2.7/imaplib.py", line 174, in init self.open(host, port) File "/usr/lib/python2.7/imaplib.py", line 1181, in open self.sslobj = ssl.wrap_socket(self.sock, self.keyfile, self.certfile) File "/usr/lib/python2.7/ssl.py", line 931, in wrap_socket ciphers=ciphers) File "/usr/lib/python2.7/ssl.py", line 599, in init self.do_handshake() File "/usr/lib/python2.7/ssl.py", line 828, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: DH_KEY_TOO_SMALL] dh key too small (_ssl.c:727)

3. Get the python SSL library path used, in this case:

   /usr/lib/python2.7/ssl.py

4. Edit it:

   cp /usr/lib/python2.7/ssl.py /usr/lib/python2.7/ssl.py.bak

   vim /usr/lib/python2.7/ssl.py

And replace:

_DEFAULT_CIPHERS = (
    'ECDH+AESGCM:ECDH+CHACHA20:DH+AESGCM:DH+CHACHA20:ECDH+AES256:DH+AES256:'
    'ECDH+AES128:DH+AES:ECDH+HIGH:DH+HIGH:RSA+AESGCM:RSA+AES:RSA+HIGH:'
    '!aNULL:!eNULL:!MD5:!3DES'
    )

By:

_DEFAULT_CIPHERS = (
    'AES128-SHA'
    )

Answer 4

Disabling warnings or certificate validation will not help. The underlying problem is a weak DH key used by the server which can be misused in the Logjam Attack.

To work around this you need to chose a cipher which does not make any use of Diffie Hellman Key Exchange and thus is not affected by the weak DH key. And this cipher must be supported by the server. It is unknown what the server supports but you might try with the cipher AES128-SHA or a cipher set of HIGH:!DH:!aNULL

Using requests with your own cipher set is tricky. See Why does Python requests ignore the verify parameter? for an example.

Answer 5

On CentOS 7, search for the following content in /etc/pki/tls/openssl.cnf:

[ crypto_policy ]
.include /etc/crypto-policies/back-ends/opensslcnf.config  
[ new_oids ]  

Set 'ALL:@SECLEVEL=1' in /etc/crypto-policies/back-ends/opensslcnf.config.

Answer 6

Someone from the requests python library's core development team has documented a recipe to keep the changes limited to one or a few servers:

https://lukasa.co.uk/2017/02/Configuring_TLS_With_Requests/

If your code interacts with multiple servers, it makes sense not to lower the security requirements of all connections because one server has a problematic configuration.

The code worked for me out of the box. That is, using my own value for CIPHERS, 'ALL:@SECLEVEL=1'.