How to enable FIPS mode for libcrypto and libssl packaged with Python?

Question

I have a python application which comes packaged with Python and Libcrypto and LibSSL shared objects. The application was built with Openssl Fips Module 2.0. These shared ob

Answer 1

I've built the OpenSSL-fips module using regular flags (e.g.: no-asm, shared, some ancient ciphers disabled):

[cfati@cfati-ubtu16x64-0:~/Work/Dev/StackOverflow/q049320993]> ~/sopr.sh
*** Set shorter prompt to better fit when pasted in StackOverflow (or other) pages ***

[064bit-prompt]> ls ssl/build/bin ssl/build/lib
ssl/build/bin:
c_rehash  openssl

ssl/build/lib:
engines  libcrypto.a  libcrypto.so  libcrypto.so.1.0.0  libssl.a  libssl.so  libssl.so.1.0.0  pkgconfig

And started playing a little bit with it:

[064bit-prompt]> ssl/build/bin/openssl version
OpenSSL 1.0.2h-fips  3 May 2016 (Library: OpenSSL 1.0.2g  1 Mar 2016)

Note the "(Library: OpenSSL 1.0.2g 1 Mar 2016)" part. That (being present) states that the openssl executable is OK (expected version), but it's using a wrong libcrypto (it's the one that comes installed by default on the system - under /lib - and typically that one isn't built with FIPS support).
It must load our libraries, and that is done by setting LD_LIBRARY_PATH (the same behavior could have also been achieved by setting an env var when building OpenSSL that would have set the rpath in the openssl executable, but I forgot, and I didn't want to build it again):

[064bit-prompt]> LD_LIBRARY_PATH=ssl/build/lib ssl/build/bin/openssl version
OpenSSL 1.0.2h-fips  3 May 2016

Now, that the setup is successful, let's dive into OPENSSL_FIPS env var:

[064bit-prompt]> LD_LIBRARY_PATH=ssl/build/lib ssl/build/bin/openssl md5 ./code.py
MD5(./code.py)= d41d8cd98f00b204e9800998ecf8427e
[064bit-prompt]> LD_LIBRARY_PATH=ssl/build/lib ssl/build/bin/openssl sha1 ./code.py
SHA1(./code.py)= da39a3ee5e6b4b0d3255bfef95601890afd80709
[064bit-prompt]> OPENSSL_FIPS=1 LD_LIBRARY_PATH=ssl/build/lib ssl/build/bin/openssl sha1 ./code.py
SHA1(./code.py)= da39a3ee5e6b4b0d3255bfef95601890afd80709
[064bit-prompt]> OPENSSL_FIPS=1 LD_LIBRARY_PATH=ssl/build/lib ssl/build/bin/openssl md5 ./code.py
Error setting digest md5
139778679649944:error:060A80A3:digital envelope routines:FIPS_DIGESTINIT:disabled for fips:fips_md.c:180:

As seen from above, the md5 hash behavior is influenced by the OPENSSL_FIPS env var (when FIPS mode is on, its usage is not allowed).

Notes:

• Most likely, newer openssl-fips versions will also have sha1 disabled since it's considered weak, so the invariant should be switched to one of the sha2 hash functions family (e.g. sha256) or even better, sha3 (older OpenSSL versions might not have it)
• From my PoV this is a little too restrictive, as there might be cases when a hashing algorithm is needed for purposes that don't care about security, and more complex (and also time consuming) allowed algorithms still have to be used

Since OPENSSL_FIPS env var is handled at openssl executable level, which will be bypassed (as libcrypto will be used directly), it's no use for the current situation, so we have to go deeper. These are the functions that control FIPS mode in a loaded libcrypto instance:

• [OpenSSL.Wiki]: FIPS mode()
• [OpenSSL.Wiki]: FIPS mode set()

They will be used to read/write FIPS mode. In order to test whether FIPS mode is really set, md5 hash (from the example above) will be used.

code.py:

#!/usr/bin/env python3


import sys
import ssl
import ctypes


libcrypto = ctypes.CDLL("libcrypto.so.1.0.0")

fips_mode = libcrypto.FIPS_mode
fips_mode.argtypes = []
fips_mode.restype = ctypes.c_int

fips_mode_set = libcrypto.FIPS_mode_set
fips_mode_set.argtypes = [ctypes.c_int]
fips_mode_set.restype = ctypes.c_int

text = b""


if __name__ == "__main__":
    print("Python {:s} on {:s}\n".format(sys.version, sys.platform))
    print("OPENSSL_VERSION: {:s}".format(ssl.OPENSSL_VERSION))
    enable_fips = len(sys.argv) > 1

    print("FIPS_mode(): {:d}".format(fips_mode()))
    if enable_fips:
        print("FIPS_mode_set(1): {:d}".format(fips_mode_set(1)))
    print("FIPS_mode(): {:d}".format(fips_mode()))

    import hashlib
    print("SHA1: {:s}".format(hashlib.sha1(text).hexdigest()))
    print("MD5: {:s}".format(hashlib.md5(text).hexdigest()))

Notes:

• Setting the argtypes and restype for the 2 functions like specified in [Python 3.Docs]: ctypes - A foreign function library for Python
• The md5 hashing algorithm is provided at Python level by [Python 3.Docs]: hashlib - Secure hashes and message digests
• Important: the import hashlib statement is located after setting the FIPS mode (and not at the file beginning, as it should be), because hashlib does some caching at import time, so it captures the FIPS value at import time, and doesn't care if it changes afterwards

Output:

[064bit-prompt]> LD_LIBRARY_PATH=ssl/build/lib ./code.py
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux

OPENSSL_VERSION: OpenSSL 1.0.2h-fips  3 May 2016
FIPS_mode(): 0
FIPS_mode(): 0
SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
MD5: d41d8cd98f00b204e9800998ecf8427e
[064bit-prompt]> LD_LIBRARY_PATH=ssl/build/lib ./code.py 1
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux

OPENSSL_VERSION: OpenSSL 1.0.2h-fips  3 May 2016
FIPS_mode(): 0
FIPS_mode_set(1): 1
FIPS_mode(): 1
SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
fips_md.c(149): OpenSSL internal error, assertion failed: Digest Final previous FIPS forbidden algorithm error ignored
Aborted (core dumped)

As seen, setting FIPS mode via ctypes, really sets it.
I don't know why it segfaults, but the md5 related code is there only for testing purposes, so it's not needed in production.

I remember that on some Lnx version (might be RH based), FIPS mode could also be set (globally for the system), by editing some entry (under /proc ?), but I can't remember it.

A more elegant approach would be to expose Python wrappers for the 2 functions.
Check [Python.Bugs]: FIPS_mode() and FIPS_mode_set() functions in Python (ssl), I've also submitted a patch for Python 3.4 (where they were exposed by the ssl module), but it was rejected based on the following arguments (out of which the 1^st 2 are relevant):

1. FIPS is a bad standard
2. OpenSSL will drop support for it
3. It breaks up generality

You can apply it to Python 3.6 (I don't think it will work OOTB, since line numbers most likely changed), and (obviously) you'll have to build Python from sources.

Bottom line:

• There's a big difference between FIPS working and FIPS validated, as I'm sure you've read on [OpenSSL]: User Guide for the OpenSSL FIPS Object Module v2.0
• [AskUbuntu]: Enable FIPS 140-2 in ubuntu might also contain some useful info

---

Update #0

It just stroke me, the behavior that you're encountering on [SO]: Not able to call FIPS_mode_set() of libcrypto.so with Python ctypes [duplicate] might also be related to the wrong libcrypto being loaded (check the openssl version tests w/wo LD_LIBRARY_PATH from the beginning).
A non FIPS capable OpenSSL will still export the 2 functions, but they both simply return 0.

[064bit-prompt]> ./code.py 1
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux

OPENSSL_VERSION: OpenSSL 1.0.2g  1 Mar 2016
FIPS_mode(): 0
FIPS_mode_set(1): 0
FIPS_mode(): 0
SHA1: da39a3ee5e6b4b0d3255bfef95601890afd80709
MD5: d41d8cd98f00b204e9800998ecf8427e

So, make sure to load the correct libraries by specifying LD_LIBRARY_PATH ! (there are other ways, but this is the most straightforward one).