I have installed libffi on my Linux server as well as correctly set the PKG_CONFIG_PATH environment variable to the correct directory, as pip
libffi
PKG_CONFIG_PATH
pip
You need to install the development package for libffi.
On RPM based systems (Fedora, Redhat, CentOS etc) the package is named libffi-devel.
libffi-devel
Not sure about Debian/Ubuntu systems, I'm sure someone else will pipe up with that.