Scrapy Installation Fails with error 'cannot open include: 'openssl/aes.h '

Question

I am trying to install Scrapy with easy_install -U Scrapy but it ends up in a strange error "Can not open include file " while trying to install it. Does any one know what is going on? Here is my complete traceback:

C:\Users\Mubashar Kamran>easy_install -U Scrapy
Searching for Scrapy
Reading https://pypi.python.org/simple/Scrapy/
Best match: scrapy 0.24.4
Processing scrapy-0.24.4-py2.7.egg
scrapy 0.24.4 is already the active version in easy-install.pth
Installing scrapy-script.py script to C:\Python27\Scripts
Installing scrapy.exe script to C:\Python27\Scripts
Installing scrapy.exe.manifest script to C:\Python27\Scripts

Using c:\python27\lib\site-packages\scrapy-0.24.4-py2.7.egg
Processing dependencies for Scrapy
Searching for cryptography>=0.2.1
Reading https://pypi.python.org/simple/cryptography/
Best match: cryptography 0.5.4
Downloading https://pypi.python.org/packages/source/c/cryptography/cryptography-
0.5.4.tar.gz#md5=4fd1f10e9f99009a44667fabe7980aec
Processing cryptography-0.5.4.tar.gz
Writing c:\users\mubash~1\appdata\local\temp\easy_install-jjms3i\cryptography-0.
5.4\setup.cfg
Running cryptography-0.5.4\setup.py -q bdist_egg --dist-dir c:\users\mubash~1\ap
pdata\local\temp\easy_install-jjms3i\cryptography-0.5.4\egg-dist-tmp-ry6bwd
C:\Python27\lib\distutils\dist.py:267: UserWarning: Unknown distribution option:
 'setup_requires'
  warnings.warn(msg)
_Cryptography_cffi_684bb40axf342507b.c
   Creating library c:\users\mubash~1\appdata\local\temp\easy_install-jjms3i\cry
ptography-0.5.4\cryptography\hazmat\primitives\__pycache__\Release\cryptography\
hazmat\primitives\__pycache__\_Cryptography_cffi_684bb40axf342507b.lib and objec
t c:\users\mubash~1\appdata\local\temp\easy_install-jjms3i\cryptography-0.5.4\cr
yptography\hazmat\primitives\__pycache__\Release\cryptography\hazmat\primitives\
__pycache__\_Cryptography_cffi_684bb40axf342507b.exp
_Cryptography_cffi_8f86901cxc1767c5a.c
   Creating library c:\users\mubash~1\appdata\local\temp\easy_install-jjms3i\cry
ptography-0.5.4\cryptography\hazmat\primitives\__pycache__\Release\cryptography\
hazmat\primitives\__pycache__\_Cryptography_cffi_8f86901cxc1767c5a.lib and objec
t c:\users\mubash~1\appdata\local\temp\easy_install-jjms3i\cryptography-0.5.4\cr
yptography\hazmat\primitives\__pycache__\Release\cryptography\hazmat\primitives\
__pycache__\_Cryptography_cffi_8f86901cxc1767c5a.exp
_Cryptography_cffi_4ed9e37dx4000d087.c
cryptography\hazmat\bindings\__pycache__\_Cryptography_cffi_4ed9e37dx4000d087.c(
194) : fatal error C1083: Cannot open include file: 'openssl/aes.h': No such fil
e or directory
error: c:\users\mubash~1\appdata\local\temp\easy_install-jjms3i\cryptography-0.5
.4\cryptography\hazmat\primitives\__pycache__\_Cryptography_cffi_684bb40axf34250
7b.pyd: Access is denied

Answer 1

I got same error installing different python app. I was missing OpenSSL dev package, solved by:

sudo apt-get install libssl-dev

Answer 2

On OSX

brew install openssl and then possibly brew link openssl --force if you are informed that links were not created.

Install Scrapy using the following command

env CRYPTOGRAPHY_OSX_NO_LINK_FLAGS=1 LDFLAGS="$(brew --prefix openssl)/lib/libssl.a $(brew --prefix openssl)/lib/libcrypto.a" CFLAGS="-I$(brew --prefix openssl)/include" pip install scrapy

You can, if you wish, substitute openssl for libressl.

Answer 3

It is enough to install openssl

brew install openssl

and export

export LDFLAGS=-L/usr/local/opt/openssl/lib
export CPPFLAGS=-I/usr/local/opt/openssl/include

in ~/.bash_profile. Try to avoid linking with --force parameter because it can lead to the conflicts with system libraries.

Answer 4

To All who end up in the errors while installing scrapy should follow http://community.spiceworks.com/how_to/38490-setting-up-scrapy-on-windows-7. This link describes the most accurate and easy way to install scrapy without coming across any errors on windows 7 and 8 .