I want to run a Python script using Selenium WebDriver with PhantomJS as a headless browser on my Raspberry Pi running Raspbian.
I originally wrote the script in OS X where it works fine. But in trying to make it work on the Raspberry, I'm having problems.
When trying to run the script, I get this error:
raise WebDriverException("Can not connect to the Service %s" % self.path) selenium.common.exceptions.WebDriverException: Message: Can not connect to the Service /usr/bin/phantomjs
Brief version of the script:
from selenium import webdriver from selenium.webdriver.common.desired_capabilities import DesiredCapabilities user_agent = ("Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) " + "AppleWebKit/537.36 (KHTML, like Gecko) Chrome/29.0.1547.57 Safari/537.36") dcap = dict(DesiredCapabilities.PHANTOMJS) dcap["phantomjs.page.settings.userAgent"] = user_agent serv_args = ["--ignore-ssl-errors=false", "--ssl-protocol=tlsv1", "--disk-cache=false"] driver = webdriver.PhantomJS(executable_path="/usr/bin/phantomjs", desired_capabilities = dcap, service_arguments = serv_args, port=65000)
Questions
- If the solution involves installing the binaries manually to
/usr/local/bin or such, how would I do that? The binaries available on the PhantomJS webpage is for linux-x86 and linux-i686, so I'm assuming they won't work on an Raspberry Pi 2 B ARM Cortex A-7 processor. - I have also tried to build PhantomJS myself according to these instructions, but the process froze midway through. The Raspberry also doesn't meet the recommended hardware requirements for building.
Background info
- I am using
Python 2.7.9 - I have created a
virtualenv and installed all the Python modules in it; e.g. pip install selenium, and attempt to run the script here - I have installed the latest version of PhantomJS via
sudo apt-get install phantomjs - I have disabled my
ufw firewall while testing
Ok I will start with the solution, there is a version compiled for arm here phantomjs-linux-armv6l, on the pi run the following commands:
$ cd /tmp $ wget https://github.com/aeberhardo/phantomjs-linux-armv6l/archive/master.zip $ unzip master.zip $ cd phantomjs-linux-armv6l-master $ bunzip2 *.bz2 && tar xf *.tar
I added:
sudo cp phantomjs-1.9.0-linux-armv6l/bin/phantomjs /usr/bin
So phantomjs will be on your path.
pi@raspberrypi ~ $ phantomjs --version 1.9.0 pi@raspberrypi ~ $ phantomjs phantomjs>
Now we have that done, time to test:
pi@raspberrypi ~ $ cat test.py #!/usr/bin/python from selenium import webdriver driver = webdriver.PhantomJS() driver.get('http://stackoverflow.com/questions/36314771/how-to-install-phantomjs-for-use-with-python-selenium-on-the-raspberry-pi/36388824#36388824') a = driver.find_element_by_xpath('//*[@id="question-header"]/h1/a') print(a.text) print(driver) pi@raspberrypi ~ $ python test.py How to install PhantomJS for use with Python Selenium on the Raspberry Pi? <selenium.webdriver.phantomjs.webdriver.WebDriver (session="b184e110-f9c4-11e5-aede-7f5c42f062d7")>
From the faq. Starting with PhantomJS 1.5, it is pure headless and there is no need to run X11/Xvfb anymore..
I tried using xvfb-run and exporting the display, using a shell script in init.d to start xvfb, I got a little further being able to run iceweasel from bash headless no problem but still no cigar when it came to phantomjs and selenium. I think it may just come down to an incompatibility between selenium and the version of phantomjs, regardless having 1.9.0 and real headless browsing is a lot more desirable.
I was in the middle of setting up a toolchain and was going to try to compile myself when I found the link above, for anyone interested in cross compiling, crosstools-ng makes life a lot easier.
I am running an arm6, there is also a compiled version for arm7 using 2.0.0, the dependencies are:
sudo apt-get install flex bison gperf ruby perl libsqlite3-dev libfontconfig1-dev libicu-dev libfreetype6 libssl-dev libpng-dev libjpeg-dev python libX11-dev libxext-dev
The install procedure, I have extracted the binary to dropbox:
wget https://www.dropbox.com/s/epj1rji9d239dco/phantomjs chmod +x phantomjs sudo cp phantomjs /usr/bin
The original github link is phantomjs-2.0.0-armv7
