In Version 1.5 we got rid of the Xvfb and Xlib dependencies.
Now this should be pretty simple.
I'd suggest:
- checkout the source
- build a binary with the given script (check the options to build the version without xvfb)
- be happy :)
Ariya, the main author, also did a good job at incorporating Qt in our codebase directly, so we have a tailored, stripped down Qt that does just what PhantomJS needs and no more :)