Is PhantomJS CPU (core) bound?

Question

I'm starting to do some parallel browser based testing and want to see how many browsers I can run in parallel on an EC2 large box before hitting 100% CPU.

I'm using the JMeter webdriver plugin to actually run the browsers.

With FireFox it's literally 1 browser per CPU core. 4 browsers on a 4 core box is about 80% CPU. With 5 browsers it's 95% (which isn't good for performance testing).

Before I go down the path of PhatomJS, are there any benchmarks that compare PhantomJS to FireFox (or even Chrome) in terms of CPU usage when running in parallel?

Any idea how many PhantomJS instances I'll be able to run in parallel on a EC2 large box before hitting 85% CPU?

Thanks James.

Answer 1

As per Web Driver Tutorial

it is important to note that this is mainly used to complement JMeter's HTTP Sampler. The Web Driver Sampler is meant to be run in conjunction with a HTTP Sampler so that from the server's perspective, the load is production like. At the same time, the web sampler will simulate the user experience of interacting with the website/webapp whilst the server is under load, hence measuring the real user's experience at the same time.

So your idea of running multiple browsers, no matter full or headless, doesn't look good. The idea of WebDriver JMeter plugin is to have one real browser instance crawling web application under test to see whether load has negative impact on real-life users on areas which JMeter can't cover by design like executing JavaScript / AJAX calls, etc.

There are several configuration elements available in JMeter to make it pretend more like a real browsers. Consider using:

• HTTP Cookie Manager - to represent browser cookies and deal with cookie-based authentication challenges
• HTTP Cache Manager - to act like browser cache (i.e. download embedded resources only once)
• HTTP Header Manager - to represent browser-derived headers like User-Agent

See How to make JMeter behave more like a real browser guide for more tips and tricks.

If you use case assumes only browser-based load testing I'd recommend looking into Selenium Grid which is capable of orchestrating multiple WebDriver instances running on different hosts.