phantomjs

Set a string as the response of a webpage in phantomjs

别说谁变了你拦得住时间么 提交于 2019-12-02 14:53:33
问题 Hi what i am trying to do is not to get the webpage as page.open(url); but to set a string that has already been retrieved as the page response. Can that be done? 回答1: Yes, and it is as easy as assigning to page.content. It is usually also worth setting a page.url (as otherwise you might hit cross-domain issues if doing anything with Ajax, SSE, etc.), and the setContent function is helpful to do both those steps in one go. Here is the basic example: var page = require('webpage').create();

PhantomJs Crashes while running with grunt-karma test cases ????/

岁酱吖の 提交于 2019-12-02 10:23:02
We are facing an issue while running karma test cases with phantomJs our phantomJs crashes and gets disconnected. Is that due to memory leakage or some other issue.Kindly let me know if some one has some suitable solution. I found that the workaround is to break test cases into multiple grunt task but since we have a lot of test cases more than 1500 so that would not be a feasible task. We are using the below versions Node:- 0.10.32 Karma:- 0.12.24 PhantomJs:- 1.9.8 (karma-phantomJs-Launcher) Please let me know the solutions asap. There are two reasons I found that this can happen. PhantomJS

Does PhantomJS use QtWebKit or Blink?

早过忘川 提交于 2019-12-02 10:16:17
问题 Does PhantomJS (CasperJS) use WebKit or Chromium as default browser-driver? If WebKit, is it direct WebKit or qt or gtk? 回答1: PhantomJS is based on QtWebKit. The FAQ says: Q: Which WebKit version is used by PhantomJS? A: If you want to know HTML5/CSS3/other features supported by PhantomJS, using WebKit version is not a good idea. See Supported Web Standards documentation page for details. If you really like to get the WebKit version, find it via the user agent, run the examples/useragent.js.

java抓取动态生成的网页--吐槽

旧街凉风 提交于 2019-12-02 10:15:37
最近在做项目的时候有一个需求:从网页面抓取数据,要求是首先抓取整个网页的html源码(后期更新要使用到)。刚开始一看这个简单,然后就稀里哗啦的敲起了代码(在这之前使用过Hadoop平台的分布式爬虫框架Nutch,使用起来是很方便,但是最后因为速度的原因放弃了,但生成的统计信息在后来的抓取中使用到了),很快holder.html和finance.html页面成功下载完成,然后解析完holder.html页面之后再解析finance.html,然后很沮丧的发现在这个页面中我需要的数据并没有在html源码中,再去浏览器查看源码果然是这样的,在源码中确实没有我需要的数据,看来不是我程序写错了,接下来让人身心疲惫的事情来了---获取包含动态内容的html页面。   在所谓的中国最强搜索引擎---百度上面行走了好长的时间,发现大部分的人都在将使用WebDriver和HttpUnit(其实前者已经包含了后者),这个高兴,终于找到了解决办法。怀着万分的激动使用WebDriver,我要想骂人了。   下面是关于WebDriver的吐槽   WebDriver是一个测试框架,原本设计的时候就不是用来服务爬虫的,但是我想说的是:八字就差一撇了,你就不能多往前做一步吗?为什么网上还有那么多的人推荐WebDriver呢?我想这些人没有从实际出发

How can I send POST data to a phantomjs script

时光怂恿深爱的人放手 提交于 2019-12-02 10:03:16
I am working with PHP/CURL and would like to send POST data to my phantomjs script, by setting the postfields array below: $ch = curl_init(); curl_setopt($ch, CURLOPT_COOKIEJAR, $cookieFile); curl_setopt($ch, CURLOPT_FOLLOWLOCATION, TRUE); curl_setopt($ch, CURLOPT_RETURNTRANSFER, TRUE); curl_setopt($ch, CURLOPT_USERAGENT, "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)"); curl_setopt($ch, CURLOPT_POST, TRUE); curl_setopt($ch, CURLOPT_POSTFIELDS, $postFieldArray); curl_setopt($ch, CURLOPT_URL, $url); $output = curl_exec($ch); The problem is I don't know how to parse a POST request inside

Screenshot of video element using phantomjs

梦想与她 提交于 2019-12-02 10:01:25
I am trying to take a screenshot of page with a video element using phantomjs. Currently, i set the time of the video ahead to some desired seconds (i.e. 60). Then I use page.render(output). However, the screenshot created is always black. Can anyone see the problem or verify if what Im trying to do is possible? Running the features.js example (using modernizr.js), you will see >phantomjs.exe examples\features.js Detected features (using Modernizr 2.0.6): Supported: touch generatedcontent fontface flexbox canvas canvastext postmessage websqldatabase hashchange history draganddrop websockets

Collecting client side Performance Metrics from Phantomjs through Jmeter

若如初见. 提交于 2019-12-02 09:49:54
Throughout PhantonJS Driver Config is there any way we can capture the following client-side performance metrics through phantomjs by using Jmeter DomLoad windLoad ResourceCount Resource size. Thanks I would recommend capturing the metrics using WDS.browser.executeScript() method which allows executing arbitrary JavaScript code. Once done you can directly store "interesting" values into JMeter Variables for later use. Here is an example of using Navigation API to get extended information on page loading: WDS.sampleResult.sampleStart() WDS.browser.get('http://jmeter-plugins.org') var timings =

python - sending selenium chrome instance to the background

限于喜欢 提交于 2019-12-02 09:45:36
问题 I am trying to open a simple chrome instance using Python and selenium. Please find my code below: import time, datetime, sys, os start_time = time.time() from datetime import datetime os.system("cls") from selenium import webdriver from selenium.webdriver.common.keys import Keys from selenium.common.exceptions import TimeoutException from selenium.webdriver.support.ui import WebDriverWait from selenium.webdriver.support import expected_conditions as EC from selenium.webdriver.common.by

A specific site is returning a different response on python and in chrome

删除回忆录丶 提交于 2019-12-02 08:38:05
I am trying to access a specific site using python, and no matter which lib I use I just can't seem to access it. I have tried Selenium+PhantomJS, I have tried requests and urllib. Whenever I try to access the site from the browser I get a json file, and whenever I try to access it from a python script I get an html file (which has a huge minified script inside it) I suspect this site is detecting I'm sending the request headlessly and is blocking my requests, but I can't figure out how. The site address is: http://www.yesplanet.co.il/presentationsJSON I would very much appreciate if anyone

How do I clear the phantomjs cache on a mac?

爱⌒轻易说出口 提交于 2019-12-02 08:18:29
问题 I somehow corrupted the phantomjs cache while testing with karma (writing pouchdb tests). How can I clear this cache and get back to my original testing enviornment? 回答1: It took me days to figure this out so I figured I'd post it here, just rm -rf ~/Library/Application\ Support/Ofi\ Labs/PhantomJS/* I hope this saves someone else the hassle! 来源: https://stackoverflow.com/questions/25578054/how-do-i-clear-the-phantomjs-cache-on-a-mac