casperjs

How to login into a website with CasperJS?

隐身守侯 提交于 2019-11-27 10:52:39
How can I login with CasperJS by submitting a form. I searched google and haven't found any good examples about it. Ngo Hung You will need to use Casper fill() function. Below is an example which login to Facebook and print out your name after login. Note that you need to put in your username and password: var casper = require('casper').create({ verbose: true, logLevel: 'debug', pageSettings: { loadImages: false, // The WebPage instance used by Casper will loadPlugins: false, // use these settings userAgent: 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/537.4 (KHTML, like Gecko)

Write results into a file using CasperJS

自古美人都是妖i 提交于 2019-11-27 10:28:24
问题 How do I create a file in the file system and place the contents of this.getPageContent() inside it? 回答1: var fs = require('fs'); fs.write(myfile, myData, 'w'); for saving daily scrapes I do: var currentTime = new Date(); var month = currentTime.getMonth() + 1; var day = currentTime.getDate(); var year = currentTime.getFullYear(); var myfile = "data-"+year + "-" + month + "-" + day+".html"; 回答2: You could also append to a text file using the method below var casper = require('casper').create(

Understanding the evaluate function in CasperJS

走远了吗. 提交于 2019-11-27 08:47:37
I want to understand in which case I should or have to use the evaluate function. I have read the API doc about the evaluate function of CasperJS, but I'm unsure in which case I should use this function. And what does DOM context mean? Can somebody provide an example? The CasperJS documentation has a pretty good description of what casper.evaluate() does. To recap: You pass a function that will be executed in the DOM context (you can also call it the page context). You can pass some primitives as arguments to this function and return one primitive back. Keep in mind that this function that you

Grab the resource contents in CasperJS or PhantomJS

[亡魂溺海] 提交于 2019-11-27 08:38:15
I see that CasperJS has a "download" function and an "on resource received" callback but I do not see the contents of a resource in the callback, and I don't want to download the resource to the filesystem. I want to grab the contents of the resource so that I can do something with it in my script. Is this possible with CasperJS or PhantomJS? This problem has been in my way for the last couple of days. The proxy solution wasn't very clean in my environment so I found out where phantomjs's QTNetworking core put the resources when it caches them. Long story short, here is my gist. You need the

CasperJS passing variable to evaluate can't get it to work [duplicate]

前提是你 提交于 2019-11-27 08:12:21
问题 This question already has an answer here: Passing variable into page.evaluate - PhantomJS 3 answers okay so here's my casperjs function : if(casper.exists(ac2)){ var accountnumber = this.fetchText('div.arabic:nth-child(2) > table:nth-child(1) > tbody:nth-child(1) > tr:nth-child(2) > td:nth-child(2) > a:nth-child(1)'); var redir = accountnumber.substr(1); casper.then(function() { var uel = "https://example.ws/send.html?f=" + redir; this.thenOpen(uel, function() { casper.wait(10000, function()

waitForSelector passes, but assertExists fails for the same selector

馋奶兔 提交于 2019-11-27 08:07:08
问题 When I call the following function, waitForSelector passes for 'selector', but assertExists fails for the same selector. How is it possible? casper.waitForSelector(selector, function() { casper.test.assertExists(selector, sectionName + " opened up successfully."); }, function() { casper.test.fail(sectionName + " did not load in given time"); }, max_timeout); Here is a complete example to reproduce the issue with an :nth-child selector. 回答1: This is a known bug (see #11632, #11737) in the Qt4

Run phantomjs with --ignore-ssl-errors=true from casperjs

这一生的挚爱 提交于 2019-11-27 06:43:53
问题 I have a problem with https page. Page is completly ok, it exist but phantomjs tell me something else: 'loading resource failed with status fail'. I read about it for a while and for now i know it's phantomjs bug and the solution to this problem is: --ignore-ssl-errors=true So I know solution, but don't how to use it. How can I pass this to phantomjs from casper? Where should I do that ? EDIT: Entire code: var casper = require('casper').create({ verbose: true, logLevel: 'warning',

Why doesn't this.evaluate return DOM nodes correctly?

房东的猫 提交于 2019-11-27 05:41:02
I'm trying to get an object from a web page through the evaluate() method so I can work with it outside the scope of evaluate . The element selected with name symbol is a <select> tag with 148 <options> (=dropdown menu). casper.then(function () { var elmnt = this.evaluate(function () { return document.getElementsByName("symbol")[0]; }); console.log(elmnt.options[14].index); }); //Returns TypeError: 'null' is not an object (evaluating 'elmnt.options[14].index') casper.then(function () { var elmnt = this.evaluate(function () { return document.getElementsByName("symbol")[0].options[14].index; });

What Does 'Then' Really Mean in CasperJS

人盡茶涼 提交于 2019-11-27 05:03:07
问题 I'm using CasperJS to automate a series of clicks, completed forms, parsing data, etc through a website. Casper seems to be organized into a list of preset steps in the form of then statements (see their example here: http://casperjs.org/quickstart.html) but it's unclear what triggers the next statement to actually run. For example, does then wait for all pending requests to complete? Does injectJS count as a pending request? What happens if I have a then statement nested - chained to the end

CasperJS loop or iterate through multiple web pages?

别说谁变了你拦得住时间么 提交于 2019-11-27 04:19:35
问题 I have a CasperJS script that scrapes ratings and dates from one webpage. Now I want to scrape the same data from multiple pages under the same website. How can I loop through the different subpages given this code: var ratings = []; var dates = []; var casper = require('casper').create({ pageSettings: { loadImages: false, loadPlugins: false }, logLevel: "debug", verbose: true }); var fs = require('fs'); function getRatings() { var ratings = document.querySelectorAll('#BVRRRatingOverall