casperjs

Wait for a child process in CasperJS

我的梦境 提交于 2019-11-28 08:28:49
问题 I have a CasperJS process that loads some page and then it needs to call a go-process to analyze the page and to decide what page should be loaded next. go-process takes a while to execute. My problem is that CasperJS doesn't wait for the go-process to complete and exits. casper.then(function(){ var p = cp.execFile('/path/parse', [], {}, function(error, stdout, stderr) { console.log(stdout); }); }); How can I wait for my child process to complete? 回答1: All then* and wait* functions are

how to persist cookies between different casperjs processes

放肆的年华 提交于 2019-11-28 03:16:18
this is a question about how to persist cookies from one casperjs page to another.. so basically i got a nodejs file that spawns casperjs as a worker to do certain tasks.. one is to login, once logged in I store the cookie in a file. when i spawn the next casper worker.. i want it to to use the cookie rather having to login again.. both these methods failed : first: when i spawn the worker capserjs I add the --cookies-file=./cookiefilename ie var child = spawn('casperjs',['scrape.js','--cookies-file=./'+cookieFileName]); second: within the casperjs worker file.. I make it read and set the

What Does 'Then' Really Mean in CasperJS

二次信任 提交于 2019-11-28 02:50:42
I'm using CasperJS to automate a series of clicks, completed forms, parsing data, etc through a website. Casper seems to be organized into a list of preset steps in the form of then statements (see their example here: http://casperjs.org/quickstart.html ) but it's unclear what triggers the next statement to actually run. For example, does then wait for all pending requests to complete? Does injectJS count as a pending request? What happens if I have a then statement nested - chained to the end of an open statement? casper.thenOpen('http://example.com/list', function(){ casper.page.injectJs('

CasperJs and Jquery with chained Selects

六眼飞鱼酱① 提交于 2019-11-28 01:41:36
I'm trying to create a testing case for a web site which includes a form with 3 chained selects. The first select is populated by default when the web page is loaded. If any option from the first select is selected, then the second select is populated via an ajax call. In the same way, when an option is selected on the second selected, so the third select is populated via an ajax call. Finally, when an option is selected on the third select, a html table is populated with the information than I need to validate. The three interconnected selects have this struct <select id="s1" name="s1">

CasperJS: Iterating through URL's

孤街浪徒 提交于 2019-11-28 01:27:24
问题 I'm pretty new to CasperJS, but isn't there a way to open a URL and execute CasperJS commands in for loops? For example, this code doesn't work as I expected it to: casper.then(function() { var counter = 2013; for (i = counter; i < 2014; i++) { var file_name = "./Draws/wimbledon_draw_" + counter + ".json"; // getting some local json files var json = require(file_name); var first_round = json["1"]; for (var key in first_round) { var name = first_round[key].player_1.replace(/\s+/g, '-'); var

CasperJS skip step on timeout

为君一笑 提交于 2019-11-28 00:36:09
I have one page in my casperjs test that has images , I dont what to wait until this page loaded to go to the next step. How can I do it ? I have tryed this way var casper = require("casper").create({ onStepTimeout: function() { this.echo("TIMEOUT" + this.requestUrl,"RED_BAR"); // Some skip page controlling code }, ); var timeout = ~~casper.cli.get(0); casper.start("http://127.0.0.1/index2.php", function () { this.echo("FIRST GOOD PAGE", "GREEN_BAR"); casper.options.stepTimeout = timeout; }); casper.thenOpen("http://127.0.0.1/slowpage.php", function() { this.echo("SECOND PAGE LOADED - I want

casperjs download csv file

杀马特。学长 韩版系。学妹 提交于 2019-11-27 19:32:45
I am trying to download a csv file(advert report) from a site using the below code. The issue is, it will download the HTML page and not the csv file. I cannot give you the URL as it is behind the login, but it is similar case when you download Firefox from the below URL http://www.mozilla.org/en-US/firefox/new/ It is a GET request and when I do inspect element Network Tab the get request gets Cancelled. I am new to Casper and don't know how to handle such requests. Any help would be appreciated casper.then(function() { var downloadURL = ""; this.evaluate(function() { var downloadURL = "http:/

CasperJS loop or iterate through multiple web pages?

∥☆過路亽.° 提交于 2019-11-27 18:51:46
I have a CasperJS script that scrapes ratings and dates from one webpage. Now I want to scrape the same data from multiple pages under the same website. How can I loop through the different subpages given this code: var ratings = []; var dates = []; var casper = require('casper').create({ pageSettings: { loadImages: false, loadPlugins: false }, logLevel: "debug", verbose: true }); var fs = require('fs'); function getRatings() { var ratings = document.querySelectorAll('#BVRRRatingOverall_Review_Display > div.BVRRRatingNormalImage > img'); return Array.prototype.map.call(ratings, function(e) {

How to wait for page loading when using casperjs?

假如想象 提交于 2019-11-27 13:50:10
问题 I am trying to scrape a webpage which has a form with many dropdowns and values in the form are interdependent. At many point I need the code to wait till the refresh of the page complete. Eg after selecting an option from the list, the code should wait till the next list is populated based on this selection. It would be really helpful if someone could give pointers because strangely my code is working only after I gave so much unnecessary logging statements which in-turn created some delay.

Reliably detecting PhantomJS-based spam bots

∥☆過路亽.° 提交于 2019-11-27 11:35:46
Is there any way to consistently detect PhantomJS/CasperJS? I've been dealing with a spat of malicious spambots built with it and have been able to mostly block them based on certain behaviours, but I'm curious if there's a rock-solid way to know if CasperJS is in use, as dealing with constant adaptations gets slightly annoying. I don't believe in using Captchas. They are a negative user experience and ReCaptcha has never worked to block spam on my MediaWiki installations. As our site has no user registrations (anonymous discussion board), we'd need to have a Captcha entry for every post. We