casperjs

Click on all links matching a selector

心不动则不痛 提交于 2019-11-29 10:20:40
I have a list of links that I have to simulate a click on using CasperJS. They all share the same class. However using this.click('.click-me') only clicks on the first link. What's the proper way of clicking on all the links? I'm thinking that maybe I should try to get the number of links via evaluate() and then use a for loop. But if I use evaluate() with the number of links I have to use messages to communicate back and that seems complicated. Is there a better way? Tyson Nero I ended up using the nth-child() selector to accomplish this. Here's how... Page: <ul id="links"> <li><a href="#1">1

Automate daily csv file download from website button click

[亡魂溺海] 提交于 2019-11-29 08:15:39
I would like to automate the process of visiting a website, clicking a button, and saving the file. The only way to download the file on this site is to click a button. You can't navigate to the file using a url. I have been trying to use phantomjs and casperjs to automate this process, but haven't had any success. I recently tried to use brandon's solution here Grab the resource contents in CasperJS or PhantomJS Here is my code for that var fs = require('fs'); var cache = require('./cache'); var mimetype = require('./mimetype'); var casper = require('casper').create(); casper.start('http:/

CasperJS: Iterating through URL's

梦想与她 提交于 2019-11-29 07:54:21
I'm pretty new to CasperJS, but isn't there a way to open a URL and execute CasperJS commands in for loops? For example, this code doesn't work as I expected it to: casper.then(function() { var counter = 2013; for (i = counter; i < 2014; i++) { var file_name = "./Draws/wimbledon_draw_" + counter + ".json"; // getting some local json files var json = require(file_name); var first_round = json["1"]; for (var key in first_round) { var name = first_round[key].player_1.replace(/\s+/g, '-'); var normal_url = "http://www.atpworldtour.com/Tennis/Players/" + name; // the casper command below only

php execution phantom js works but casperjs does not work permission denied

女生的网名这么多〃 提交于 2019-11-29 05:22:54
okay so i am running centos 6.5 with zpanel setup. this then has phantom js and casperjs both setup and working it the /usr/local/bin directory. i use this php command for phantomjs and casperjs echo exec('/usr/local/bin/casperjs --version 2>&1'); phantomjs works fine but casperjs gives me this error sh: /usr/local/bin/casperjs: Permission denied can anyone out there help me fix this problem it is just a pain output for ls -al casperjs [root@cyber-hosted ~]# ls -al casperjs total 100 drwxr-xr-x 10 apache root 4096 Feb 4 15:49 . dr-xr-x---. 5 root root 4096 Feb 4 15:49 .. drwxr-xr-x 2 root root

Installing CasperJS on Windows: How to do it correctly?

六眼飞鱼酱① 提交于 2019-11-28 23:40:41
问题 I know there is a documentation from CasperJS website about how to install CasperJS on Windows, but bear with me these guys only explained for the pros only. If you are new to all this CasperJS and PhantomJS world, you dont stand a chance to understand not even spending two days trying to search the net for a better explanation. I am working on a project that requires a screenshot of each website listed on my project website, looking around i found out that PhantomJS would be great for this

CasperJS: swallows special keys like Enter?

送分小仙女□ 提交于 2019-11-28 22:03:24
I'm trying to write a test using CasperJS for a scenario where pressing Enter in an input is the trigger for the page to do something with the text otherwise typed into the input. An abbreviated/simplified version of the CasperJS test: casper.start('http://localhost:3000/input-demo', function() { this.sendKeys('#demo-input', 'demo text'); this.sendKeys('#demo-input', '\uE007'); this.test.assertEquals(this.getHTML('#stage'), 'input demo'); }); casper.run(); (Where we run it as casperjs test this-test.js ) I've verified that sendKeys is getting the text into the input, but that text never shows

How to wait for page loading when using casperjs?

爱⌒轻易说出口 提交于 2019-11-28 21:13:55
I am trying to scrape a webpage which has a form with many dropdowns and values in the form are interdependent. At many point I need the code to wait till the refresh of the page complete. Eg after selecting an option from the list, the code should wait till the next list is populated based on this selection. It would be really helpful if someone could give pointers because strangely my code is working only after I gave so much unnecessary logging statements which in-turn created some delay. Any suggestions to improve the code would be very helpful. var casper = require('casper').create({

how to scrape links with phantomjs

一笑奈何 提交于 2019-11-28 20:35:22
问题 Can PhantomJS be used an an alternative to BeautifulSoup? I am trying to search on Etsy and visit all the links in term. In Python, I know how to do this (with BeautifulSoup) but today I want to see if I can do the same with PhantomJS. I'm not getting very far. This script should search "hello kitty" on Etsy and return all the of products <a class="listing-thumb" href=...></a> and print them in the console. Ideally I'd visit them later on and get the information I need. Right now it just

Write results into a file using CasperJS

孤人 提交于 2019-11-28 17:26:04
How do I create a file in the file system and place the contents of this.getPageContent() inside it? var fs = require('fs'); fs.write(myfile, myData, 'w'); for saving daily scrapes I do: var currentTime = new Date(); var month = currentTime.getMonth() + 1; var day = currentTime.getDate(); var year = currentTime.getFullYear(); var myfile = "data-"+year + "-" + month + "-" + day+".html"; You could also append to a text file using the method below var casper = require('casper').create(); var fs = require('fs'); var fname = new Date().getTime() + '.txt'; var save = fs.pathJoin(fs.workingDirectory,

Taking reliable screenshots of websites? Phantomjs and Casperjs both return empty screen shots on some websites

非 Y 不嫁゛ 提交于 2019-11-28 17:13:30
问题 Open a web page and take a screenshot. Using ONLY phantomjs: (this is a simple script, in fact it is the example script used in their docs. http://phantomjs.org/screen-capture.html var page = require('webpage').create(); page.open('http://github.com/', function() { page.render('github.png'); phantom.exit(); }); Problem is that for some websites (like github) funny enough are somehow detecting and not serving phantomjs and nothing is being rendered. Result is github.png is a blank white png