问题
I've just started using casperjs after trying to use python (selenium / requests and mechanise) to scrape a page only after some javascript loaded some dynamic content on the page.
Since this was very hard to do or very slow with selenium it was suggested I turn to Casper js (which requires phantomjs).
One thing I am wondering (I am quite new to javascript) is relating to a javascript onclick event.
The page I want to scrape by default shows ten names per page, and at the bottom has options to show (5) or show (100).
After diving into this code and inspecting it with firebug I am wondering if it is possible to change the onclick=loaditems(100) to something like... onclick=loaditems(Load X items), where X could be 200. (or whatever number it needs to be to load all the content on one page and make it easier for scraping. Is this possible?
update * reviewer asked for the code used to select the 100 items per page....
The code (HTML) is..
<a title="Show 100 items per page"
onclick="lconn.profiles.Friending.setItemsPerPage(this,100)" href="javascript:void(0);">100</a>
and the Xpath is...
/html/body/div/div[2]/div[3]/div[3]/span/div/div/div/div/div[2]/div/div/form/div??/div[4]/div/ul/li[4]/a
problem I am able to edit the onclick command and change the value to a higher number, however I do not know how to then execute it with the higher number of elements I want to display per page to see if it works.
回答1:
I used a simple CSS selector for this task. You can change the onclick
attribute with string operations. In this case I replace "100"
with num
. I also added a clickIt
argument to click the changed link.
casper.changeItemsPerPageLink = function(num, clickIt){
casper.evaluate(function(num, clickIt){
num = ""+num;
var a = document.querySelector('a[title="Show 100 items per page"]');
a.innerHTML = num;
a.title = a.title.replace("100", num);
a.setAttribute("onclick", a.getAttribute("onclick").replace("100", num));
if (clickIt) {
a.click();
}
}, num, clickIt);
};
See my test code gist.
来源:https://stackoverflow.com/questions/23761795/injecting-javascript-code-into-an-on-click-event-with-javascript-and-casper-js