How to wait for page loading when using casperjs?

Question

I am trying to scrape a webpage which has a form with many dropdowns and values in the form are interdependent. At many point I need the code to wait till the refresh of the

Answer 1

I have the same experience doing the same thing as you. script these way in user perspective never gone well. it crash in middle of nowhere and very unreliable. I was doing search from salesforce that also require login.

You need to keep your step as minimum as possible. script in a cron job way. don't do form fill/button click unless you are doing UI testing. I would advice you to break the process into two parts

// this part do search and find out the exact url of your screen capture.
// save it in a db/csv file
1 - start by POST to http://www.abc.com/forum/member.php with username password in body.
2 - POST/GET to http://www.abc.com/search/index.php with your search criteria, you look at what the website require. if they do POST, then POST.

// second part read your input
1 - login same as first part.
2 - casper forEach your input save your capture. (save the capture result in db/csv)

my script now is pure phantomjs, casper script just keep crashing for no reason. even phantomjs is unreliable. I save the result/status on each successful search/download, whenever there is error I exit the script if not the rest of result is unpredictable(good result in chrome turn out bad in phantomjs).