phantomjs

How do I set a variable from casper.evaluate()?

由 你离开我真会死。 提交于 2019-12-25 03:43:09
问题 I'm trying to set a value from within casper.evaluate() that I will check later to run a test on, but it doesn't seem to be working. isArticleOrReview = false; casper.waitFor(function check() { //here I'm just waiting for jQuery to load return this.evaluate(function() { return jQuery.fn.jquery == '1.2.6'; }); }, function then() { //once jQuery has been loaded, do this stuff this.evaluate(function() { isArticleOrReview = (jQuery('body').hasClass('node-type-review') || jQuery('body').hasClass(

Using R to scrape tables when URL does not change

由 爱⌒轻易说出口 提交于 2019-12-25 03:26:40
问题 I'm relatively new to scraping in R and have had great luck using "rvest", but I've run into an issue I cannot solve. The website I am trying to scrape has the same URL no matter what page of the table you are on. For example, the main webpage is www.blah.com with one main table on it that has 10 other "next" pages of the same table, but just the next in order (I apologize for not linking to the actual page as I cannot due to work issues). So, if I'm on page 1 of the table, the URL is www

Scraping Javascript-rendered webpage that references external javascript scripts in R

由 我是研究僧i 提交于 2019-12-25 03:22:41
问题 I am trying to scrape this webpage: https://www.mustardbet.com/sports/events/302698 Since the webpage seems to be rendered dynamically, I am following this tutorial: https://www.datacamp.com/community/tutorials/scraping-javascript-generated-data-with-r#gs.dZEqev8 As the tutorial suggests, I save a file named "scrape_mustard.js" with the following code: // scrape_mustard.js var webPage = require('webpage'); var page = webPage.create(); var fs = require('fs'); var path = 'mustard.html' page

highchart exporting with phantomjs implementation

由 你离开我真会死。 提交于 2019-12-25 02:45:00
问题 I am trying to implement highchart exporting function in my jsp/glassfish website project in which i need charts to be converted to png,jpgs and pdfs formats but in offline mode.i had followed the steps and instruction given in official exporting site but i encountered following issues. i have downloaded the phantom.js and highchart exporting folder i change values in "app-convert.properties" file located in "highcharts-export\highcharts-export-web\src\main\webapp\WEB-INF\spring" folder and

python爬虫学习笔记（一）——环境配置（windows系统）

由 蓝咒 提交于 2019-12-24 21:08:15
在进行 python 爬虫学习前，需要进行如下准备工作： python3+pip 官方配置 1. Anaconda（ 推荐，包括 python 和相关库） 【推荐地址：清华镜像】 　　 https://mirrors.tuna.tsinghua.edu.cn/anaconda/archive/ 【安装过程中注意选择自动添加 path到环境变量中，未选择需要自己添加 】 　　 　　 红色提示的意思是：自动添加的环境变量会处于最前面，自动成为默认，可能会使原本使用默认的软件报错，未防止以上问题，可以打开环境变量后，调整顺序将Anaconda相关path移到最后。 【环境变量设置成功】 　　 2.官方安装 　　 官方安装指的是只安装 python ，需要的库通过手动安装，下文会涉及库的安装 【 python 下载】 　　 官方地址： https://www.python.org/ 【自行添加环境变量】 　　 【注】【 环境变量如何添加 】 　 　我的电脑（右键）---属性---高级系统设置---环境变量-----path----在后面加上分号添加python安装地址及下层scipt（对pip的配置），如上图 3.pycharm安装 　 　 PyCharm是一种Python IDE，带有一整套可以帮助用户在使用Python语言开发时提高其效率的工具，比如调试、语法高亮、Project管理

WebDriverException: Message: {“errorMessage”:"null is not an object

由 …衆ロ難τιáo~ 提交于 2019-12-24 20:45:45
问题 I get the error when executing the following code for this HTML page, but this error only happens at /html/body/div[3]/div[1]/div[1]/div[1]/div/div[10]/a/div[1]/div[2] : WebDriverException: Message: {"errorMessage":"null is not an object (near '...ull).singleNodeValue.click();...')","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"223","Content-Type":"application/json;charset=UTF-8","Host":"127.0.0.1:34955","User-Agent":

Running PhantomJs from command prompt using C#

由 戏子无情 提交于 2019-12-24 17:43:08
问题 I am trying to run PhantomJs.exe throw C# code. My Code : Process process = new Process(); ProcessStartInfo startInfo = new ProcessStartInfo(); startInfo.WindowStyle = ProcessWindowStyle.Normal; startInfo.FileName = "cmd.exe"; startInfo.WorkingDirectory = @"E:\"; startInfo.Arguments = "some string code here"; startInfo.CreateNoWindow = true; process.StartInfo = startInfo; process.Start(); When I run it is going to WorkingDirectory E:/ but Arguments are not writing on cmd prompt. Can any buddy

CasperJS cannot click on a DIV button

由 橙三吉。 提交于 2019-12-24 16:15:14
问题 I have been working on a website scraping project using CasperJS. It is an ASPX website. I could login to the site and then fill the form that runs a search but after filling the form I cannot simulate a click on the DIV button. The search runs using AJAX but when I capture page after waiting few seconds it does not show results in the captured image. The Search button is made of DIV and upon clicking it runs other hidden JavaScript functions that send the AJAX request to retrieve the search

Including libraries fails - what is document root?

由 只愿长相守 提交于 2019-12-24 16:00:23
问题 I have a file ( src/inc.js ) that loads certain files into my browser using jquery's ajax: $.ajax({url: 'somefile.js', async: false, dataType: 'script', error: function(qXHR, textStatus, errorThrown) { console.log( '[' + textStatus + '] ' + errorThrown + ' with: ' + this.url ); } }); and this file gets included via browserification into my browser (phantomjs). Gruntfile.js : browserify: { libs: { src: ['src/*.js'], dest: 'test/libs.js' } } and handed to karma (in Gruntfile.js ): karma: {

How can I turn part of my casperjs script into a function so I can use it multiple times

由 Deadly 提交于 2019-12-24 15:27:31
问题 Okay, so here is a part of my casperjs script below which works fine if(casper.exists(ac1)){ var uel = "https://example.ws/send.html?f=1099817"; this.thenOpen(uel, function() { casper.wait(10000, function() { casper.then(function() { this.evaluate(function() { var amount = 0.29 var result = amount * 0.019 var result2 = result.toFixed(6); var fresult = amount - result2; var needed = fresult.toFixed(3); document.getElementById('account').value = 'ydfg028'; document.getElementsByName('data')[0]
订阅 phantomjs