How to overwrite and perform actions on webpage with AbotX Javascriptrendering

一曲冷凌霜 提交于 2019-12-13 16:08:56

问题


I am trying to use the AbotX crawler to crawl a site where I need to render the javascript and the press a span tag on it.

I've used the Abot crawler a lot and expected to having to overwrite some of the classes just as I have on previous occasions had to expand on for instance, the CrawlDecisionMaker.

But I can't seem to find out where to start, I expect I have to writing something like:

var implemnts = new ImplementationOverride(config);
implemnts.JavascriptRenderer = new PhantomJsRendererTest(config);

and extend the PhantomJsRenderer to be able to get it to click a button on the rendered page.

But since I can not see what the existing class does or find any information about where to start, I am completely lost.

I wanted to tag the post Abot and AbotX, but I do not have the required reputation for that.


回答1:


AbotX (with IsJavascriptRenderingEnabled = true) will retrieve the page source and execute all javascript to get a final page rendering. This final rendering includes all content (html/css) even content that was added/modified using javascript. It does not allow you to execute javascript any further.

If you need to interact with the page further then a web crawler is not likely what you need. You would need to use a browser automation framework like PhantomJs or Selenium. Those are headless browsers that will allow you to interact with the page just like you can in a browser.



来源:https://stackoverflow.com/questions/35484689/how-to-overwrite-and-perform-actions-on-webpage-with-abotx-javascriptrendering

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!