get a browser rendered html+javascript

前端 未结 2 1775
无人及你
无人及你 2020-12-01 19:40

I need a comandline tool (or Javascript/PHP, but i think commandline is the one way) for render and get the rendered content of URL, but the important its I need to renderer

2条回答
  •  被撕碎了的回忆
    2020-12-01 20:25

    • Selenium : very complete solution with bindings in many languages
    • puppeteer : headless Chrome API, usable in NodeJS or as a command-line tool
    • HTtrack : command-line tool
    • Apache Notch & webmagic : open source Java web crawlers
    • pholcus : "distributed & high concurrency" web crawler written in Go
    • Xvfb a display server implementing the X11 display server protocol, without showing any screen output. I have used it successfully with Travis CI and Protractor as an example. Alternative: XDummy
    • PhantomJS (first suggested by nvuono) : can export the rendered page as non-HTML (pdf, png...). PhantomJS development is suspended until further notice (more details). Closely related: SlimerJS, CasperJS

    And there are many Python web scrapping libraries:

    • Scrapy
    • pyspider
    • ghost.py
    • splinter

提交回复
热议问题