crawl dynamic web page using htmlunit

后端 未结 2 549
你的背包
你的背包 2020-12-16 01:00

I am crawling data using HtmlUnit from a dynamic webpage, which uses infinite scrolling to fetch data dynamically, just like facebook\'s newsfeed. I used the following sente

2条回答
  •  长情又很酷
    2020-12-16 01:19

    I had similiar problem where the content were post-loaded during page scrolling. I solved it using:

    webClient.getCurrentWindow().setInnerHeight(Integer.MAX_VALUE);

提交回复
热议问题