1. 技术文章
  2. HtmlUnit not creating HtmlPage object

HtmlUnit not creating HtmlPage object

由 给你一囗甜甜゛ 提交于 2019-12-05 13:35:04

Exceptions are thrown for several reasons, wrong html, errors on script's page, resources not found such css, script files or images files (for example <img src="bla.gif"> <- bla.gif not found HTML404)

So we use those options to keep html navegating without stoping on first error/problem we use :

webClient.getOptions().setThrowExceptionOnScriptError(false);
webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);

You can also implement empty classes to stop htmlUnity go verbose on console about css/javaScript errors with:

webClient.setCssErrorHandler(new SilentCssErrorHandler());    
webClient.setJavaScriptErrorListener(new JavaScriptErrorListener(){});

The little sample test case:

@Test
public void TestCall() throws FailingHttpStatusCodeException, MalformedURLException, IOException {      
    WebClient webClient = new WebClient(BrowserVersion.CHROME);
    webClient.getOptions().setUseInsecureSSL(true); //ignore ssl certificate
    webClient.getOptions().setThrowExceptionOnScriptError(false);
    webClient.getOptions().setThrowExceptionOnFailingStatusCode(false);
    String url = "https://www.wearvr.com/#game_id=game_4";
    HtmlPage myPage = webClient.getPage(url);
    webClient.waitForBackgroundJavaScriptStartingBefore(200);
    webClient.waitForBackgroundJavaScript(20000);
    //do stuff on page ex: myPage.getElementById("main")
    //myPage.asXml() <- tags and elements
    System.out.println(myPage.asText());

}

Try to use a different browser, e.g.:

String myUrl = "https://www.wearvr.com/#game_id=game_4";
try (WebClient webClient = new WebClient(BrowserVersion.CHROME)) {
    HtmlPage myPage = ((HtmlPage) webClient.getPage(myUrl));
    System.out.println(myPage.asXml());
} catch (FailingHttpStatusCodeException | IOException e) {
    e.printStackTrace();
}

However, it could also be a bug in IE8 simulation

标签
javascript
html
web-scraping
htmlunit
headless-browser
易学教程内所有资源均来自网络或用户发布的内容，如有违反法律规定的内容欢迎反馈！
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!