How to Get Crawl content in Crawljax

问题

I have crawl Dynamic webpage using Crawljax. i can able to get crawl current id, status and dom. but i can't get the Website content.. Any one help me??

CrawljaxConfigurationBuilder builder =
            CrawljaxConfiguration.builderFor("http://demo.crawljax.com/");
    builder.addPlugin(new OnNewStatePlugin() {



        @Override
        public String toString() {
            return "Our example plugin";
        }

                @Override
                public void onNewState(CrawlerContext cc, StateVertex sv) {

                    LOG.info("Found a new dom! Here it is:\n{}", cc.getBrowser().getStrippedDom());
                       String name = cc.getCurrentState().getName();
String url = cc.getBrowser().getCurrentUrl();
System.out.println(cc.getCurrentState().getDom());
System.out.println("New State: " + name + "; url: " + url);
                }
    });
    CrawljaxRunner crawljax = new CrawljaxRunner(builder.build());
    crawljax.call();

How to get dynamic/java script Webpage content..

回答1:

We can able to get website source code cc.getBrowser().getStrippedDom()); or cc.getCurrentState().getDocument(); This coding are Return Source code (css/java script file)..

Not possible.Because its testing tool.This tool only check Text are available, assign temp data to Fields.

回答2:

To get the website content, use the following function:

cc.getCurrentState().getDom()

This function does not return a DOM node, but actually returns the page's HTML text instead. This is the right function to use if you want the page content, but it sounds like it returns a DOM node, so the name getDom is a misnomer. To get a DOM node instead, use:

cc.getCurrentState().getDocument()

which returns the Document DOM node.

You can retrieve the page content with:

cc.getCurrentState().getDocument().getTextContent()

(EDIT: This won't work -- getTextContent always returns null when called on Documents.)

来源：https://stackoverflow.com/questions/27936719/how-to-get-crawl-content-in-crawljax