jsoup don't get full data

问题

I have a project for school to parse web code and use it like a data base. When I tried to down data from (https://www.marathonbet.com/en/betting/Football/), I didn't get it all?

Here is my code:

Document doc = Jsoup.connect("https://www.marathonbet.com/en/betting/Football/").get();
Elements newsHeadlines = doc.select("div#container_EVENTS");

for (Element e: newsHeadlines.select("[id^=container_]")) {
    System.out.println(e.select("[class^=block-events-head]").first().text());
    System.out.println(e.select("[class^=foot-market]").select("[class^=event]").text());
} 

for result you get (this is last of displayed leagues):

Football. Friendlies. Internationals All bets Main bets
1. USA 2. Mexico 16 Apr 01:30 +124 7/5 23/10 111/50 +124

on top of her are all leagues displayed.

Why don't I get full data? Thank you for your time!

回答1:

Jsoup has a default body response limit of 1MB. You can change it to whatever you need with maxBodySize(int)

Set the maximum bytes to read from the (uncompressed) connection into the body, before the connection is closed, and the input truncated. The default maximum is 1MB. A max size of zero is treated as an infinite amount (bounded only by your patience and the memory available on your machine).

E.g.:

Document doc = Jsoup.get(url).userAgent(ua).maxBodySize(0).get();

You might like to look at the other options in Connection, on how to set request timeouts, the user-agent, etc.

来源：https://stackoverflow.com/questions/29656765/jsoup-dont-get-full-data