Jsoup: select(div[class=rslt prod]) returns null when it shouldn't

问题

I am trying to select the all div with class="rlts prod" from this page http://www.amazon.fr/s/field-keywords=samsung

Document doc = Jsoup.connect("http://www.amazon.fr/s/field-keywords=samsung").get();
Elements divProd = doc.select("div[class=rslt prod]");      
System.out.println("\nsize: "+divProd.size());

But it returns 0 and it shouldn't, any idea why ?

example of what should be selected:

<div id="result_4" class="rslt prod" name="B006O9QNHU">
[...]
</div>

回答1:

You have to change the user agent, otherwise you get a differnt website from amazon.

Document doc = Jsoup.connect("http://www.amazon.fr/s/field-keywords=samsung")
        .userAgent("Mozilla/17.0") // you can use any other user agent here
        .get();

for( Element element : doc.select("div[class=rslt prod]") )
{
    System.out.println(element);
    System.out.println("");
}

Now the output is a list like

<div id="result_1" class="rslt prod" name="B007XOM6SU"> 
  ...
</div>

<div id="result_2" class="rslt prod" name="B006SXSF4Q"> 
  ...
</div>

...

来源：https://stackoverflow.com/questions/15367765/jsoup-selectdivclass-rslt-prod-returns-null-when-it-shouldnt