how to fix HTTP error fetching URL. Status=500 in java while crawling?
问题 I am trying to crawl the user's ratings of cinema movies of imdb from the review page: (number of movies in my database is about 600,000). I used jsoup to parse pages as below: (sorry, I didn't write the whole code here since it is too long) try { //connecting to mysql db ResultSet res = st .executeQuery("SELECT id, title, production_year " + "FROM title " + "WHERE kind_id =1 " + "LIMIT 0 , 100000"); while (res.next()){ ....... ....... String baseUrl = "http://www.imdb.com/search/title