JSOUP - How to crawl a “login required” page using JSOUP

Question

I\'m having trouble at crawling a determined website I wish to crawl. The problem is: after successfully logging in to that website I can\'t access a link which requires a valid

Answer 1

Get the cookie after you login:

    Connection.Response loginForm = Jsoup.connect(url)
            .method(Connection.Method.GET)
            .execute();

    Connection.Response mainPage = Jsoup.connect(login-validation-url)
            .data("user", user)
            .data("senha", password)
            .cookies(loginForm.cookies())
            .execute();

    Map<String, String> cookies = mainPage.cookies();

    Document evaluationPage = Jsoup.connect(login-required-url)
            .cookies(cookies)
            .execute.parse();

   return evaluationPage;

When you get your second webpage, you also have to use the cookie:

(Source: I had this problem a few days ago)

So it's easier to just put the cookies in a Map:

Map<String, String> cookies = loginForm.cookies();

And submit the forms using these cookies.