How to manage cookies with Jsoup?

后端 未结 2 895
一整个雨季
一整个雨季 2020-12-17 04:39

Is there a simple cookie manager in Jsoup that stores the cookies by host? the example in this thread is quite lacking.

2条回答
  •  南笙
    南笙 (楼主)
    2020-12-17 05:17

    I didn't find a standard solution that works with Jsoup. Here's my simple cookie handling using a HashMap. It's probably missing a bunch of functionalities but I hope it'll work well enough for my basic crawler:

    private static HashMap> host2cookies = new HashMap>();
    
    public static String[] DownloadPage(URL url) throws Exception
    {
        Connection con = Jsoup.connect(url.toString()).timeout(600000);
        loadCookiesByHost(url, con);
    
    
        Document doc = con.get();
        url = con.request().url();
    
        storeCookiesByHost(url, con);
    
        return new String[]{url.toString(), doc.html()};
    }
    
    private static void loadCookiesByHost(URL url, Connection con) {
        try {
            String host = url.getHost();
            if (host2cookies.containsKey(host)) {
                HashMap cookies = host2cookies.get(host);
                for (Entry cookie : cookies.entrySet()) {
                    con.cookie(cookie.getKey(), cookie.getValue());
                }
            }
        } catch (Throwable t) {
            // MTMT move to log
            System.err.println(t.toString()+":: Error loading cookies to: " + url);
        }
    }
    
    private static void storeCookiesByHost(URL url, Connection con) {
            try {
                String host = url.getHost();
                HashMap cookies = host2cookies.get(host);
                if (cookies == null) {
                    cookies = new HashMap();
                    host2cookies.put(host, cookies);
                }
                cookies.putAll(con.response().cookies());
            } catch (Throwable t) {
                // MTMT move to log
                System.err.println(t.toString()+":: Error saving cookies from: " + url);
            }    
    }   
    

提交回复
热议问题