Access session cookie in scrapy spiders

Question

I am trying to access the session cookie within a spider. I first login to a social network using in a spider:

    def parse(self, response):

        retur         


        

Answer 1

Maybe this is an overkill, but i don't know how are you going to use those cookies, so it might be useful (an excerpt from real code - adapt it to your case):

from scrapy.http.cookies import CookieJar

class MySpider(BaseSpider):

    def parse(self, response):

        cookieJar = response.meta.setdefault('cookie_jar', CookieJar())
        cookieJar.extract_cookies(response, response.request)
        request = Request(nextPageLink, callback = self.parse2,
                      meta = {'dont_merge_cookies': True, 'cookie_jar': cookieJar})
        cookieJar.add_cookie_header(request) # apply Set-Cookie ourselves

CookieJar has some useful methods.

If you still don't see the cookies - maybe they are not there?

---

UPDATE:

Looking at CookiesMiddleware code:

class CookiesMiddleware(object):
    def _debug_cookie(self, request, spider):
        if self.debug:
            cl = request.headers.getlist('Cookie')
            if cl:
                msg = "Sending cookies to: %s" % request + os.linesep
                msg += os.linesep.join("Cookie: %s" % c for c in cl)
                log.msg(msg, spider=spider, level=log.DEBUG)

So, try request.headers.getlist('Cookie')