问题
By default, scrapy stores and passes cookies along requests. But how do I access or clear the stored cookies at certain point in the spider? Thanks?
回答1:
to set cookies to a specific request use request cookies field for example from docs:
request_with_cookies = Request(url="http://www.example.com",
cookies={'currency': 'USD', 'country': 'UY'})
do access request cookies:
request.headers.getlist('Cookie')
response cookies:
response.headers.getlist('Set-Cookie')
for more details see cookies middleware
回答2:
one way to avoid the cookies on your next requests would be to use the meta argument dont_merge_cookies:
Request(url, meta={'dont_merge_cookies': True})
回答3:
Not quite clearing them, but setting COOKIES_ENABLED to false in scrapy settings will prevent them from being used.
来源:https://stackoverflow.com/questions/21924220/how-to-clear-cookies-in-scrapy