Scrapy:How to print request referrer

后端 未结 2 755
遇见更好的自我
遇见更好的自我 2020-12-15 20:05

Is it possible to get the request referrer from the response object in parse function?

10x

2条回答
  •  悲哀的现实
    2020-12-15 20:33

    HTTP Referer field is set up by HTTP client in request headers, not in response headers, as this header tells server where did client come from to current page.

    It would be rather weird to receive http Referer header in response.

    But when talking about scrapy, there's a reference to Request object on which the Response was generated, in response's request field, so the next call result:

    response.request.headers.get('Referer', None)
    

    can contain Referer header if it was set when making request.

提交回复
热议问题