How to give delay between each requests in scrapy?

后端 未结 6 1063
感情败类
感情败类 2020-12-23 13:29

I don\'t want to crawl simultaneously and get blocked. I would like to send one request per second.

6条回答
  •  青春惊慌失措
    2020-12-23 14:12

    You can also set 'download_delay' attribute on spider if you don't want a global download delay. See http://doc.scrapy.org/en/latest/faq.html#what-does-the-response-status-code-999-means

提交回复
热议问题