How to make gocolly crawl slower

后端 未结 0 970
南方客
南方客 2021-02-04 15:43

I am using gocolly for harvesting data from my website, the challenge is, gocolly is too aggressive when crawling the URLs. I have added a RandomDelay

c.Limit(&am         


        
相关标签:
回答
  • 消灭零回复
提交回复
热议问题