一次scrapy失败的提示信息:由于连接方在一段时间后没有正确答复或连接的主机没有反 应,连接尝试失败

独自空忆成欢 提交于 2019-12-27 12:35:17

2017-10-31 19:09:26 [scrapy.extensions.logstats] INFO: Crawled 8096 pages (at 67 pages/min), scraped 7720 items (at 62 items/min)
2017-10-31 19:09:27 [scrapy.downloadermiddlewares.retry] DEBUG: Gave up retrying <GET http://collection.sina.com.cn/auction/zjgd/2017-10-19/doc-ifymzqpq2471870.shtml> (failed 3 times): TCP connection timed out: 10060: 由于连接方在一段时间后没有正确答复或连接的主机没有反应,连接尝试失败。.
2017-10-31 19:09:27 [scrapy.core.scraper] ERROR: Error downloading <GET http://collection.sina.com.cn/auction/zjgd/2017-10-19/doc-ifymzqpq2471870.shtml>: TCP connection timed out: 10060: 由于连接方在一段时间后没有正确答复或连接的主机没有反 应,连接尝试失败。.
2017-10-31 19:09:27 [scrapy.core.engine] INFO: Closing spider (finished)
2017-10-31 19:09:27 [scrapy.statscollectors] INFO: Dumping Scrapy stats:
{'downloader/exception_count': 126,
'downloader/exception_type_count/twisted.internet.error.DNSLookupError': 1,
'downloader/exception_type_count/twisted.internet.error.TCPTimedOutError': 125,
'downloader/request_bytes': 3278403,
'downloader/request_count': 8298,
'downloader/request_method_count/GET': 8298,
'downloader/response_bytes': 269632727,
'downloader/response_count': 8172,
'downloader/response_status_count/200': 8153,
'downloader/response_status_count/301': 5,
'downloader/response_status_count/302': 1,
'downloader/response_status_count/404': 13,
'dupefilter/filtered': 7040,
'finish_reason': 'finished',
'finish_time': datetime.datetime(2017, 10, 31, 11, 9, 27, 889696),
'httperror/response_ignored_count': 4,
'httperror/response_ignored_status_count/404': 4,
'item_scraped_count': 7720,
'log_count/DEBUG': 16020,
'log_count/ERROR': 197,
'log_count/INFO': 27,
'request_depth_max': 2,
'response_received_count': 8096,
'retry/count': 121,
'retry/max_reached': 5,
'retry/reason_count/twisted.internet.error.DNSLookupError': 1,
'retry/reason_count/twisted.internet.error.TCPTimedOutError': 120,
'scheduler/dequeued': 8274,
'scheduler/dequeued/memory': 8274,
'scheduler/enqueued': 8274,
'scheduler/enqueued/memory': 8274,
'start_time': datetime.datetime(2017, 10, 31, 10, 53, 26, 916526)}
2017-10-31 19:09:27 [scrapy.core.engine] INFO: Spider closed (finished)

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!