Scrapy Twisted ConnectionLost error
问题 I am learning scrapy and am having a hard time trying to figure out this issue. My spider will not crawl the macys website and keeps throwing the following error: [<twisted.python.failure.Failure twisted.internet.error.ConnectionLost: Connection to the other side was lost in a non-clean fashion: Connection lost.>] Things I've tried so far: Setting headers and robotstxt obey per this thread: Scrapy Shell: twisted.internet.error.ConnectionLost although USER_AGENT is set Changing the user agent