Running Scrapy tasks in Python

Question

My Scrapy script seems to work just fine when I run it in \'one off\' scenarios from the command line, but if I try running the code twice in the same python session I get t

Answer 1

crawler.start() starts Twisted reactor. There can be only one reactor.

If you want to run more spiders - use

another_spider = MyAnotherSpider()
crawler.queue.append_spider(another_spider)