CrawlerProcess vs CrawlerRunner

Question

Scrapy 1.x documentation explains that there are two ways to run a Scrapy spider from a script:

• using CrawlerProcess
• using CrawlerRunner

Answer 1

CrawlerRunner:

This class shouldn’t be needed (since Scrapy is responsible of using it accordingly) unless writing scripts that manually handle the crawling process. See Run Scrapy from a script for an example.

CrawlerProcess:

This utility should be a better fit than CrawlerRunner if you aren’t running another Twisted reactor within your application.

It sounds like the CrawlerProcess is what you want unless you're adding your crawlers to an existing Twisted application.