How to save Scrapy crawl Command output

Question

I am trying to save the output of the scrapy crawl command I have tried scrapy crawl someSpider -o some.json -t json >> some.text But it doesn\'t worked ...c

Answer 1

For all scrapy commands you can add --logfile NAME_OF_FILE to log to a file e.g.

scrapy crawl someSpider -o some.json --logfile some.text

There are two other useful command line options for logging:

• -L or --loglevel to control the logging level e.g. -L INFO (the default is DEBUG)

• --nolog to disable logging completely

These commands are documented here.