Scrapy Unit Testing

Question

I\'d like to implement some unit tests in a Scrapy (screen scraper/web crawler). Since a project is run through the \"scrapy crawl\" command I can run it through something

Answer 1

https://github.com/ThomasAitken/Scrapy-Testmaster

This is a package I wrote that significantly extends the functionality of the Scrapy Autounit library and takes it in a different direction (allowing for easy dynamic updating of testcases and merging the processes of debugging/testcase-generation). It also includes a modified version of the Scrapy parse command (https://docs.scrapy.org/en/latest/topics/commands.html#std-command-parse)