How to use PyCharm to debug Scrapy projects

Question

I am working on Scrapy 0.20 with Python 2.7. I found PyCharm has a good Python debugger. I want to test my Scrapy spiders using it. Anyone knows how to do that please?

<

Answer 1

You just need to do this.

Create a Python file on crawler folder on your project. I used main.py.

• Project
  • Crawler
    • Crawler
      • Spiders
      • ...
    • main.py
    • scrapy.cfg

Inside your main.py put this code below.

from scrapy import cmdline    
cmdline.execute("scrapy crawl spider".split())

And you need to create a "Run Configuration" to run your main.py.

Doing this, if you put a breakpoint at your code it will stop there.