Access django models inside of Scrapy

后端 未结 8 2256
执笔经年
执笔经年 2020-11-28 19:11

Is it possible to access my django models inside of a Scrapy pipeline, so that I can save my scraped data straight to my model?

I\'ve seen this, but I don\'t really

8条回答
  •  谎友^
    谎友^ (楼主)
    2020-11-28 20:04

    Minor update to solve KeyError. Python(3)/Django(1.10)/Scrapy(1.2.0)

    from django.core.management.base import BaseCommand
    
    class Command(BaseCommand):    
        help = 'Scrapy commands. Accessible from: "Django manage.py". '
    
        def __init__(self, stdout=None, stderr=None, no_color=False):
            super().__init__(stdout=None, stderr=None, no_color=False)
    
            # Optional attribute declaration.
            self.no_color = no_color
            self.stderr = stderr
            self.stdout = stdout
    
            # Actual declaration of CLI command
            self._argv = None
    
        def run_from_argv(self, argv):
            self._argv = argv
            self.execute(stdout=None, stderr=None, no_color=False)
    
        def handle(self, *args, **options):
            from scrapy.cmdline import execute
            execute(self._argv[1:])
    

    The SCRAPY_SETTINGS_MODULE declaration is still required.

    os.environ.setdefault('SCRAPY_SETTINGS_MODULE', 'scrapy_project.settings')
    

提交回复
热议问题