Access django models inside of Scrapy

后端 未结 8 2244
执笔经年
执笔经年 2020-11-28 19:11

Is it possible to access my django models inside of a Scrapy pipeline, so that I can save my scraped data straight to my model?

I\'ve seen this, but I don\'t really

8条回答
  •  独厮守ぢ
    2020-11-28 19:50

    The opposite solution (setup scrapy in a django management command):

    # -*- coding: utf-8 -*-
    # myapp/management/commands/scrapy.py 
    
    from __future__ import absolute_import
    from django.core.management.base import BaseCommand
    
    class Command(BaseCommand):
    
        def run_from_argv(self, argv):
            self._argv = argv
            self.execute()
    
        def handle(self, *args, **options):
            from scrapy.cmdline import execute
            execute(self._argv[1:])
    

    and in django's settings.py:

    import os
    os.environ['SCRAPY_SETTINGS_MODULE'] = 'scrapy_project.settings'
    

    Then instead of scrapy foo run ./manage.py scrapy foo.

    UPD: fixed the code to bypass django's options parsing.

提交回复
热议问题