How to import Scrapy item keys in the correct order?

前端 未结 2 2030
小蘑菇
小蘑菇 2021-01-24 06:09

I am importing the Scrapy item keys from items.py, into pipelines.py. The problem is that the order of the imported items are differe

2条回答
  •  情书的邮戳
    2021-01-24 06:40

    The only way I could get this to work, was to use this solution in the following manner.

    My items.py file:

    from scrapy.item import Item, Field
    from collections import OrderedDict
    from types import FunctionType
    
    class StaticOrderHelper(type):
        # Requires Python3
        def __prepare__(name, bases, **kwargs):
            return OrderedDict()
    
        def __new__(mcls, name, bases, namespace, **kwargs):
            namespace['_field_order'] = [
                    k
                    for k, v in namespace.items()
                    if not k.startswith('__') and not k.endswith('__')
                        and not isinstance(v, (FunctionType, classmethod, staticmethod))
            ]
            return type.__new__(mcls, name, bases, namespace, **kwargs)
    
    class NewAdsItem(metaclass=StaticOrderHelper):
        AdId        = Field()
        DateR       = Field()
        AdURL       = Field()
    

    Then import the _field_order item into your piplines.py with:

    ...
    from adbot.items import NewAdsItem
    ...
    class DbPipeline(object):
        ikeys = NewAdsItem._field_order
        ...
        def createDbTable(self):
            print("Creating new table: %s" % self.dbtable )
            print("Keys in creatDbTable: \t%s" % ",".join(self.ikeys) )
            ...
    

    I can now create new DB tables in the correct order of appearance, without worrying of Python's weird way of sorting dicts in unexpected ways.

提交回复
热议问题