Scrapy: Passing item between methods

Question

Suppose I have a Bookitem, I need to add information to it in both the parse phase and detail phase

def parse(self, response)
    data = json.loads(response)         


        

Answer 1

You can define variable in init method:

class MySpider(BaseSpider):
    ...

    def __init__(self):
        self.item = None

    def parse(self, response)
        data = json.loads(response)
        for book in data['result']:
            self.item = BookItem();
            self.item['id'] = book['id']
            url = book['url']
            yield Request(url, callback=self.detail)

    def detail(self, response):        
        hxs = HtmlXPathSelector(response)
        self.item['price'] = ....

Answer 2

There is an argument named meta for Request:

yield Request(url, callback=self.detail, meta={'item': item})

then in function detail, access it this way:

item = response.meta['item']

See more details here about jobs topic.