Scrapy image download how to use custom filename

前端 未结 6 1248
眼角桃花
眼角桃花 2020-11-28 07:27

For my scrapy project I\'m currently using the ImagesPipeline. The downloaded images are stored with a SHA1 hash of their URLs as the file names.

How can I s

6条回答
  •  野趣味
    野趣味 (楼主)
    2020-11-28 07:54

    I rewrite the code, changing, in thumb_path def, "response." by "request.". If no, it won't work because "response is set to None".

    class MyImagesPipeline(ImagesPipeline):
    
        #Name download version
        def file_path(self, request, response=None, info=None):
            #item=request.meta['item'] # Like this you can use all from item, not just url.
            image_guid = request.url.split('/')[-1]
            return 'full/%s' % (image_guid)
    
        #Name thumbnail version
        def thumb_path(self, request, thumb_id, response=None, info=None):
            image_guid = thumb_id + request.url.split('/')[-1]
            return 'thumbs/%s/%s.jpg' % (thumb_id, image_guid)
    
        def get_media_requests(self, item, info):
            #yield Request(item['images']) # Adding meta. Dunno how to put it in one line :-)
            for image in item['images']:
                yield Request(image)
    

提交回复
热议问题