Scrapy: constructing non-duplicative list of absolute paths from relative paths

Question

Question: how do I use Scrapy to create a non-duplicative list of absolute paths from relative paths under the img src tag?

<

Answer 1

What about:

def url_join(self,response):
    item=MyItem()
    item['url']=[]
    relative_url=response.xpath('//img/@src').extract()
    for link in relative_url:
        item['url'] = response.urljoin(link)
        yield item