convert lxml to scrapy xxs selector

对着背影说爱祢 提交于 2019-12-11 08:14:35

问题


How can I convert this pure python lxml to scrapy built in xxs selectors? This one works but i want to convert this to the scrapy xxs selectors.

    def parse_device_list(self, response):
    self.log("\n\n\n List of devices \n\n\n")
    self.log('Hi, this is the parse_device_list page! %s' % response.url)
    root = lxml.etree.fromstring(response.body)
    for row in root.xpath('//row'):
        allcells = row.xpath('./cell')
        # first cell contain the link to follow
        detail_page_link = allcells[0].get("href")
        yield Request(urlparse.urljoin(response.url, detail_page_link ), callback=self.parse_page)

回答1:


Give it a try:

def parse_page(self, response):
    xxs = XmlXPathSelector(response)
    for row in xxs.select('//row'):
        detail_page_link = row.select('.//cell[1]/@href')[0].extract()
        yield Request(urlparse.urljoin(response.url, detail_page_link), callback=self.parse_page)


来源:https://stackoverflow.com/questions/17861781/convert-lxml-to-scrapy-xxs-selector

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!