Scrapy: Extract commented (hidden) content

Question

How can I extract content from within commented tags with scrappy ?

For instance, how to extract \"Yellow\" in the following example:

Answer 1

First of all, use below xpath to get all the comments from the page.

data = response.xpath('//comment()').extract()

Now, using any key value identity your meaning comments.

up_data = []
for d in data:
    if 'key' in d:
        up_data.append(d)

define,

html_template = '%s'
for up_d in up_data:
    up_d = html_template % up_d.replace('', '')
    sel = Selector(text=up_d)
    sel.xpath('//div[@class="table_outer_container"]')

    // DO what you want