Executing Javascript Submit form functions using scrapy in python

前端 未结 3 1320
栀梦
栀梦 2020-12-02 07:13

I am scrapping a site using scrapy framework and having trouble clicking on a javascript link for opening another page.

I can identify the code on the page as:

3条回答
  •  星月不相逢
    2020-12-02 07:42

    As far as I know, scrappy crawler implemented over urrlib2 and urllib obviously dont work with js. For working with js you can use qt webkit or selenium for example. Or you could find all ajax links on page and see how implemented a data exchange with the server and send response to server api indirectly.

提交回复
热议问题