Python Scrape With Normal Chrome or IE Browser (Not Chromedriver)

只谈情不闲聊 提交于 2021-02-19 16:33:39

问题


I'm using Selenium and Chromdriver which will load the site just fine including the Javascript loaded data.

My problem is a normal Chrome browser will update over time when this data changes and the Chromedriver holds the first static data it was sent. I haven't had any better luck with PhantomJS or firefox as a webdriver.

So is there anyway to use a normal Chrome browser? Or even IE?

I know in theory I could have it load a Chrome browser and watch the network traffic for the data I'm looking for but that seems pretty ugly.

I could have it constantly refresh the page but that doesn't seem neat and means more bandwidth usage and loading times.

I don't need it to be headless or have complex automation. Just load the page and let me read the updating data rather than just whatever it first saw.

I know Auto Hot Key supports this and that's my fallback at this stage. A macro script to dump the data into a file for Python to read.

来源:https://stackoverflow.com/questions/35804751/python-scrape-with-normal-chrome-or-ie-browser-not-chromedriver

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!