How do I scrape a full instagram page in python?

前端 未结 3 2056
独厮守ぢ
独厮守ぢ 2021-01-01 09:25

Long story short, I\'m trying to create an Instagram python scraper, that loads the entire page and grabs all the links to the images. I have it working, only problem is, it

3条回答
  •  清酒与你
    2021-01-01 09:54

    @zero's answer is incomplete (at least as of 1/15/19). c.send_keys is not a valid method. Instead, this is what I did:

    c = webdriver.Chrome()
    c.get(some_url)
    
    element = c.find_element_by_tag_name('body') # or whatever tag you're looking to scrape from
    
    for i in range(10):
        element.send_keys(Keys.END)
        time.sleep(1)
    
    soup = BeautifulSoup(c.page_source, 'html.parser')
    

提交回复
热议问题