Web Scraping with Selenium Python [Twitter + Instagram]

Question

I am trying to web scrape both Instagram and Twitter based on geolocation. I can run a query search but I am having challenges in reloading the web page to to more and store

Answer 1

I managed to make it work using requests. Your code would look something like this:

from bs4 import BeautifulSoup
import requests

query = "geocode%3A35.68501%2C139.7514%2C30km%20since:2016-03-01%20until:2016-03-02&f=tweets"

twitter = 'https://twitter.com/search?q=' + query

content = requests.get(twitter)
soup = BeautifulSoup(content.text)

print(soup)

Then you can use the soup object to parse what you need. The same thing should work for Instagram, if your query is correct.