I want to parse a website\'s followers count with BeautifulSoup. This is what I have so far:
username_extract = \'lazada_my\'
url = \'https://www.instagram.
soup.find('head', attrs={'class':'count'}) searches for something that looks like , which doesn't exist anywhere in the HTML. The data you're after is contained in the tag that starts with window._sharedData:
script = soup.find('script', text=lambda t: t.startswith('window._sharedData'))
From there, you can just strip off the variable assignment and the semicolon to get valid JSON:
#
# ^^^
# JSON
page_json = script.text.split(' = ', 1)[1].rstrip(';')
Parse it and everything you need is contained in the object:
import json
data = json.loads(page_json)
follower_count = data['entry_data']['ProfilePage'][0]['user']['followed_by']['count']