How to scrape a website that requires login first with Python
First of all, I think it's worth saying that, I know there are a bunch of similar questions but NONE of them works for me... I'm a newbie on Python, html and web scraper. I'm trying to scrape user information from a website which needs to login first. In my tests I use scraper my email settings from github as examples. The main page is ' https://github.com/login ' and the target page is ' https://github.com/settings/emails ' Here are a list of methods I've tried ##################################### Method 1 import mechanize import cookielib from BeautifulSoup import BeautifulSoup import