I am trying to log in to an ASP.NET website using the requests
module in Python.
While logging in manually in the website I can see the following header
I was initially using requests+bs4 as well however I was running into similar issues with the ASPX site I'm scrapping. I found another library called robobrowser that wraps requests+bs4. With this you no longer have to manually set items such as "__VIEWSTATE" and friends when interacting with ASPX sites.
from robobrowser import RoboBrowser
url = ' http://www11.davidsonsinc.com'
login_url = url + '/Login/Login.aspx'
username = "username"
password = "password"
browser = RoboBrowser(history=True)
# This retrieves __VIEWSTATE and friends
browser.open(login_url)
signin = browser.get_form(id='aspnetForm')
signin["ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$UserName"].value = username
signin["ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$Password"].value = password
signin["ctl00$ContentPlaceHolderNavPane$LeftSection$UserLogin$LoginButton"].value = "Log In"
browser.submit_form(signin)
print browser.url