Python: Login to ajax website using request

Question

I am trying to connect a website which seems to be in Ajax. The html page I want to get has the same URL as the landing page, it just changes once you login. Here\'s my code :

Answer 1

It doesn't look like you sent the actual login request. Try something like:

URL = 'http://www.pogdesign.co.uk/cat/'
LOGIN_URL = 'http://www.pogdesign.co.uk/login/' # Or whatever the login request url is
payload = {' password': 'password', ' sub_login': 'Account Login', 'username': 'email'}

s = requests.Session()
s.post(LOGIN_URL, data=payload)
s.get(URL)
s.content
# >> your /cat/ content

The nice thing about Session is that it carries your cookies for you by default so once a session is authenticated it will continue working. I have an example at https://github.com/BWStearns/WhiteTruffleScraper which uses a session login.

You can find the login request URL by watching the traffic in developer tools and logging in.