Python requests login with redirection

Question

Here is a site http://pro.wialon.com/ where I want to login with python requests module. Login and pass are demo.

import requests
with requests.Session()as          


        

Answer 1

I was also facing the same thing. Our product actually redirects the login auth request to a third party application and if login credentials are valid, third-party app again redirects to our application with required cookies.

A solution I got is, use selenium in a headless browser to enter login credentials. And click login using selenium. Then redirection and authentication will happen and then your browser will receive the required cookie. Now just get the cookie using driver.getcookies() and store it in a variable. Then set this cookie in the header and use this header for future REST API calls.

Done!

Answer 2

The post url is incorrect and you are missing form data, you need to also do an initial request, post to the correct url and then get http://pro.wialon.com/service.html:

data = {"user": "demo",
    "passw": "demo",
    "submit": "Enter",
    "lang": "en",
    "action": "login"}

 head = {"User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"}

with requests.Session() as c:
    c.get('http://pro.wialon.com/')
    url = 'http://pro.wialon.com/login_action.html'
    c.post(url, data=data, headers=head)
    print(c.get("http://pro.wialon.com/service.html").content)

You can see the post in chrome dev tools under the network tab:

Also the default for post or get requests is to allow redirects so you don't need to specify it here.

You can see in the login page source, the form action:

<form class="login_bg_form" id="login_form" action="login_action.html" method="POST">

Instead of hard coding the path we can parse it from the form, use bs4:

import requests
from bs4 import BeautifulSoup
from urlparse import urljoin

data = {"user": "demo",
        "passw": "demo",
        "submit": "Enter",
        "lang": "en",
        "action": "login"}

head = {"User-Agent":"Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"}

with requests.Session()as c:
    soup = BeautifulSoup(c.get('http://pro.wialon.com/').content)
    redir = soup.select_one("#login_form")["action"]
    url = 'http://pro.wialon.com/login_action.html'
    c.post(url, data=data, headers=head)
    print(c.get(urljoin("http://pro.wialon.com/", redir)).content)

The only problem now is the data is mostly populated using ajax requests so if you want to scrape data you will need to mimic the requests.