HTTP POST and GET with cookies for authentication in python

Question

I\'m trying to create a python program that logs in to my university\'s site using my id and password. This is the formal page for logging in: https://webapp.pucrs.br/consul

Answer 1

I would try using the requests library. The documentation is excellent, and the code ends up being much cleaner than with urllib*

$ pip install requests

Using a session (see comment by Piotr) that handles cookies on its own, the result looks like this

import requests
url_0 = "http://webapp.pucrs.br/consulta/principal.jsp"
url = "https://webapp.pucrs.br/consulta/servlet/consulta.aluno.ValidaAluno"
data = {"pr1": "123456789", "pr2": "1234"}

s = requests.session()
s.get(url_0)
r = s.post(url, data)

It seems to work fine, as I get a "Usuario inexistente" notice for pr1 123456789 and "Sehna inválida" with your user-number.

Answer 2

You have to use the same "opener" you have created for all your requests, and it will handle the cookies all by itself.

here is an extract of something i wrote recently

opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cookielib.CookieJar()))

# then for all requests

if postData:     
    pData =  urllib.urlencode(postData)
else:
    pData = None

httpReq = urllib2.Request(url, pData, self._headers)
page =  opener.open(httpReq)

Answer 3

Converting MatthieuW's Answer to Python 3 gives.

import urllib, http.cookiejar

opener = urllib.request.build_opener(urllib.request.HTTPCookieProcessor(http.cookiejar.CookieJar()))
# then for all requests

if postData:     
    pData =  urllib.parse.urlencode(postData)
else:
    pData = None

httpReq = urllib.request.Request(url, pData)
page =  opener.open(httpReq)

Answer 4

urllib is no good, use requests!

from requests import Request, Session

url = "https://webapp.pucrs.br/consulta/principal.jsp"
s = requests.Session()

p = dict(pb1 = 'dd', pb2 = 'cc')
r = s.get(url, params = p) 
# use the cert=/path/to/certificate.pem if you need it
# elsewhere use verify = False to bypass ssl verification

c = r.cookies

# Then send back a response using those same cookies

r = requests.get(other_url, cookies = c, verify = False)

Answer 5

I recommend you to use mechanize, it automatically handles sessions/cookies/logins for you, furthermore it provides a urllib-like API and e.g. form-filling, so you don't have to mess with the right POST-request, since it gets constructed by mechanize.