python urllib2 document.login

问题

How would you go about logging into a website that is set up like so, using python urllib2

The below is the javascript handler on the form and a onsubmit. How would I process this in python?

<script>

function handleLogin() {document.login.un.value = document.login.username.value;document.login.width.value = screen.width;document.login.height.value = screen.height;}

</script>

Below is the html form with all the components to send as post. What holds me up is the onsubmit function.

<form id='login' name='login' method='post' onsubmit="handleLogin();" action='login.php'>

<input class="txtbox glow" type="text" id="username" name="username" value="blanksss">

<input class="txtbox glow" type="password" id="password" name="pw" size="18" autocomplete="off" onkeypress="checkCaps(event)">

<input class="checkbox" type="checkbox" id="rememberUn" name="rememberUn" checked="checked">

<input class="loginButton" type="submit" id="Login" name="Login" value="Login">

</form>

Python Code I have that works with easy sites without javascript is:

cj = cookielib.CookieJar()
opener = urllib2.build_opener(urllib2.HTTPCookieProcessor(cj))
opener.addheaders.append(('User-agent', 'Mozilla/4.0'))
opener.addheaders.append( ('Referer', 'https://login.site.com/') )
login_data = urllib.urlencode({'username' : 'mylogin',
                               'pw' : 'mypass',
                               'Login' : 'Login',
                               'rememberUn' : 'checked'
                               })
resp = opener.open('https://login.site.com/', login_data)
website = opener.open('https://eu1.site.com/a03/o')

bowl = BeautifulSoup(website)

Any thoughts on how to process javascript code then post the finished results to the action of the form?

Thanks

回答1:

use requests

import requests

data = {
      "username" : "youruser",
      "password" : "yourpass",
      "Login" : "Login",
}
r = requests.post(url, data=data)

this Script Will Work On Python 2.7

回答2:

Yeah, you can either use the web developer tools to try and figure out what is being sent to the server or you can use selenium webdriver to just drive an actual browser. You can even do this headless using xvfb.

from selenium import webdriver

browser = webdriver.Chrome()
browser.get('http://some_url.com')
browser.find_element_by_id('username').send_keys('your_login') 
browser.find_element_by_id('password').send_keys('your_password')
browser.find_element_by_id('Login').click() 

print browser.page_source
browser.close()

You need to have the chromedriver executable in your path.

来源：https://stackoverflow.com/questions/11804565/python-urllib2-document-login

标签

python

web-scraping

urllib2

cookielib