mechanize

问题 I am trying to submit a login form on a web page that looks something like this. I have also tried submit the nested form as well as submit both forms, same error every time. <form method="post" name="loginform"> <input type='hidden' name='login' value='1'> <form action="#" method="post" id="login"> Username <input type="text" name="username" id="username" /> Password <input type="password" name="password" id="password" /> <input type="submit" value='Login' class="submit" /> here is my the

问题 I'm trying to get a page with an ISO-8859-1 encoding clicking on a link, so the code is similar to this: page_result = page.link_with( :text => 'link_text' ).click So far I get the result with a wrong encoding, so I see characters like: 'T�tulo:' instead of 'Título:' I've tried several approaches, including: Stating the encoding in the first request using the agent like: @page_search = @agent.get( :url => 'http://www.server.com', :headers => { 'Accept-Charset' => 'ISO-8859-1' } ) Stating the

问题 I'm trying to get a page with an ISO-8859-1 encoding clicking on a link, so the code is similar to this: page_result = page.link_with( :text => 'link_text' ).click So far I get the result with a wrong encoding, so I see characters like: 'T�tulo:' instead of 'Título:' I've tried several approaches, including: Stating the encoding in the first request using the agent like: @page_search = @agent.get( :url => 'http://www.server.com', :headers => { 'Accept-Charset' => 'ISO-8859-1' } ) Stating the

问题 I know how to do a HEAD request with httplib, but I have to use mechanize for this site. Essentially, what I need to do is grab a value from the header (filename) without actually downloading the file. Any suggestions how I could accomplish this? 回答1: Mechanize itself only sends GETs and POSTs, but you can easily extend the Request class to send HEAD. Example: import mechanize class HeadRequest(mechanize.Request): def get_method(self): return "HEAD" request = HeadRequest("http://www.example

问题 I am trying to implement a Ruby script that will take in a username and password, then proceed to fill in the account details on a login form on another website and return the then follow a link and retrieve the account history. To do this I am using the Mechanize gem. I have been following the examples here but still I cant seem to get it to work. I have simplified this down greatly to try get it to work in parts but a supposedly simple filling in a form is holding me up. Here is my code: #

问题 I recently wanted to port my scraper that uses mechanize.Browser to Python 3. I haven't found any python3-mechanize package or anything like that. I'm not yet familiar with Python module building with some special procedures so I don't know how would I use 2to3 there. Any suggestions? @edit: I'm accepting Sibi's answer because it is currently the best. I will be pleased to award any other answer this way, if it just points to a better solution. 回答1: There's been no activity in the Mechanize

问题 I was wondering if there is something like Perl's/Python's mechanize for Java. Thanks! 回答1: Take a look at HtmlUnit. It is similar to Perl's WWW::Mechanize. There is also a Perl version of it named WWW::HtmlUnit (it uses Inline::Java to expose the Java library's methods to Perl). 回答2: Yes, there is now! I too wanted mechanize in Java and we've written it. Please check out http://gistlabs.com/software/mechanize-for-java/ (and the GitHub here https://github.com/GistLabs/mechanize). Feedback,

问题 I wrote a script that will go through all of the customers in our database, verify that their website URL works, and try to find a twitter link on their homepage. We have a little over 10,000 URLs to verify. After a fraction of if the urls are verified, we start getting getaddrinfo errors for every URL. Here's a copy of the code that scrapes a single URL: def scrape_url(url) url_found = false twitter_name = nil begin agent = Mechanize.new do |a| a.follow_meta_refresh = true end agent.get

问题 I wrote a script that will go through all of the customers in our database, verify that their website URL works, and try to find a twitter link on their homepage. We have a little over 10,000 URLs to verify. After a fraction of if the urls are verified, we start getting getaddrinfo errors for every URL. Here's a copy of the code that scrapes a single URL: def scrape_url(url) url_found = false twitter_name = nil begin agent = Mechanize.new do |a| a.follow_meta_refresh = true end agent.get

问题 I'm having issues submitting the result of a form submission (I can submit a form, but I can't submit the form on the page that follows the first). I have: browser = mechanize.Browser() browser.set_handle_robots(False) browser.open('https://www.example.com/login') browser.select_form(nr=0) browser.form['j_username'] = 'username' browser.form['j_password'] = 'password' req = browser.submit() This works, as print req results in ` <body onload="document.forms[0].submit()"> <noscript> <p> <strong