mechanize

mechanize select form using id

由 亡梦爱人 提交于 2019-11-28 02:28:43
问题 I am working on mechanize with python. <form action="/monthly-reports" accept-charset="UTF-8" method="post" id="sblock"> The form here does not have a name. How can I parse the form using it's id ? 回答1: I found this as a solution for the same problem. br is the mechanize object: formcount=0 for frm in br.forms(): if str(frm.attrs["id"])=="sblock": break formcount=formcount+1 br.select_form(nr=formcount) I'm sure the loop counter method above could be done more pythonic, but this should select

Python, mechanize, proper syntax for setting multiple headers?

由 杀马特。学长 韩版系。学妹 提交于 2019-11-28 00:28:36
问题 I can't seem to find how to do this anywere, I am trying to set multiple headers with python's mechanize module, such as: br.addheaders = [('user-agent', ' Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.2.3) Gecko/20100423 Ubuntu/10.04 (lucid) Firefox/3.6.3')] br.addheaders = [('accept', 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8')] But it seems that it only takes the last br.addheaders.. so it only shows the 'accept' header, not the 'user-agent' header, which leads me

Use mechanize to submit form without control name

由 假如想象 提交于 2019-11-28 00:25:52
I'm trying to use mechanize for python to submit a form but the form control I need to fill in doesnt have a name assigned to it. <POST https://sample.com/anExample multipart/form-data <HiddenControl(post_authenticity_token=) (readonly)> <HiddenControl(iframe_callback=) (readonly)> <TextareaControl(<None>=)>> The control I'm trying to edit is the last control in the above object, <TextareaControl(<None>=)> . I've looked at the documentation and cant seem to find a way to assign a value to that control since it doesnt have a name associated with it. forms = [f for f in br.forms()] print forms[2

Installing mechanize for python 3.4

由 北慕城南 提交于 2019-11-27 20:33:55
I'm trying to retrieve the mechanize module for python 3.4. Can anybody guide me in the right direction and perhaps walk me through the steps that I would need to take in order to make the correct installation? I'm currently using Windows 10. Carlos Guzman unfortunately mechanize only works with Python 2.4, Python 2.5, Python 2.6, and Python 2.7. The good news is there are other projects you can take a look at: RoboBrowser , MechanicalSoup There are more alternatives in this thread as well: Are there any alternatives to Mechanize in Python? . (edit) Mechanize 0.4.0 was release a few days ago

How to handle IncompleteRead: in python

由 自作多情 提交于 2019-11-27 19:44:20
I am trying to fetch some data from a website. However it returns me incomplete read . The data I am trying to get is a huge set of nested links. I did some research online and found that this might be due to a server error (A chunked transfer encoding finishing before reaching the expected size). I also found a workaround for above on this link However, I am not sure as to how to use this for my case. Following is the code I am working on br = mechanize.Browser() br.addheaders = [('User-agent', 'Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1;Trident/5.0)')] urls = "http://shop.o2.co.uk

Programmatic Python Browser with JavaScript

由 落花浮王杯 提交于 2019-11-27 18:51:34
I want to screen-scrape a web-site that uses JavaScript. There is mechanize , the programmatic web browser for Python. However, it (understandably) doesn't interpret javascript. Is there any programmatic browser for Python which does? If not, is there any JavaScript implementation in Python that I could use to attempt to create one? You might be better off using a tool like Selenium to automate the scraping using a web browser, so the JS executes and the page renders just like it would for a real user. The PyV8 package nicely wraps Google's V8 Javascript engine for Python. It's particularly

Python Auto Fill with Mechanize

由 痴心易碎 提交于 2019-11-27 16:57:35
问题 Could someone help me or share some code to auto fill a login with mechanize (http://wwwsearch.sourceforge.net/mechanize/)? I want to make a python script to log me into my favorite sites when I run it. Thanks! 回答1: This will help you to login to one site and download a page for example: import mechanize br=mechanize.Browser() br.open('http://www.yourfavoritesite.com') br.select_form(nr=0) #check yoursite forms to match the correct number br['Username']='Username' #use the proper input type

using tor as a SOCKS5 proxy with python urllib2 or mechanize

由 ＊爱你＆永不变心＊ 提交于 2019-11-27 16:53:05
问题 My goal is to use python's mechanize with a tor SOCKS proxy. I am not using a GUI with the following Ubuntu version: Description: Ubuntu 12.04.1 LTS Release: 12.04 Codename: precise Tor is installed and is listening on port 9050 according to the nmap scan: Starting Nmap 5.21 ( http://nmap.org ) at 2013-01-22 00:50 UTC Nmap scan report for localhost (127.0.0.1) Host is up (0.000011s latency). Not shown: 996 closed ports PORT STATE SERVICE 22/tcp open ssh 80/tcp open http 3306/tcp open mysql

Python: Clicking a button with urllib or urllib2

由 风流意气都作罢 提交于 2019-11-27 14:05:09
问题 I want to click a button with python, the info for the form is automatically filled by the webpage. the HTML code for sending a request to the button is: INPUT type="submit" value="Place a Bid"> How would I go about doing this? Is it possible to click the button with just urllib or urllib2? Or will I need to use something like mechanize or twill? 回答1: Use the form target and send any input as post data like this: <form target="http://mysite.com/blah.php" method="GET"> ...... ...... ......

Maintaining cookies between Mechanize requests

由 两盒软妹~` 提交于 2019-11-27 13:03:53
问题 I'm trying to use the Ruby version of Mechanize to extract my employer's tickets from a ticket management system that we're moving away from that does not supply an API. Problem is, it seems Mechanize isn't keeping the cookies between the post call and the get call shown below: require 'rubygems' require 'nokogiri' require 'mechanize' @agent = Mechanize.new page = @agent.post('http://<url>.com/user_session', { 'authenticity_token' => '<token>', 'user_session[login]' => '<login>', 'user
订阅 mechanize