mechanize

python mechanize forms() err

由 情到浓时终转凉″ 提交于 2019-12-07 22:29:40
问题 I'm using Python 2.7.6 and mechanize 0.2.5 and I want to log in to 'dining.ut.ac.ir' (I have the username and password)- but when I try to run the below script to get the forms list : import mechanize br = mechanize.Browser() br.set_handle_robots(False) br.addheaders = [('User-agent', 'Firefox')] br.open("http://dining.ut.ac.ir/") br.forms() I get this error: Traceback (most recent call last): File "script.py", line 8, in <module> br.forms() File "/home/arman/workspace/python/mechanize/venv

Possible to use timeout in WWW::Mechanize on https?

由 浪尽此生 提交于 2019-12-07 20:43:29
We have a Perl script which uses WWW::Mechanize to download content from a secured (https) website via our company proxy using POST action in WWW::Mechanize. Sometimes this post action runs for hours for unknown reasons. I want to control this. I checked for timeout but I also read in one of the post in Stackoverflow that it does not work with https websites. Any idea how I can use the timeout mechanism? I want to stop processing that link say after a minute or so to proceed further and not wait for hours together. Any help would be really appreciated. Use LWPx::ParanoidAgent and mix it into

Using Python Requests to Select Forms

由 本秂侑毒 提交于 2019-12-07 14:48:28
问题 I would like to use a python library capable of filling out forms and handling redirects: The "home" page has a form {'username':'user', 'password':'pass'} The "redirect" page brings me to a new page The "new" page has a link to the final page The "final" page has a form {'Field 1':'Data 1', 'Field 2':'Data 2'} I would like to get to the "final" page and fill out the form. I have already looked through every post in SO for python-requests, read the API doc and the entire user-guide. I have

mechanize open Url python

由 ☆樱花仙子☆ 提交于 2019-12-07 14:08:51
问题 I am trying to open a URL using mechanize in python. The code executes with no errors, but nothing actually happens. What am I missing? Also, is there a way to set the browser? This is python 2.7. import mechanize url='http://www.google.com/' op = mechanize.Browser() # use mecahnize's browser op.set_handle_robots(False) #tell the webpage you're not a robot op.open(url) 回答1: mechanize doesn't use real browsers - it is a tool for programmatic web-browsing. For example, print out the page title

Nokogiri Error: undefined method `radiobutton_with' - Why?

由 时间秒杀一切 提交于 2019-12-07 12:59:50
问题 I try to access a form using mechanize (Ruby). On my form I have a gorup of Radiobuttons. So I want to check one of them. I wrote: target_form = (page/:form).find{ |elem| elem['id'] == 'formid'} target_form.radiobutton_with(:name => "radiobuttonname")[2].check In this line I want to check the radiobutton with the value of 2. But in this line, I get an error: : undefined method `radiobutton_with' for #<Nokogiri::XML::Element:0x9b86ea> (NoMethodError) 回答1: The problem occured because using a

SSL errors with Mechanize

由 邮差的信 提交于 2019-12-07 12:10:28
问题 I got those commands on irb require 'mechanize' agent = Mechanize.new agent.get('https://monabo.lemonde.fr/customer/account/forgotpassword/') I got this error: OpenSSL::SSL::SSLError: SSL_connect returned=1 errno=0 state=unknown state: sslv3 alert handshake failure I tried on mac, and it works I don't have this error. However, it doesn't work on my computer (running Linux Mint 17). What I tried: Exporting this variable: export SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt Setting this

Save WWW::Mechanize::File to disk using FileUtils

由 蹲街弑〆低调 提交于 2019-12-07 12:03:00
问题 Using Mechanize with Ruby I get a certain file using agent.get('http://example.com/foo.torrent') , with FileUtils or otherwise, how do I save this file to my hard drive (for instance, in a directory wherefrom the script is running)? P.S. class => WWW::Mechanize::File 回答1: Well, WWW::Mechanize::File has a save_as instance method, so I suppose something like this might work: agent.get('http://example.com/foo.torrent').save_as 'a_file_name' 回答2: Please note that the Mechanize::File class is not

How to get Mechanize to auto-convert body to UTF8?

由 送分小仙女□ 提交于 2019-12-07 11:10:48
问题 I found some solutions using post_connect_hook and pre_connect_hook , but it seems like they don't work. I'm using the latest Mechanize version (2.1). There are no [:response] fields in the new version, and I don't know where to get them in the new version. https://gist.github.com/search?q=pre_connect_hooks https://gist.github.com/search?q=post_connect_hooks Is it possible to make Mechanize return a UTF8 encoded version, instead of having to convert it manually using iconv ? 回答1: Since

“Out of Memory” error with mechanize

由 最后都变了- 提交于 2019-12-07 06:43:33
问题 I was trying to scrape some information from a website page by page, basically here's what I did: import mechanize MechBrowser = mechanize.Browser() Counter = 0 while Counter < 5000: Response = MechBrowser.open("http://example.com/page" + str(Counter)) Html = Response.read() Response.close() OutputFile = open("Output.txt", "a") OutputFile.write(Html) OutputFile.close() Counter = Counter + 1 Well, the above codes ended up throwing out "Out of Memory" error and in task manager it shows that the

Python Mechanize: how to select a dropdown list when two have the same name in web page?

由 那年仲夏 提交于 2019-12-07 06:21:25
问题 The html that I'm trying to make mechanize parse is: <select id="topic_id2" name="topics[]" title="blabla" tabindex="4" class="createSelect"> here go options But then right below it there is another dropdown, with the following code: <select id="topic_id3" name="topics[]" title="optional" tabindex="5" class="createSelect"> Now if it helps at all, I need not select any value from the latter one, since it is optional. When I try br = mechanize.Browser() br.select_form(name="form") br["topics[]"
订阅 mechanize