urllib2

urllib2 with cookies

可紊 提交于 2019-11-30 20:39:27
I am trying to make a request to an RSS feed that requires a cookie, using python. I thought using urllib2 and adding the appropriate heading would be sufficient, but the request keeps saying unautherized. Im guessing it could be a problem on the remote sites' side, but wasnt sure. How do I use urllib2 along with cookies? is there a better package for this (like httplib, mechanize, curl) import urllib2 opener = urllib2.build_opener() opener.addheaders.append(('Cookie', 'cookiename=cookievalue')) f = opener.open("http://example.com/") I would use requests package, docs , it's a lot easier to

urllib downloading contents of an online directory

天大地大妈咪最大 提交于 2019-11-30 20:26:07
I'm trying to make a program that will open a directory, then use regular expressions to get the names of powerpoints and then create files locally and copy their content. When I run this it appears to work, however when I actually try to open the files they keep saying the version is wrong. from urllib.request import urlopen import re urlpath = urlopen('http://www.divms.uiowa.edu/~jni/courses/ProgrammignInCobol/presentation/') string = urlpath.read().decode('utf-8') pattern = re.compile('ch[0-9]*.ppt') #the pattern actually creates duplicates in the list filelist = pattern.findall(string)

How to download a webpage that require username and password?

我与影子孤独终老i 提交于 2019-11-30 19:47:16
问题 For example, I want to download this page after inserting username and password: http://forum.ubuntu-it.org/ I have tryed with wget but doesn't work. Is there a solution with python ? You can test with these username and password: username: johnconnor password: hellohello 回答1: Like @robert says, use mechanize. To get you started: from mechanize import Browser b = Browser() b.open("http://forum.ubuntu-it.org/index.php") b.select_form(nr=0) b["user"] = "johnconnor" b["passwrd"] = "hellohello" b

Trying to post multipart form data in python, won't post

 ̄綄美尐妖づ 提交于 2019-11-30 17:57:50
问题 I'm fairly new to python, so I apologize in advance if this is something simple I'm missing. I'm trying to post data to a multipart form in python. The script runs, but it won't post. I'm not sure what I'm doing wrong. import urllib, urllib2 from poster.encode import multipart_encode from poster.streaminghttp import register_openers def toqueXF(): register_openers() url = "http://localhost/trunk/admin/new.php" values = {'form':open('/test.pdf'), 'bandingxml':open('/banding.xml'), 'desc':

HandShake Failure in python(_ssl.c:590)

丶灬走出姿态 提交于 2019-11-30 17:28:56
When I execute the below line, req = urllib2.Request(requestwithtoken) self.response = urllib2.urlopen(req,self.request).read() I am getting the following exception: SSLError: [SSL: SSLV3_ALERT_HANDSHAKE_FAILURE] sslv3 alert handshake failure (_ssl.c:590) The thing is I am able to get the token by pinging the service by using curl . During the process of retrieving the token, all the certificates were verified. In turn, by using the generated token, i am not able to connect to the service. I am getting the above error while trying. What could be the reason for that? martin I was having the

How to “keep-alive” with cookielib and httplib in python?

生来就可爱ヽ(ⅴ<●) 提交于 2019-11-30 16:36:52
In python, I'm using httplib because it "keep-alive" the http connection (as oppose to urllib(2)). Now, I want to use cookielib with httplib but they seem to hate each other!! (no way to interface them together). Does anyone know of a solution to that problem? HTTP handler for urllib2 that supports keep-alive Serge Broslavsky You should consider using the Requests library instead at the earliest chance you have to refactor your code. In the mean time; HACK ALERT! :) I'd go other suggested way, but I've done a hack (done for different reasons though), which does create an interface between

Python urllib2 cannot open localhost on alternate port (not 80)? Error 10013

孤街浪徒 提交于 2019-11-30 16:12:22
Here is my server.py : import BaseHTTPServer import SocketServer class TestRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler): def do_GET(self): self.wfile.write("hello world at %s" % __file__) server = BaseHTTPServer.HTTPServer(('', 10000), TestRequestHandler) #server = SocketServer.ThreadingTCPServer(('', 8888), TestRequestHandler) server.serve_forever() Here is my client.py : import urllib2 req = urllib2.Request('http://127.0.0.1:10000/') handle = urllib2.urlopen(req) content = handle.read() Then I start server.py, it works. When I start client.py, I get this error on Windows 7, Python 2

Python urllib2 cannot open localhost on alternate port (not 80)? Error 10013

不想你离开。 提交于 2019-11-30 16:00:34
问题 Here is my server.py : import BaseHTTPServer import SocketServer class TestRequestHandler(BaseHTTPServer.BaseHTTPRequestHandler): def do_GET(self): self.wfile.write("hello world at %s" % __file__) server = BaseHTTPServer.HTTPServer(('', 10000), TestRequestHandler) #server = SocketServer.ThreadingTCPServer(('', 8888), TestRequestHandler) server.serve_forever() Here is my client.py : import urllib2 req = urllib2.Request('http://127.0.0.1:10000/') handle = urllib2.urlopen(req) content = handle

How do I authenticate a urllib2 script in order to access HTTPS web services from a Django site?

99封情书 提交于 2019-11-30 15:57:48
问题 everybody. I'm working on a django/mod_wsgi/apache2 website that serves sensitive information using https for all requests and responses. All views are written to redirect if the user isn't authenticated. It also has several views that are meant to function like RESTful web services. I'm now in the process of writing a script that uses urllib/urllib2 to contact several of these services in order to download a series of very large files. I'm running into problems with 403: FORBIDDEN errors

Python Requests Multipart HTTP POST

我的梦境 提交于 2019-11-30 14:53:53
I was wondering how do you translate something like this using Python Requests? In urllib2, you can manually manipulate the data that is being sent over the wire to the API service, but Requests claims multipart file uploads are easy. However, when trying to send over the same request using the Requests library, I believe that it is not specifying some key parameters in the content-type for each of the two parts correctly. Can someone please shed some light on this matter. Thank you in advance! def upload_creative(self, account_id, file_path): """""" boundary = '-----------------------------'