urllib2

PIL / urllib2 - cannot identify image file when passing file using StringIO

半世苍凉 提交于 2019-12-06 12:06:59
问题 I'm downloading an image from the web using urllib2. Once I have downloaded it I want to do some stuff with it using an image module called PIL. I don't want to save the file to disk then reopen but rather pass it from memory using StringIO from PIL import Image image_buff = urllib2.urlopen(url) image = Image.open(StringIO.StringIO(image_buff)) However when I do this I get the following error IOError: cannot identify image file <StringIO.StringIO instance at 0x101afa2d8 I think this is

EVENTVALIDATION error while scraping asp.net page

依然范特西╮ 提交于 2019-12-06 11:18:38
问题 I need to get some values from this website. Basically I need to get the Area for every city. I am using Python and beautifulsoup for this. What I am doing is : First making a Get request to this page and getting __VIEWSTATE AND __EVENTVALIDATION to make a POST request to get cities for a particular state.Till here its working and I am getting cities for every states. To get Area I need to make another POST with new __VIEWSTATE AND __EVENTVALIDATION and this time i need to send city as well

Python SOAP request using urllib2

时光怂恿深爱的人放手 提交于 2019-12-06 11:07:47
I'm attempting to write a script to communicate with sharepoint via SOAP using urllib2 in Python. My code is connecting successfully to a sharepoint list, but does not do anything once connected. Could my SOAP request be wrong? It seems to be returning nothing, despite 2 list items existing on the sharepoint site. import urllib2 from ntlm import HTTPNtlmAuthHandler user = r'DOMAIN\myusername' password = "password" url = "https://mysecuresite.com/site/_vti_bin/Lists.asmx" passman = urllib2.HTTPPasswordMgrWithDefaultRealm() passman.add_password(None, url, user, password) # create the NTLM

getting value of location header using python urllib2

戏子无情 提交于 2019-12-06 10:46:01
when I use urllib2,and list the headers,I cannot see the 'Location' header. In [19]:p = urllib2.urlopen('http://www.example.com') In [21]: p.headers.items() Out[21]: [('transfer-encoding', 'chunked'), ('vary', 'Accept-Encoding'), ('server', 'Apache/2.2.3 (CentOS)'), ('last-modified', 'Wed, 09 Feb 2011 17:13:15 GMT'), ('connection', 'close'), ('date', 'Fri, 25 May 2012 03:00:02 GMT'), ('content-type', 'text/html; charset=UTF-8')] If I use telnet and GET telnet www.example.com 80 Trying 192.0.43.10... Connected to www.example.com. Escape character is '^]'. GET / HTTP/1.0 Host:www.example.com

urllib2.urlopen fails in Django

假如想象 提交于 2019-12-06 10:39:27
I use urllib2.urlopen(url) to get HTML content. The URL is http://127.0.0.1:8000/m.html/ . This method succeeds in getting the HTML content. But in Django, if I try to get the HTML content, it stops in the function: urllib2.urlopen('http://127.0.0.1:8000/m.html/'). It just stops. It does not report an error and the server also stops. I don't know why it works in a single file, but has problems in Django. The Django development server is single-threaded. It can't both serve the view that's requesting the separate page, and the seperate page itself. However, I've no idea why you would want to do

urllib2 doesn't use proxy (Fiddler2), set using ProxyHandler

点点圈 提交于 2019-12-06 09:48:22
I have Fiddler2 listening on 0.0.0.0:8888. try: data = '' proxy = urllib2.ProxyHandler({'http': '127.0.0.1:8888'}) //also tried {'http': 'http://127.0.0.1:8888/'} opener = urllib2.build_opener(proxy) urllib2.install_opener(opener) req = urllib2.Request('http://www.google.com') response = urllib2.urlopen(req) the_page = response.read() print the_page except Exception, detail: print "Err ", detail I don't see the GET or any request to google in Fiddler (but I can see other requests) is there a way to debug it? is seems like python bypasses Fiddler or ignores the proxy. I also configured WinHTTP

urllib2 timeout

此生再无相见时 提交于 2019-12-06 09:01:34
i'm using urllib2 library for my code, i'm using a lot of (urlopen) EDIT: loadurl i have a problem on my network, when i'm browsing sites, sometimes my browser gets stuck on "Connecting" to a certain website and sometimes my browser returns a timeout My question is if i use urllib2 on my code it can timeout when trying to connect for too long to a certain website or the code will get stuck on that line. i know that urllib2 can handle timeouts without specifying it on code but it can apply for this kind of situation ? Thanks for your time EDIT : def checker(self) try: html = self.loadurl("MY

Form Submission in Python Without Name Attribute

大城市里の小女人 提交于 2019-12-06 08:33:21
Background: Using urllib and urllib2 in Python, you can do a form submission. You first create a dictionary. formdictionary = { 'search' : 'stackoverflow' } Then you use urlencode method of urllib to transform this dictionary. params = urllib.urlencode(formdictionary) You can now make a url request with urllib2 and pass the variable params as a secondary parameter with the first parameter being the url. open = urllib2.urlopen('www.searchpage.com', params) From my understanding, urlencode automatically encodes the dictionary in html and adds the input tag. It takes the key to be the name

urllib2 won't use my proxy

你离开我真会死。 提交于 2019-12-06 08:02:11
I'm trying to open a URL with urllib2 using an opener I built with a HTTPS proxy, however it is requesting it with my normal IP, and not the proxy I give it. import urllib2 proxy = urllib2.ProxyHandler({'https': 'IP:PORT'}) opener = urllib2.build_opener(proxy) my_ip = opener.open('http://whatthehellismyip.com/?ipraw').read() print my_ip Can anyone please tell me what I am doing wrong here? You forgot to install opener. This should work: import urllib2 proxy = urllib2.ProxyHandler({'https': 'IP:PORT'}) opener = urllib2.build_opener(proxy) urllib2.install_opener(opener) my_ip = urllib2.urlopen(

urllib2 and cookielib thread safety

末鹿安然 提交于 2019-12-06 07:24:10
问题 As far as I've been able to tell cookielib isnt thread safe; but then again the post stating so is five years old, so it might be wrong. Nevertheless, I've been wondering - If I spawn a class like this: class Acc: jar = cookielib.CookieJar() cookie = urllib2.HTTPCookieProcessor(jar) opener = urllib2.build_opener(cookie) headers = {} def __init__ (self,login,password): self.user = login self.password = password def login(self): return False # Some magic, irrelevant def fetch(self,url): req =