Requests - get content-type/size without fetching the whole page/content
问题 I have a simple website crawler, it works fine, but sometime it stuck because of large content such as ISO images, .exe files and other large stuff. Guessing content-type using file extension is probably not the best idea. Is it possible to get content-type and content length/size without fetching the whole content/page? Here is my code: requests.adapters.DEFAULT_RETRIES = 2 url = url.decode('utf8', 'ignore') urlData = urlparse.urlparse(url) urlDomain = urlData.netloc session = requests