Scrape title by only downloading relevant part of webpage

Question

I would like to scrape just the title of a webpage using Python. I need to do this for thousands of sites so it has to be fast. I\'ve seen previous questions like retrieving jus

Answer 1

You're scraping webpages using standard REST requests and I'm not aware of any request that only returns the title, so I don't think it's possible.

I know this doesn't necessarily help get the title only, but I usually use BeautifulSoup for any web scraping. It's much easier. Here's an example.

Code:

import requests
from bs4 import BeautifulSoup

urls = ["http://www.google.com", "http://www.msn.com"]

for url in urls:
    r = requests.get(url)
    soup = BeautifulSoup(r.text, "html.parser")

    print "Title with tags: %s" % soup.title
    print "Title: %s" % soup.title.text
    print

Output:

Title with tags: 
Title: Google

Title with tags: 
Title: MSN.com - Hotmail, Outlook, Skype, Bing, Latest News, Photos & Videos