Scraping from a website that requires a login?

Question

Can this be done if so, how? I want to scrape data from xbox.com but the pages I need to scrape only appear after a successful login.

Answer 1

You will have to go through the required login transaction by sending POST data with your CURL requests. That said, it is a bad idea to scrape data from behind a login - the site didn't put that information in the public for a reason, and for you to do so might constitute copyright infringement,