So I was following a tutorial online but the content is outdated. I\'m stuck as to why it doesn\'t scrape the next page? here is the code.
Also I set the ROBOTSTXT_OB
