Pick appropriate job queue implementation for my mutlithreaded scraper (.net Core)

Question

I`m building type of a scraper. As input scraper receives a link to an arbitrary page of the site (for example, https://www.wikipedia.org/ ).

The scraper loads the we