I am interested to do web crawling. I was looking at solr.
solr
Does solr do web crawling, or what are the steps to do web crawling?
You might also want to take a look at
http://www.crawl-anywhere.com/
Very powerful crawler that is compatible with Solr.