问题
I want to scrape html content from a couple of websites and view them on my website a kind of mashup. I will reference and link to them aswelll!
Thank you
回答1:
Go ahead and do it but check their robots.txt and make sure there is a way for them to contact you if they have a problem with it. Most people will be happy to get traffic from your mash-up. Anyway the burden is on them to ask you not to.
回答2:
It is not considered "polite," but it is done often nonetheless. Some websites take countermeasures against such activity, but in general you'll be able to do it without any repercussions.
If you want to do it the right way, you'll simply inquire with the website operator. For all you know they'll be okay with it, or they may even have an API set you can use.
But if you do the scraping without permission, if your site gets popular and the original site(s) discover what you're doing, they could send you a cease and desist letter and/or take additional legal steps if they feel you're infringing on any copyrighted material.
http://en.wikipedia.org/wiki/Web_scraping
回答3:
It's generally Not The Done Thing. If the authors of those websites want to make their data available they will probably have done so via an API or feed of some sort.
The best thing to do is to ask them directly. They might even offer you a better method than scraping.
来源：https://stackoverflow.com/questions/9960104/are-web-developers-allowed-to-scrape-html-content