Protection from screen scraping [closed]

Answer 1

Prevent? -- impossible, but you can make it harder.

Disincentivise? -- possible, but you won't like the answer: provide bulk data exports for interested parties.

On the long run, all your competitors will have the same data if you publish it, so you need other means of diversifying your website (e.g. update it more frequently, make it faster or easier to use). Nowdays even Google is using scraped information like user reviews, what do you think you can do about it? Sue them and get booted from their index?

Answer 2

It's pretty hard to prevent screen scraping but if you really, really wanted to you could change your HTML frequently or change the HTML tag names frequently. Most screen scrapers work by using string comparisons with tag names, or regular expressions searching for particular strings etc. If you are changing the underlying HTML it will make them need to change their software.

Answer 3

You could put everything in flash, but in most cases that would annoy many legitimate users, myself included. It can work for some information such as stock prices or graphs.

Answer 4

Given that most sites want a good search engine ranking, and search engines are scraper bots, there's not much you can do that won't harm your SEO.

You could make an entirely ajax loaded site or flash based site, which would make it harder for bots, or hide everything behind a login, which would make it harder still, but either of these approaches is going to hurt your search rankings and possibly annoy your users, and if someone really wants it, they'll find a way.

The only guaranteed way of having content that can't be scraped is to not publish it on the web. The nature of the web is such that when you put it out there, it's out there.

Answer 5

If its not much information you want to protect you can convert it to a picture on the fly. Then they must use OCR wich makes it easier to scrape another site instead of yours..

Answer 6

I tried to "screen scrape" some PDF files once, only to find that they'd actually put the characters in the PDF in semi-random order. I guess the PDF format allows you to specify a location for each block of text, and they'd used very small blocks (smaller than a word). I suspect that the PDFs in question weren't trying to prevent screen scraping so much as they were doing something weird with their render engine.

I wonder if you could do something like that.