I\'m looking into building a content site with possibly thousands of different entries, accessible by index and by search.
What are the measures I can take to preven
You could try using Flash / Silverlight / Java to display all your page contents. That would probably stop most crawlers in their tracks.