How to protect/monitor your site from crawling by malicious user

Question

Situation:

• Site with content protected by username/password (not all controlled since they can be trial/test users)
• a normal search engine can\'t get at i

Answer 1

Apache has some bandwidth-by-IP limiting modules AFAIK, and for my own largeish Java/JSP application with a lot of digital content I rolled my own servlet filter to do the same (and limit simultaneous connections from one IP block, etc).

I agree with comments above that it's better to be subtle so that a malicious user cannot tell if/when they've tripped your alarms and thusy don't know to take evasive action. In my case my server just seems to become slow and flaky and unreliable (so no change there then)...

Rgds

Damon