How to identify web-crawler?

Question

How can I filter out hits from webcrawlers etc. Hits which not is human..

I use maxmind.com to request the city from the IP.. It is not quite cheap if I have to pay

Answer 1

The user agent ($_SERVER['HTTP_USER_AGENT']) often identifies whether the connecting agent is a browser or a robot. Review logs/analytics for the user agents of crawlers that visit your site. Filter accordingly.

Take note that the user agent is a header supplied by the client application. As such it can be pretty much anything and shouldn't be trusted 100%. Plan accordingly.