How to identify web-crawler?

后端 未结 6 2019
死守一世寂寞
死守一世寂寞 2020-11-30 21:45

How can I filter out hits from webcrawlers etc. Hits which not is human..

I use maxmind.com to request the city from the IP.. It is not quite cheap if I have to pay

6条回答
  •  旧巷少年郎
    2020-11-30 22:46

    I found this package, it's actively being developed and I'm quite liking it so far:

    https://github.com/JayBizzle/Crawler-Detect

    It's simple as this:

    use Jaybizzle\CrawlerDetect\CrawlerDetect;
    
    $CrawlerDetect = new CrawlerDetect;
    
    // Check the user agent of the current 'visitor'
    if($CrawlerDetect->isCrawler()) {
        // true if crawler user agent detected
    }
    
    // Pass a user agent as a string
    if($CrawlerDetect->isCrawler('Mozilla/5.0 (compatible; Sosospider/2.0; +http://help.soso.com/webspider.htm)')) {
        // true if crawler user agent detected
    }
    
    // Output the name of the bot that matched (if any)
    echo $CrawlerDetect->getMatches();
    

提交回复
热议问题