About the author

Vijay Kodali
E-mail me Send mail

Site Statistics

Site Meter

Recent comments



The opinions expressed herein are my own personal opinions and do not represent my employer's view in anyway.

© Copyright 2017

Detecting crawlers,bots,spiders in Asp.Net

Here is a way of detecting search engine (Google, live…) crawlers,spiders etc..

            System.Web.HttpBrowserCapabilities clientBrowserCaps = Request.Browser;
            if (((System.Web.Configuration.HttpCapabilitiesBase)clientBrowserCaps).Crawler)
                Response.Write ( "Browser is a search engine.");
               Response.Write ("Browser is not a search engine.");

//Here is another approach..

            if (Request.ServerVariables["HTTP_USER_AGENT"].Contains("Googlebot"))
                //log Google bot visit
            else if (Request.ServerVariables["HTTP_USER_AGENT"].Contains("msnbot"))
                //log MSN bot visit
            else if (Request.ServerVariables["HTTP_USER_AGENT"].Contains("Yahoo"))
                //log yahoo bot visit
                //similarly you can check for other search engines


Posted by vijay on Tuesday, July 15, 2008 11:51 AM
Permalink | Comments (0) | Post RSSRSS comment feed

Add comment

(Will show your Gravatar icon)

  Country flag

  • Comment
  • Preview