___/ On Friday 26 August 2005 06:49, [Alan Cole] wrote : \___
> Is there any way to stop the Googlebot visiting a site quite so
> often.... I know a robots.txt file can restrict their access to certain
> pages but I don't want to do that as I still want all pages listed, I'd
> just rather the Googlebot didn't visit so often as it eats up all of my
> bandwidth.
>
> It seems to visit my site every few days and manages to crawl through
> something like 30,000 pages each time it does so.
>
> Thanks.
> Al.
http://www.google.com/bot.html#fast
Hope it helps,
Roy
--
Roy S. Schestowitz chmod a-r *.mbox
http://Schestowitz.com
|
|