Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: Googlebot becoming obnoxious

__/ [Ignoramus25589] on Friday 14 October 2005 05:31 \__

> It crawls my websites algebra.com and cooldictionary.com way too
> fast.
> 
> Today, for instance, algebra.com received 67,000 accesses from
> googlebot.
> 
> They also misbehave and do not honor Crawl-delay setting.
> 
> I am not going to ban googlebot from crawling my site, but I am
> becoming somewhat apprehensive of them.
> 
> I want to limit their accesses to, say, no more than every 5
> seconds. How can I do it, realistically speaking.
> 
> i

{
* Nice to have you back in this NG

* Mikkel, I am aware of the subject line which will be bound to my name in
SEPR's... ouch!
}

Someone in this newsgroup asked a similar question a month or two back. It
is not quite possible to limit the demands that reach from Google if I
recall correctly. That was the conclusion reached, I think. My recollection
tells me that the best move would be to contact Google saying that your
dedicated server is knocked on too hard.

(since I bound my name to the subject line, I might as well go all the way)

001 650 330 0100

Call them to get an E-mail address and then be in touch with a real person
promptly.

Hope it helps,

Roy

-- 
Roy S. Schestowitz      | Have you hugged your penguin today?
http://Schestowitz.com  |    SuSE Linux    |     PGP-Key: 74572E8E
  6:10am  up 49 days 18:24,  4 users,  load average: 0.82, 0.84, 0.73
      http://iuron.com - next generation of search paradigms

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index