Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: google cacheless

__/ [Borek] on Sunday 20 November 2005 10:28 \__

> On Sun, 20 Nov 2005 05:09:35 +0100, Roy Schestowitz
> <newsgroups@xxxxxxxxxxxxxxx> wrote:
>> I am not too sure. In fact, I wonder if it will ever drop pages from its
>> index as a result of /modified/ robots.txt. I really hope so because it
>> indexed many pages I did not intend for it to ever have access to.
> http://services.google.com:8882/urlconsole/controller
> Best,
> Borek

Interesting! Thanks for that.

Just in case this entailed a penalty, I have also submitted a reinclusion
request this morning:



As part of my collaboratory research, I share output of my experiments
among my colleagues. I put these under my domain. Only recently it
slipped my mind that crawlers might reach it sooner or late because
there was a link (it took about 10 months to reach the experiments). As
a result, tens of thousands of pages which contain numerical results got
indexed. As soon as I noticed the jump from ~25k to ~80k pages in the
datacentres (over the period of just days or weeks), I was shocked and
immediately added the following to my robots.txt.

Disallow: /Research/Resources/Experiments/

That is where all of these newly-added pages lie. I hope you can remove
the penalty (Google referral dropped by over 80%).

Many thanks in advance,





Roy S. Schestowitz
http://Schestowitz.com  |    SuSE Linux     |     PGP-Key: 0x74572E8E
  2:35pm  up 17 days 10:29,  4 users,  load average: 0.40, 0.62, 0.37
      http://iuron.com - next generation of search paradigms

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index