__/ [Roy Schestowitz] on Sunday 20 November 2005 14:42 \__
> __/ [Borek] on Sunday 20 November 2005 10:28 \__
>> On Sun, 20 Nov 2005 05:09:35 +0100, Roy Schestowitz
>> <newsgroups@xxxxxxxxxxxxxxx> wrote:
>>> I am not too sure. In fact, I wonder if it will ever drop pages from its
>>> index as a result of /modified/ robots.txt. I really hope so because it
>>> indexed many pages I did not intend for it to ever have access to.
> Interesting! Thanks for that.
> Just in case this entailed a penalty, I have also submitted a reinclusion
> request this morning:
> As part of my collaboratory research, I share output of my experiments
> among my colleagues. I put these under my domain. Only recently it
> slipped my mind that crawlers might reach it sooner or late because
> there was a link (it took about 10 months to reach the experiments). As
> a result, tens of thousands of pages which contain numerical results got
> indexed. As soon as I noticed the jump from ~25k to ~80k pages in the
> datacentres (over the period of just days or weeks), I was shocked and
> immediately added the following to my robots.txt.
> Disallow: /Research/Resources/Experiments/
> That is where all of these newly-added pages lie. I hope you can remove
> the penalty (Google referral dropped by over 80%).
> Many thanks in advance,
...Oh, more importantly, apart from the very obvious typos (I didn't
proofread), I stupidly forget to mention that the experiments are in
computer vision. I neglected to remember that people out there experiment
with SEO. Too late to retract the request though or re-clarify...
Roy S. Schestowitz | Have you hugged your penguin today?
http://Schestowitz.com | SuSE Linux | PGP-Key: 0x74572E8E
2:45pm up 17 days 10:39, 4 users, load average: 0.58, 0.68, 0.52
http://iuron.com - next generation of search paradigms