Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: Yahoo, MSN Don't Index or Refer

__/ [Carol W] on Tuesday 08 November 2005 01:39 \__

> On Thu, 03 Nov 2005 16:55:29 GMT, info_at_1-script_dot_com@xxxxxxx
> (www.1-script.com) wrote:
>>Roy Schestowitz wrote:
>>> I  have  noticed that MSN and Yahoo index less than 5% of the pages in
>>> my
>>> site  while  Google  is 'aware' of the existence of them all. I  tried
>>> to
>>> think  if  MSN  and  Yahoo  are hindered  by  JavaScript,  but  there
>>> are
>>> JavaScript-free routes to all pages.
>>> Why  is it that Yahoo and MSN will not crawl deep or at least index
>>> pages
>>> that  are  buried  deeper in the site? Judging by the logs,  they
>>> consume
>>> (spider) just about as much traffic as Google. When it comes to
>>> referrals,
>>> Google  delivers  almost 100 times the amount of referrals from Yahoo
>>> and
>>> MSN combined. How can this be?
>>> Help appreciated,
>>> Roy
>>Looking at last 7 days data, I got 95% of SE traffic from Google, 2.2%
>>from Yahoo and 0.6% from MSN.
>>Yet, the downloaded amount data goes like this:
>>Yahoo! Slurp: 603.6Mb
>>msnbot: 278.5MB
>>Googlebot (Freshbot):112.7MB
>>Googlebot (Deepbot): 155.9Mb
>>The only way I can see this happening is that Y and MSN do not deem my
>>pages important enough to include in the indices, but still important
>>enough to keep banging on them.
> I can't talk about MSN but, in my past observations, Yahoo is pretty
> slow in updating their index.

That seems plausible. I wonder how they survive if that's true... maybe an
improper search engine for anything recent.

> One time I posted here that Yahoo kept showing a cache copuy of one of
> my pages that was over a year old - even though the site design had
> been changed (drastically) twice within that year - and some of the
> contents were changes as well. In terms of new pages, I don't hold my
> breath with Yahoo. Seen it take as long as 6 to 9 months before my
> "new" pages are finally part of the index there. Wherease Google
> reflects changes or new pages within a few short hours.
> No idea why Yahoo is so slow on refreshing their index, or even the
> cahce,  as slurpbot is rather active in visiting sites so takes note
> of new pages and changed contents but that apparently is how it is
> with Yahoo .. and I can't see how MSN could do any worse.
> Carol

I just keep wondering why both MSN and Yahoo stop at roughly the same
threshold of 2000 pages. If they were both out-of-date, would the number of
pages be so closely related? I am sure nothing is negotiated among the two
and I would like to believe that what you say is true. If Yahoo and MSN will
index more pages (*and* display them) in due time, then I truly look forward
to it.


Roy S. Schestowitz
http://Schestowitz.com  |    SuSE Linux     |     PGP-Key: 0x74572E8E
  5:25am  up 5 days  1:23,  5 users,  load average: 0.13, 0.22, 0.31
      http://iuron.com - next generation of search paradigms

[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index