Home Messages Index
[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index

Re: Yahoo, MSN Don't Index or Refer

__/ [www.1-script.com] on Thursday 03 November 2005 16:55 \__

> Roy Schestowitz wrote:
>> I  have  noticed that MSN and Yahoo index less than 5% of the pages in
>> my
>> site  while  Google  is 'aware' of the existence of them all. I  tried
>> to
>> think  if  MSN  and  Yahoo  are hindered  by  JavaScript,  but  there
>> are
>> JavaScript-free routes to all pages.
>> Why  is it that Yahoo and MSN will not crawl deep or at least index
>> pages
>> that  are  buried  deeper in the site? Judging by the logs,  they
>> consume
>> (spider) just about as much traffic as Google. When it comes to
>> referrals,
>> Google  delivers  almost 100 times the amount of referrals from Yahoo
>> and
>> MSN combined. How can this be?
>> Help appreciated,
>> Roy
> Looking at last 7 days data, I got 95% of SE traffic from Google, 2.2%
> from Yahoo and 0.6% from MSN.

Same here, roughly.

> Yet, the downloaded amount data goes like this:
> Yahoo! Slurp: 603.6Mb
> msnbot: 278.5MB
> Googlebot (Freshbot):112.7MB
> Googlebot (Deepbot): 155.9Mb

Similar figures to mine, but exchange Yahoo with Google. Google Images
account for much of the traffic from Google.

> The only way I can see this happening is that Y and MSN do not deem my
> pages important enough to include in the indices, but still important
> enough to keep banging on them. I would already disallow both Y! and MSN
> if I would not be #1 on one of the most important keywords in both. That
> said, however, I have to admit that being #1 on Y! or MSN only warms one's
> heart and nothing else :-) as far as traffic is concerned.

I  have done some further work yesterday, trying to figure how this  could
be attributed to.

Smaller  sites of mine are fully indexed by all (almost identical  numbers
across  search  engines). I looked up some other sites which contain  many
pages. I could observe that it is not only me who is affected. Maybe it is
the  innate  behaviour  of the algorithms? Imposing a limit on  number  of
pages? If so, it's an interesting situation.

Google   focuses  on  breadth  and  will  therefore  be  able  to  provide
answers/solutions  to many answers/problems whereas Yahoo and MSN are nar-
row, focused, and more specific. They concentrate on fewer 'quality' pages
at the expense of sparseness of 'knowledge'.

Just try 'site:http://example.org' which works in MSN and Google. The dif-
ference  is  rather  glaring to to the eye. In MSN and  Yahoo,  performing
searches  that target deep pages will produce nothing, which implies  that
the  indices  are indeed depleted. It is not the case of  misleading  page
count  (saturation). Since Yahoo and MSN's combined share of the 'pie'  is
approximately equal to that of Google, I was wishin' and hopin'...


[Date Prev][Date Next][Thread Prev][Thread Next]
Author IndexDate IndexThread Index