Roy Schestowitz <newsgroups@xxxxxxxxxxxxxxx> wrote:
>* Is Yahoo more reluctant to crawl /new/ sites? Its crawling capacity is
>lower, so it sounds likely. Google is usually among the first (if not /the/
>first) to 'come to the party'.
That's interesting, I've observed that slurp and msnbot have belched
and gone out the door while googlebot is munching the index.
I have the feeling that the algorithms used by slurp/msnbot are fairly
similar and straightforward (eg, stuff everything in a bag and chew on
it later), while the algorithm used by googlebot is more
"contemplative" (grab the menu and split, come back later if there's
anything you decide you want). That's just inferrence from observed
behaviour, a little better than guesswork but certainly not definite.
--
http://www.ren-prod-inc.com/hug_soft/store.php?action=contact
|
|
