indexing faster than google

[Advisor]

I have to give MSN/Ink credit. They've already got my new personal blog site Jill Whalen Exposed (http://www.jillwhalen.com) in their database, and Google doesn't!

No money was involved either.

Jill

[Webmaster T]

Jill, has the Microsoft protype been by its called "Microsoft Prototype Crawler"? I'm trying to ascertain how it is finding sites for the new index. This would show a correlation between the Inktomi index and Prototype crawls. Thanks for any help.

[Advisor]

Terry, I have no idea! Would it look like anything special in my logs?

Not sure how good my stats program is...will check.

Jill

[Webmaster T]

How did you identify Inktomi by IP of the referrer or the name of the UA (User Agent)? Don't know the IP for MPC but the UA is "Microsoft Prototype Crawler". If your log tracks browsers that is where the "Microsoft Prototype Crawler" should be found if it has indeed crawled the site. I'm trying to learn about this new crawler as Inktomi looks like it will be replaced by this at some point. My feeling is they are presently building an index. It would be useful to know how it is doing this. Whether it is using Inktomis index as a start point or whether it is just randomly following links from directories etc.

[Advisor]

Google has it now!

I didn't identify Inktomi. I was testing out Mamma.com to see what sort of results they were showing for things, and out of curiousity I put in my name. Up popped my new site, which they had taken from MSN. I then went to MSN and found that it was there also. Number one spot for my name under the looksmart/zeal articles.

I do see the inktomi spider in my logs, but not anything that seems related to microsoft.

Jill

[ihelpyou]

Yes. That is an Inktomi listing that is in MSN under "web pages". js31.inktomisearch.com is what the bot looks like in the logs. Or any number like that. Inktomi has been crawling like nutty over the last two months, picking up everything under the sun.