I wish someone coul tell me whats happening with Inktomi at the moment

[sly]

I am having problems getting our pages indexed. For some reason we are getting indexed very well by google but not inktomi, this is strange because up until now this has always been the other way around.

can someone tell me if this is just inktomi taking its time or is there something else wrong.

its picking up our home page but not any of the pages linked from it.

any help would be much appreciated

cheers

sly

[ihelpyou]

Hi Sly, If you are talking about the site in your profile, I don't know how Google would be picking up anything but the home page either. It's in frames and you don't have anything in between the <noframes> tag, so spiders don't have anything to index OR follow.

Hope this helps.

[Farhan]

Did you pay Inktomi to index your homepage?

[sly]

Hi i am not talking about the site in my profile, does it matter what site i am talking about. I just have some issues with getting indexed, I have tried many many times to get to the bottom of these issues and would really appreciate it if anyone could offer me some advice as to why indexing is proving difficult, because it is completely baffling me, i have been getting sites indexed in inktomi for quite some time now and sometimes there are just no visible reasons why sites arent indexed

any help would be great

Sly

[sly]

pay as in $39 per year?0, beacuse I have position tech and I am confused as to the relationship between position tech and inktomi and everything really as i have never had these problems before.

The site basically has a homepage and then links to about 10 pages from that. the hompage is picked up just fine but the next pages wont get indexed, have taken out strings, have done everything i can to get them indexed google has no problem just inktomi wont pick it up

pleese help me

Sly

[ihelpyou]

No way to know why as we can't see the site in order to give you advice. :)

Does it have incoming links?
Is it brand new?
Is it in the major directories?
Do you have good navigation?
Is there anything on the pages that 'might' go against se guidelines?

[ihelpyou]

Oh, if you paid PT the 39 bucks then Inktomi is only going to spider and index the page you paid for.

For other pages, my questions above pertain as Inktomi uses another robot for the free pages.

[sly]

the answer is 'yes' to all four questions and the fifth 'no'

have changed everything to comply with search engines, the only thing @i can think of is that we have a very small amount of duplicate content on each of these 10 pages but the tags and some other content is different

[sly]

even so if they do index the home page why would they not follow the links to the next pages? i thought these would be indexed as well or is there a completely different inktomi spider for these pages

is this what happens?

you pay for inktomi $39 to index the home pages, it does this and then waits for a while and comes back with another spider to index the content

I am even more baffled

[ihelpyou]

My same response applies:

Oh, if you paid PT the 39 bucks then Inktomi is only going to spider and index the page you paid for.

For other pages, my questions above pertain as Inktomi uses another robot for the free pages.

"if you paid PT the 39 bucks then Inktomi is only going to spider and index the page you paid for."

[sly]

so i have to pay $39 for each page , I thought inktomi would index all of my site as long as the navigation is ok, does this mean you have to pay for every page to be indexed individually?

[ihelpyou]

No. This is why it's impossible to know what might be wrong with your site since the free robot is Not coming around, and we don't see the site.

Any page you need to have indexed and don't wish to figure out what is wrong, you need to pay for. We are talking about two entirely different things.

[srikanthsh]

If you submitted your website recently, you will have to wait for some time (2- 6 months) to get other pages into the index. Because, i think, they intentionally stop the other pages from being listed for some time (May be, It is to let you use paid submission service for other important urls).

Anyway, this is completely my assumption based on some of my experiences with inktomi spider.

one issue i faced with other client's website:

The spider used to list all the internal pages but not index page. That website has been online for the last 4 years. I started working on it two years back. That home page has very good SE friendly (ofsourse, also user friendly) content. But, i found home page in listings for the last 6 months only.

[sly]

this is the thing, i do wish to figure out what is going wrong so i dont need to pay I would rahter not pay if u know what I mean,

if there is anyway I can get this page indexed it would be great

you say

the free robot is not coming round and we dont see the site

What does this mean?

Does it mean that the free robot is a separate entity to the paid one

i think i am getting more confused by the minute

Does this also mean I have to pay for pages I want indexed and how much? and surely there is a way of getting the site indexed without paying for individual pages as long as the navigation is ok.

EG if i build a site purely in html and this is a five page site then all of it will be indexed. i am pretty sure of this as i have had other sites indexed before without paying,

there must be something I can do

any thougths?

[sly]

srikanthsh

thanks for your useful comments

what sort of costs areinvolved with getting more pages indexed is it like a pay per click thing ?

[MakeMeTop]

Does it mean that the free robot is a separate entity to the paid one

Yes, that is exactly what it means.

You pay for the PFI spider and it spiders the page you paid for. This has nothing to do with crawling your site.

The natural spider follows links to your web site from other sites on the web. If you have plenty of good links, you will get your site crawled - and if your site is navigable by spiders all your pages will get crawled. But it could take a long time - or it could be quite quick - depending on the number of good links you have.

The fastest crawl I have had from a site with some pages in paid inclusion is 4 days after the site was accepted by Yahoo and linked to by 3 other quality sites already in Inktomi. The longest was 9 months for the entire site to be crawled.

they intentionally stop the other pages from being listed for some time

Not true - see above.

[sly]

Thanks makemetop

I am glad I have eventually got to the bottom of that one

Sorry doug if i misunderstood what you were saying to start with

thanks everyone 4 your help

SLy

[sly]

I just have one more question though,

Makemetop: When you say links do you mean that I would have to get links from other good sites to those specific parts (pages) of the site that arent being indexed or just the homepage?

and

We currently have quite a lot of banners and advertising things that we are paying for which provide links and are indexed by inktomi, would these count as links and would i have to find a site to get links from that is being crawled well by the inktomi free spider and not in paid inclusion to have a chance of being indexed?

cheers in advance for your help

Sly

[MakeMeTop]

links from other good sites to those specific parts (pages) of the site that arent being indexed or just the homepage?

Either. You basically want links to pages that link to all other areas of your site.

would these count as links

Banners and other advertising click throughs do not count as links. Usually, they do not link to you directly but via the advertising vendor. You will get no benefit from them.

would i have to find a site to get links from that is being crawled well by the inktomi free spider and not in paid inclusion to have a chance of being indexed?

Sites in paid inclusion may also have been crawled well by the free spider, so you can't tell. Try and get links from the major directories - you know these are pure links and they are favoured by all crawling search engines.

[srikanthsh]

---------------------------------------------------------
srikanthsh : Because, i think, they intentionally stop the other pages from being listed for some time.

MakeMeTop: Not true - see above.
-----------------------------------------------------------

Thanks for giving good info MMT. Yes. May be, you were right (as we deal with new websites and it takes good time to build proper incoming links).

That's why i said - Anyway, this is completely my assumption based on some of my experiences.

And sorry for posting wrong assumption.