not being picked up from right page

[sly]

Hi All,
I have a bit of a dilemma and was wondering if any1 can help.

i have a site that I have built quite a large sitemap for as I have many pages that are quite similar.

i have basically got a page setup with a few links on it.

Im not sure why but inktomi is only pulling up the sitemap page and not the page that has a link on the sitemap

so for instance i have

www.gh.com/sitemap.html ( this is being indexed )

www.gh.com/sitemap/more.html (the "more" page is not being indexed

can any1 offer me any reasons why inktomi wouldnt index it

cheers

sly

[Farhan]

Sly, I think you should firstly check the "Robots.txt" file for the "more" page. Is it allowing the spiders to crawl?

I would also consider waiting for a while if the page (more) is relatively new in the site. May be Inktomi will pick the page in sometime.

Lastly, why don’t you consider paying Inktomi to crawl that page. Paying will ensure that your page is crawled within 2 days (I guess so).

[sly]

yeah I have paid, I am currently using position tech to pay for the url

i was wondering wether this will just pick up the site map page or the pages within coz at the mo its just picking up sitemap page

[Farhan]

I am not sure on this but when you get your page indexed through PFI, Ink does not spider any further from that page.

[ihelpyou]

Yes, you pay for 'one' page to be spidered and indexed on a regular basis. Whichever page you pay for is the page that gets spidered.

[Farhan]

Apart from the regular spidering, I don't see any a reason to pay to Ink! Once I get my sites listed in the major directories (especially Dmoz) Ink indexes them pretty fast.

[sly]

Hi Faran is that for each page

I mean do they index the actual content on the page eg meta tags title etc.

Because mine dont seem to have been indexed

cheers

sly

[Webmaster T]

Originally posted by sly
[B]Hi Faran is that for each page

I mean do they index the actual content on the page eg meta tags title etc.

Because mine dont seem to have been indexed Putting a sitemap in an Inclusion program IMO, will only be effective if there are good keyword rich descriptions. Since a sitemap is used to steer SE spiders through a site it isn't effective because links on pages in Inclusion programs aren't followed therefore they aren't indexed unless the site or pages are linked to from other sites and the regular spider finds them. Keep in mind the Inclusion spider is a totally different spider then the regular crawler (Slurp for Inktomi). AFAIK they share no information at all.