PDA

View Full Version : Inktomi PFI service. Is it crawling more than just submitted pages?


kingarthur
22-08-2002, 20:45/08:45PM
I recently removed about 300 url's from PositionTech. I removed all of these pages because they were penalized, for reasons that no one at Ink or PT cant explain to me. Now that I think of it, they couldn't even tell me IF the pages were penalized, but they were pushed to the back of the line for some reason.
I was told by positiontech to remove all of those pages from the PFI program and make new pages. I did all of that, and replaced the old list of 300 url's with a new list, but Inktomi still crawls ALL of the old pages. Do they really expect me to remove those pages from my server? I might be able to fix them, if they could tell me what was wrong, but if I dont know I cant fix the problem and just removing those pages is not a good idea because I have good placement on other search engines with some of those pages. If I am reading their service agreement correctly, Ink is only supposed to crawl the pages that I submit to them, this is definitely not the case.
After spending almost 10k I assumed that someone would be able to talk to me, but apparently no one wants my business. At this point I'm thinking I may just start over and submit to Inktomi without the PFI.
Anyone have any thoughts on why this would happen?

ihelpyou
22-08-2002, 21:53/09:53PM
It sounds like they have a problem with the pages and some form of spam. You are right, they will never tell you what the problem could be and expect you to find it and fix it yourself.

If you could post one of the pages, we could help you better.

Mel
23-08-2002, 03:05/03:05AM
Hi KingArthur:

Assuming that the site in question is the one in your profile, It appears that you may be recovering from a Google PR0 penalty.

You can set up your robots.txt fle to prevent the Ink spider from crawling the pages that you don't want them to see, this should solve one of your problems.

As to fixing what is wrong with those pages, since you are recovering from The Google PR0, you must have rectified the problems that Google penalized you for, and I don't see anything right off the cuff that seems blatently wrong with your site, but would hazard a guess that if your Google penalty was due to something like using hidden or invisible text you might be facing a similar problem with INK/MSN.

I often find these sort of problems are due to misuse of CSS style sheets to make hidden text or items positioned off the page, but cannot tell if that is what is happening here since you have blocked outside access to your css file so that it cannot be viewed.