Alta Vista ignoring Robots exclusion tag

[kneelsit]

Hi,

I have a robots exclusion tag in the Head section of my order page since I could see no point in the robots indexing it but seems that the AV robot ignored it altogether and brought up that page in its search results.

Seems that there is not much point putting them up anymore, I think I read somewhere that the tag is only a guide for the bots and they do what they like.

I also had that page included in my robots text file. Looks like there is now no point in wasting our time on this aspect of SEO.

i:(

[Matt B]

I have noticed that with most bots. Even Google has disregarded this at times. But usually it works out after the first couple of indexings. If it is a problem you can contact them directly and have them remove the specific pages, but only with good reason (i.e., corporate secrets.)

[Hope]

Greg,

Over the last 2 years AltaVista has ignored robots meta tags. They only pay attention to robots.txt about 1/3 of the time. I wouldn't worry about it. Most of the other spiders do pay attention to the robots.txt.

[kneelsit]

Thanks for those words of re-assurance Heidi. :)

Actually I was more concerned about googlebot.

AV has always treated me well and placed me near the top (often #1) for my major keyword phrases. Actually it has given me an idea for incorporating this word ( heavy duty spring) in my features page and giving it some prominence.

Must have been a heavyweight surfing the web at the time :D

[Alan Perkins]

Actually I was more concerned about googlebot. Googlebot obeys both robots.txt and the robots meta tag.

Sometimes it looks like it's ignoring robots.txt but that's because of the way Google builds its index.