robot tag

[phiar]

What usually follows the Robot tag?
--Never mind, after some research I found that the Robot tag is used for parts of your site that you DON'T want indexed.


--Phia:confused:

[Hope]

A robots meta tag is used to tell robots/spiders/crawlers what to do with specific pages. It basically tells bots that you want the page indexed or not and if you want the bot to follow the links on the page. I don't know how many spiders actually look at this tag anymore. I would recommend a robots.txt file for exclusion.

[g1smd]

There are two ways of controlling robots:

A robots tag or a robots.txt file in the root directory of the site.
The latter gives a lot more control.