PDA

View Full Version : about robots.txt


nlopes
17-07-2003, 06:13/06:13AM
I have a robots.txt disallowing the file j.php.

But I have some links j.php?m=***

SEs are indexing that files. Should I have to disallow each ?m=**??

If yes, maybe it's better to use a meta tag in that file?



[Moved from Google forum to here. Please, try to find the most appropriate place for new threads. You can read this (http://www.ihelpyouservices.com/forums/showthread.php?s=&threadid=7575) for further clarification. - Jill]

Bernard
17-07-2003, 09:52/09:52AM
I would suggest moving that page (and it's subpages) into a unique directory and disallowing the directory.

nlopes
17-07-2003, 14:42/02:42PM
But it's just one file!

Why should I create a directory to have just a file??
Robots.txt should accept wildcards.........

my files are:
j.php?m=abcd
j.php?m=efg


Why adding just j.php to robots.txt doesn't stopp these to be indexed??

g1smd
17-07-2003, 14:53/02:53PM
Putting <meta name="robots" content="noindex,follow"> on the page will sort it out.

nlopes
17-07-2003, 14:59/02:59PM
Does all search engines support that meta tag?

If not all, the most important?

g1smd
17-07-2003, 15:59/03:59PM
I think that just about all Search Engines to support that tag. Yes.