Robot Text File

[kneelsit]

How do you create a robots.txt file?

Read more here:
http://www.websitemanagementtools.com

O.K. guys and gals start pulling it apart and happy chewing

[WebSavvy]

I just bought the pro edition of their software. I see MMT has publicly endorsed it, so it must be good. :thumb:

[Alan Perkins]

That whole article is just an advertisement!

[Advisor]

I'm sure it's probably a useful tool if you have the need for that sort of thing, but I do have to agree with Alan that the article is basically just an ad for the software. Perhaps it would have been better to simply link to the article at Robin's site instead of posting it here? I'm not sure...I personally like to post links to articles as opposed to whole articles, but I guess that's a call for Dougie to decide!

Jill

[kneelsit]

Point taken Jill. I should have chopped out more of the blurb. I will go away now and repeat

"Do not post ads" 1000 yimes.:o:

[WebSavvy]

Hi Jill,

I see that this software is something Barry also uses. I thought it would be a useful tool to track spiders with, whereas the log could be put into text format as an example for my submission clients to see what spiders have visited based on what SE's are in the database.

This is much easier than handing out access to my weblogs, which I refuse to do.

The software makers claim it's a necessary tool to have when doing optimization, any thoughts on this Jill, Alan?

[Advisor]

I couldn't say, Deb. It's not something that I have ever needed. I rarely even use the robots.txt file. But that doesn't mean that some day I might not need something like it. I really didn't look at the software at all, just briefly glanced at the article that Greg posted. I wasn't making any judgements on the software itself, just that the article Robin wrote seemed to be more of an ad than an article. Which is fine, I was just agreeing with Alan as to its label!

Jill

[WebSavvy]

The software creates a robot.txt file and it can also connect to your server access logs and pull out spider visits and let you see what dates they were there and what pages they hit.

But, you know, I get all of that in my weblogs anyway. The pro version (which I bought) is $179.00

But the upside is, it'll make it much simpler to create a text file that my clients can read a copy of without giving access to my actual weblogs themselves.

There's an endorsement from Barry on there too. I was just wondering how it is that this software could make the job of optimizing pages any easier, as it alludes to in it's information prologue.

[Advisor]

I guess we should wait 'til Barry gets back. Maybe he can enlighten us!

J

[ihelpyou]

What article are you all referring to? You can pm me.