Googlebot receives 404 error

[Michele88]

:hi:

My error logs are showing that Googlebot receives a 404 error when trying to access the robots.txt file of my site, which I do not have. Is this OK (as I have no need for this file), or should I insert code to prevent this error from occuring?

Thanks!

[g1smd]

The robots.txt file is for banning robots (all, or specific named ones) from visiting specific named parts of your site.


If you don't need to ban any robots then you don't have to have a robots.txt file on your site. I have sites at #1 in the SERPs and they don't have this file. It is not a problem.


If it makes you feel better you can create a blank text file, call it robots.txt and upload it. That will eliminate the 404 error, but will not modify the robots' behaviour.

[Michele88]

:hi: g1smd,

Thanks so much for your detailed reply. I guess there is no need to add the file then...I just get a little concerned when I see anything in the error logs! I would hate to give ol' Googlebot a hard time... ;)

Thanks again for your help.