Thread: robots.txt, how important?
View Single Post
Old 10-06-2002, 07:41 AM  
SABAI
Confirmed User
 
Join Date: Jan 2001
Location: footmaniac.com
Posts: 2,880
robots.txt, how important?
i noticed on some of my servers that the most common document not found was robots.txt...

how important is it? whats the use / benefits of it and what should it contain?


thanks in advance

sabai
__________________
SABAI is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote