Thread: Robots.txt getting 404 hits?
View Single Post
Old 08-17-2005, 10:13 PM  
JD
Too lazy to set a custom title
 
Industry Role:
Join Date: Sep 2003
Posts: 22,651
all spiders look for robots.txt Just put a blank robots.txt in your root if you're tired of seeing the 404's
JD is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote