Thread: Robots.txt getting 404 hits?
View Single Post
Old 08-17-2005, 10:18 PM  
SmokeyTheBear
►SouthOfHeaven
 
SmokeyTheBear's Avatar
 
Join Date: Jun 2004
Location: PlanetEarth MyBoardRank: GerbilMaster My-Penis-Size: extralarge MyWeapon: Computer
Posts: 28,609
Quote:
Originally Posted by Sosa
is their a plus to adding anything to the robots.txt file?
you can tell certain known robots that you might not want using your site to fuck off.
__________________
hatisblack at yahoo.com
SmokeyTheBear is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote