Thread: Internet Shinchakubin -Spider crawling my site?
View Single Post
Old 01-12-2004, 03:20 AM  
Rorschach
So Fucking Banned
 
Join Date: Aug 2002
Posts: 5,579
First result on Google:

http://www.robotstxt.org/wc/active/html/myweb.html
Rorschach is offline   Share thread on Digg Share thread on Twitter Share thread on Reddit Share thread on Facebook Reply With Quote