Thread
:
robots.txt
View Single Post
12-15-2004, 01:54 PM
European Lee
Confirmed User
Join Date: Dec 2002
Location: Daytona Beach
Posts: 7,133
Why woud you want to stop them looking for one?
Just dont upload one, if there isnt one on the server, it will return a 404 error and youll at least know your site is being spidered.
Just a thought.
Regards,
Lee
__________________
YOUR TRAFFIC IS SAFE WITH US - CONDOM CASH
PUTTING THE GAY ADULT INDUSTRY ON YOUR PC!!
AFFORDABLE
2257
COMPLIANT
GAY + BI
CONTENT
European Lee
View Public Profile
Visit European Lee's homepage!
Find More Posts by European Lee