GoFuckYourself.com - Adult Webmaster Forum - View Single Post

alex79 · 10-28-2004, 08:42 AM

it is any difference for robots/spiders between these 2 siteuations:

1. the robots.txt file don't exist on my site
2. robots.txt file exist on my site but is an empty file

if yes, what is the difference for spiders?

10-28-2004, 08:42 AM
alex79 Confirmed User Join Date: Jun 2002 Location: france Posts: 996	robots.txt quick question it is any difference for robots/spiders between these 2 siteuations: 1. the robots.txt file don't exist on my site 2. robots.txt file exist on my site but is an empty file if yes, what is the difference for spiders?