do you have to have robotos.txt to get google to spider??/
google question
Collapse
X
-
Tags: None
-
No.
http://www.google.com/addurl/?continue=/addurl
Your best bet for a speedy index inclusion is asking some people who run related sites for links. -
well google is hitting my site already but my stats show robots.txt keeps getting requestedOriginally posted by hiveNo.
http://www.google.com/addurl/?continue=/addurl
Your best bet for a speedy index inclusion is asking some people who run related sites for links.Comment
-
I think the spiders default behavior is to request that file, so if it's there the spider will know what to index or not.Originally posted by JupZChriswell google is hitting my site already but my stats show robots.txt keeps getting requestedYour post count means nothing.Comment

Comment