What to put in it ?
-
-
if you really want to block some parts of your site, do a google.
It's not neccessary if you don't need to block anything from crawling. -
put sexy messages in it to turn the robots on..
hatisblack at yahoo.comComment
-
-
Nothing like a sexy robot!!!Originally posted by SmokeyTheBearput sexy messages in it to turn the robots on..
XXX TLD's - Another mosquito to swat.Comment
-
-
robots give good headComment
-
User-agent: *
Disallow: /Comment
-
This isn't quite complete, you forgot the google PriorityID:Originally posted by Furious_Female
User-agent: *
Disallow: /
Google_PriorityID: d41d8cd98f00b204e9800998ecf8427e
This google Priority ID is used by 100s of webmasters already, so it's not as effective as using a fresh ID. It's still better than using no ID at all, but I would look into obtaining your own ID.Custom Software Development, email: woj#at#wojfun#.#com to discuss details or skype: wojl2000 or gchat: wojfun or telegram: wojl2000
Affiliate program tools: Hosted Galleries Manager Banner Manager Video Manager
Wordpress Affiliate Plugin Pic/Movie of the Day Fansign Generator Zip ManagerComment
-
I just list all the robots who are allowed to visit if they should wish to.
ASIMO
C3PO
HAL
etc...
Comment
-
robots.txt is a waste of bandwith
Comment
-
Originally posted by killshot
Trust the "WOW" effect.Comment
-
good oneOriginally posted by SmokeyTheBearput sexy messages in it to turn the robots on..
Comment
Comment