![]() |
leaving out robots.txt ?
What are the negative consequences of having a missing robots.txt file?
I understand that it could waste a tiny bit of bandwidth, but could this hurt in the SE rankings? |
no......
|
Quote:
|
Not every site needs a robots.txt file - not every site would benefit by having one. Most sites however do benefit by having a robots.txt file
The largest benefit is the ability to tell google, yahoo, and ask which sections/pages (like affiliate code redirect pages) to not list in google. Doing this keeps rank values focused and not on needless pages or pages that could get you into trouble (like redirects, popup codes, iframes) I block ai archiver too. You can use it to block mass download bots for TGP's, or include the url to your sitemap. |
You probably want to get around to using robots.txt. Apparently there's a new command that lets you tell the SE bots where your sitemap is.
|
Quote:
|
You should have one even if it's blank
|
| All times are GMT -7. The time now is 05:39 AM. |
Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2026, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc123