|
robots.txt file
i noticed i don't have a robots.txt file for my sites, is that important? and if so is this all i need in it to have all bots crawl my site?
User-agent: * Disallow: |
|
| All times are GMT -7. The time now is 10:37 AM. |
Powered by vBulletin® Version 3.8.8
Copyright ©2000 - 2025, vBulletin Solutions, Inc.
©2000-, AI Media Network Inc123