GoFuckYourself.com - Adult Webmaster Forum - View Single Post - Ever Have a Look at the Whitehouse.gov Robots.txt?

minusonebit · 12-18-2007, 08:51 AM

Its quite interesting. Now why wouldn't they want the search engines indexing some of that stuff? Makes you wonder...

http://www.whitehouse.gov/robots.txt

12-18-2007, 08:51 AM
minusonebit So Fucking Banned Join Date: Feb 2006 Location: [email protected] Posts: 7,391	Ever Have a Look at the Whitehouse.gov Robots.txt? Its quite interesting. Now why wouldn't they want the search engines indexing some of that stuff? Makes you wonder... http://www.whitehouse.gov/robots.txt