GoFuckYourself.com - Adult Webmaster Forum - View Single Post

WiredGuy · 03-26-2004, 12:50 PM

I noticed the robots.txt file is gone now for GFY. Before, they used to disable any SE from spidering GFY. Removing that will get a lot of these threads indexed now

WG

03-26-2004, 12:50 PM
WiredGuy Pounding Googlebot Industry Role: Join Date: Aug 2002 Location: Canada Posts: 34,496	I noticed the robots.txt file is gone now for GFY. Before, they used to disable any SE from spidering GFY. Removing that will get a lot of these threads indexed now WG __________________ I play with Google.