I am an idiot. I accidentally downloaded GFY's robots.txt into my OWN site directory.
 

I usually have at least one SSH session open to one of my servers, and if I want to view the text of a web file I'll usually just type "fetch http://url.com/" into the unix prompt.

I just discovered that when I downloaded GFY's robots.txt to check it my working directory was the web root of one of my major sites. :uhoh


# robots.txt file for http://www.gfyboard.com/

User-agent: *
Disallow: /


So I effectively blocked all robots from indexing my site... plus overwrote the old robots.txt file. Damn!!

I'm very lucky that I just checked that robots.txt file because I was too lazy to do a web search on how to configure it for another site. Who knows when I would have noticed otherwise. :)

:Graucho you still have the whole weekend to find someway else to screw up

does google even use robots.txt any more?

Actually the reason I was checking GFY's robots.txt was because I noticed recently that it was indexed in Google. :)

Looks like GFY's robots.txt disappeared for a little while, which is how the board got into Google again.