GoFuckYourself.com - Adult Webmaster Forum - View Single Post

DrewKole · 09-17-2002, 11:10 PM

Basically, I've got a ton of mirrors of 1) TGP galleries and 2) AVS sites, same site mirrored to different avs's...

Does anyone know a definite way of having google not spider or index a page?

Id like to have all but 1 of my mirrors not-spidered, since google apparently doesnt like mirrors on the same domain, it considers them spam, and who the hell knows where the googlebot might find one of my mirrors.

Any hints on how to get away with the mirroring of pages, without using multiple domains for the same fuckin site? =)

09-17-2002, 11:10 PM
DrewKole Confirmed User Join Date: Aug 2001 Posts: 5,193	Having google NOT spider you? How? Basically, I've got a ton of mirrors of 1) TGP galleries and 2) AVS sites, same site mirrored to different avs's... Does anyone know a definite way of having google not spider or index a page? Id like to have all but 1 of my mirrors not-spidered, since google apparently doesnt like mirrors on the same domain, it considers them spam, and who the hell knows where the googlebot might find one of my mirrors. Any hints on how to get away with the mirroring of pages, without using multiple domains for the same fuckin site? =)