robots.txt question - GoFuckYourself.com

Cyrano · 01-19-2007, 11:56 PM

Is it possible to use robots.txt to prevent search engines from indexing certain areas of text on specific pages? If not, is there another method of accomplishing this?

baddog · 01-20-2007, 12:01 AM

Quote:

Originally Posted by Cyrano

Is it possible to use robots.txt to prevent search engines from indexing certain areas of text on specific pages? If not, is there another method of accomplishing this?

You can use it to keep spiders from certain pages, but I am not sure how I would hide just certain text . . . . unless you put it in a .gif or something

Elixir · 01-20-2007, 02:39 AM

User-agent: *
Disallow: /images/

you can use this.

bigalownz · 01-20-2007, 02:49 AM

but robots.txt dont work with some bots and spiders

darksoul · 01-20-2007, 03:23 AM

use iframes for that text and deny access to the page

01-19-2007, 11:56 PM	#1
Cyrano Confirmed User Join Date: Apr 2006 Location: USA Posts: 900	robots.txt question Is it possible to use robots.txt to prevent search engines from indexing certain areas of text on specific pages? If not, is there another method of accomplishing this?

01-20-2007, 02:39 AM	#3
Elixir CHECK MY SIG-AMAZING SITE Join Date: May 2006 Location: ICQ: 210-874-419 Posts: 12,870	User-agent: * Disallow: /images/ you can use this. __________________ SIG FOR SALE ICQ 210874419

01-20-2007, 02:49 AM	#4
bigalownz Confirmed User Industry Role: Join Date: Aug 2005 Location: NEW ZEALAND Posts: 1,657	but robots.txt dont work with some bots and spiders __________________ $100 free credit for all hosting needs

01-20-2007, 03:23 AM	#5
darksoul Confirmed User Join Date: Apr 2002 Location: /root/ Posts: 4,997	use iframes for that text and deny access to the page __________________ 1337 5y54\|)m1n: 157717888 BM-2cUBw4B2fgiYAfjkE7JvWaJMiUXD96n9tN Cambooth