keeping spiders away

[neewwman]

How do I keep search engines from spidering a page if I don't want it listed?

[SandraB]

Robot.txt is the magic word!
Here is "the" reading you are looking for.
A Standard for Robot Exclusion - http://www.robotstxt.org/wc/norobots.html

[marttali]

As sandra told you. robots.txt ! However inktomi 4 example doesnot tolerate/obey it and probabaly few others too but they are not important. Keep the shit out of your server, lots of spiders are not worth of your server resources. Just my 2cents

[graphicdude]

robots.txt is the right suggestion, however if it is a file or directory that only authorized users should access the best way is to protect it with a .htaccess file
http://www.freewebmasterhelp.com/tutorials/htaccess/

Otherwise if it is something that promotes your site who cares if the SE indexs it

[just a punk]

Originally posted by neewwman
How do I keep search engines from spidering a page if I don't want it listed?

Very easy! I'm not joking! Just use the X-HTML Encoder utility which is available at http://www.maturedesign.com/freestuff.html. It is freeware. The X-HTML Encoder makes your page absolutely undecipherable to ANY spider.

BTW, you may also like to check out the X-Link Encoder which is located at the same URL.

Hope this helps.