Meta Tags
 

Does anyone know the meta tags to insert so people can't mass-download from your members area with a bot?

Or a good idea to block the way a bot massive grabs your content...

Robots.txt file in your root directory of your site can keep SE's from indexing you members area, I'm not sure about downloading.

Meta Tags are read and complied with voluntarily. If you're trying to stop a bot with rogue intentions, this will not do it.

meta tags seem useless nowadays

how can a spider index your members section (SE spider that is) if it is pword protected??

I use both the metas as well as the robots.txt file AND robot.txt file (without the "s") to cover some older backward shit... just to make sure I got 'em all covered.

i was thinking of serving pages as .asp extensions

the bots don't reconize this and don't know how to ripp the content via links when they hit an asp

what do you think?