Free Trials + Spidering.
 

When offering a FREE trial how does/would the webmaster go about protecting is content from software such as spiders that could download everything?

What about people just coping text to word? I know this can be prevented via no right click but u can still get round it, is there another way??? more safe.

as anyone wrote any scripts to safe guard against people spidering the entire site to their hard drive? etc?

much advice is needed on this subject and I am sure that everyone will benefit !

so.....?

an example of the spidering program I am refering to would be:
www.webattack.com/get/superbot.shtml

and apparently this one is the best:
http://www.spidersoft.com/

I'm not adept with scripts myself, but I THINK that you can use htaccess to deny these spiders access??

Or via your robots.txt file?? Maybe?

would that work?

anyone got a defintive solution for this problem?

Darren don't put a no right click in your members area, it'll piss people off.
You can use cgi to serve your images, that's one way.

Also you can put a user agent allow/deny in your htaccess to prevent all of the known web sucking programs.

You can also set a bandwidth threshhold per username/pass. Figure about 4 or 5 times what anyone could reasonably download in a day and then cut them off. This kicks out the password traders as well :thumbsup
(Granted you MIGHT piss off a member who's just in there right clicking his ass off, but if he's trying to get that much that quick he's probably not planning on sticking around anyways.

Of course, you should probably use a host that handles some of the bigger paysites and their tech crew will most probably set up a solution for you.

good ideas - are there any actual hosts which do offer solutions if so speak up! or spacedog spam me one!

any other thoughts

Ask your current host to help you with setting up your htaccess..

I know there's articles & examples of how to use htaccess to block the user agents, I seen them, just dont remember the link..
I am sure http://www.google.com/search?hl=en&i...with+.htaccess

you could find something useful here..

actually, the first listing on this page may be exactly what you need...


http://www.google.com/search?hl=en&i...with+.htaccess

here is nice .htaccess file,,,,
http://chokinchicken.com/webmasters/htaccess.htm


i added it to my root server and found out some of my pics wouldn't load,, but added it to my domain directory and it worked fine

maybe something in the httpd.conf configuration would be better?

Limit user to 2 simultaneous connections

Which would mean he can load index.htm & pic1.jpg at one time...but no more.

Most spidering software makes use of 100's if not 1000's of threads to spider the site.

You can effectively eliminate that with bandwidth throttling, or limiting the user connections/threads.

I have used such technology while using win2k server, not sure how to set that up on unix, linux, or apache, but im sure there is a way to do it.

You can also set a bandwidth cap...IE total transfer limit per-ip address, or per username/password.

The per-ip can be bypassed by a person using a proxy-spider...however if you set a total download limit for a user/pass that would prob be your best bet :)

hope that helpz

OK, here's how to block spiders like Teleport Pro, which is the #1 spider software.

Make your navigation pages a form submit and use the images as submit buttons, instead of the usual href. The spiders cannot get past them! The action will simply be the page you want to go to. And you can do mouseovers so it looks just like a normal link.

This works great for member's areas.

quit makin me look bad lensman ; )

Lensman

That sounds great. Can you give us a site url that is doing that? I would love to see it.

Darius

I don't know if it did when this was first posted, but when I looked, it was pointing to superscripts.com, psybercore.com or whatever.. (They and drewstar.com and no telling how many other sites are all the same guys sites).

If you plan on laying out any cash for these scripts this guys sells, you might want to do a search on "psybercore" "superscripts" etc. first. It would have same some people some money if they'd have done that before they sent their cash to this company. See:

http://www.bitchdomain.com/superscriptmeoff.htm

You might try amateurpages...:Graucho