|
Good tips in this thread - thanks.
Another aspect to limiting abuse is bandwidth throttling - Apache has mods to limit number of simultaneous sockets and the rate of download (regardless of the user's connection speed).
RE the IP blocking approach -- I'm not sure if anyone pointed it out yet, but blocking an IP may block legitimate visitors on using the same provider (aol, etc.). If you do lock out a specific IP, do it for a 'cooling period' and then reinstate it so legitimate users can still get it. Fortunately with members, as soon as a violation of TOS is determined, you can kill their access.
RE only dumb site scrapers will reveal they are NOT a browser -- dealing with this type of download via UserAgents is pointless. Generally, the usage pattern will reveal an abusive agent. I lock out anyone who continuously hits my site at a rate faster than two pages per sec.
-Dino
|