|
Something like the following in a .htaccess works well...
SetEnvIfNoCase User-Agent "^Wget" bad_bot
SetEnvIfNoCase User-Agent "^ia_archiver" bad_bot
SetEnvIfNoCase User-Agent "^WebReaper" bad_bot
SetEnvIfNoCase User-Agent "^BloodHound" bad_bot
SetEnvIfNoCase User-Agent "^WebVac" bad_bot
SetEnvIfNoCase User-Agent "^w3mir" bad_bot
SetEnvIfNoCase User-Agent "^Xget" bad_bot
SetEnvIfNoCase User-Agent "^Download Express" bad_bot
SetEnvIfNoCase User-Agent "^NetAnt" bad_bot
SetEnvIfNoCase User-Agent "^FlashGet" bad_bot
SetEnvIfNoCase User-Agent "^GetRight" bad_bot
Deny from env=bad_bot
Just add any bots you wish...
|