|
|
|
|
||||
|
Welcome to the GoFuckYourself.com - Adult Webmaster Forum forums. You are currently viewing our boards as a guest which gives you limited access to view most discussions and access our other features. By joining our free community you will have access to post topics, communicate privately with other members (PM), respond to polls, upload content and access many other special features. Registration is fast, simple and absolutely free so please, join our community today! If you have any problems with the registration process or your account login, please contact us. |
 |
|
|||||||
| Discuss what's fucking going on, and which programs are best and worst. One-time "program" announcements from "established" webmasters are allowed. |
|
|
Thread Tools |
03-25-2004, 12:05 PM
|
#1 |
|
Confirmed User
Join Date: Mar 2004
Posts: 683
|
Website downloading tools
I just caught a couple bastards ripping off a ton of my galleries .. chewing up a shit-load of bandwidth .. and I'm a little upset about this. But I guess it's to be expected.
I won't stand for this and sit back and pay the bandwidth bills.. if you're trying to rip off all the free porn on my server ... fuck off ... I won't let you do it. So I'm doing something about it .. and I want to share it with GFY incase it's useful to anyone here. The tool that these guys were using is called Website eXtractor. Lucikly in this program's case .. it sets the User-Agent: to "Website eXtractor (http://www.asona.org)" Ima block this program on my server ;) To do this you need mod_access and mod_setenvif. Crack open your apache configuration file and add the following line: BrowserMatch "^Website eXtractor" downloader Now wherever you configure your virtual hosts add the line "deny from env=downloader" so it looks something like this: Code:
<Directory "/foo/myhost.com">
Order allow, deny
allow from all
deny from env=downloader
</Directory>
__________________
|
|
|
|
03-25-2004, 12:07 PM
|
#2 | |
|
jellyfish
 Join Date: Dec 2003
Posts: 71,528
|
Quote:
Another one of the big website grabbers is Wget, it eats a hell of a lot more bandwidth, but not as much as Fusker will if you dont have hotlink protection |
|
|
|
|
|
03-25-2004, 12:08 PM
|
#3 |
|
Registered User
Join Date: Nov 2003
Location: SoCal
Posts: 82
|
Is that like BlackWidow or something?
M.
__________________
If your not sending your interracial traffic to DARKCAVERN.com YOU ARE LOOSING MONEY! |
|
|
|
|
03-25-2004, 12:29 PM
|
#4 |
|
Confirmed User
Join Date: Mar 2004
Posts: 683
|
Going through the list of User-Agents I posted above..
I searched each page for "downloading tool" .. then I grep'd my access_log (which is rotated daily .. so it's only today's logs) for those User-Agents... I was fucking dumbfounded how many hits I was getting from downloading tools. I am proudly sharing my blacklist with the world: BrowserMatch "^Website eXtractor" downloader BrowserMatch "^DDD" downloader BrowserMatch "^GetBot" downloader BrowserMatch "^Irvine" downloader BrowserMatch "^JBH" downloader BrowserMatch "^JOC" downloader BrowserMatch "^dCSbot" downloader BrowserMatch "^Download.exe" downloader BrowserMatch "WebCapture" downloader BrowserMatch "^Pockey" downloader BrowserMatch "^puf/0.*" downloader BrowserMatch "^Teleport Pro/*" downloader BrowserMatch "^URLGetFile" downloader BrowserMatch "^Web Image Collector" downloader BrowserMatch "^WebAuto/*" downloader BrowserMatch "^WebFetch" downloader BrowserMatch "^WebPix" downloader BrowserMatch "^Wget/*" downloader Not all of those I was getting hits from .. but lots of them. Wget was by far the biggest offender. I don't even wanna know how much bandwidth these tools have actually used up this month. Anyway I hope I at least provided a useful little tip for some people here on how to prevent these tools from hitting your site. Feel free to use my blacklist 
__________________
|
|
|
|
