how would you do that?

referal check?

Alright.. if someone knows of a module for pro_ftpd that allows you to run an executable after certain filetypes are uploaded (or for every upload), I will post a solution that strips warez from files as they are uploaded.

I'm still looking for a better solution, but this will solve 90% of the problem for most hosts.

there's no proftpd module, but there's another way to do it. You simply use "Custom Log" into some file which will record all filenames uploaded, and then run a routine script on it - thats atleast the way i do it :)

adultwire, could you just post the solution then we can figure out the best way to impliment it? monitor log files seems fine to me. maybe someone could offer a modified wuftpd or something... whatever.

yeah, adultwire, what do you got there?

interesting thread. detecting steganographic messages in images. does this help http://www.outguess.org/detection.php ?

cheers

DrGuile: using mod_rewrite (if your webserver is apache)

I would imagine that these warez tools can spoof the http_user_agent and http_referer fields; mod_rewrite is probably not a solution (but I've never seen one of these programs).

cheers

somebody has noticed here that they know what the http_user_agent values are for that software, but they didnt say exactly what

Hey, if anyone could contact me regarding a solution for this I'd appreciate it.

Email or ICQ works.. matt @ nysus.com or 129060301

Cheers,
Matt

Hehe dork, of course they are hotlink protected.. the utility is its own web browser that downloads all the images.

Sorry.. I had to split for a while.. I have some work to finish, and then I'll post my make-shift solution.

We'll be waiting! :)

yeah, I am glad people are willing to share ideas to stop cheaters. These warez guys are pretty hooked up. They upload many gigs in 1 night

Shitty.. my solution ended up causing problems with many images.. I think it's time to start coding. I just want to strip the warez in a non-lossy mannar (without changing a single byte of the valid image data) -- this requires parsing the file and rewriting the terminating block at the correct place.

Then I want to compare the filesizes, and if they differ substancially, throw away the file. If they don't, use the stripped one (to save b-width)..

Doesn't necessarily have to be ZIP/RAR, and even if it was, the archive can be split into several parts, and reassembled, the header doesn't matter. You can split the header into 15 million files, but when you reassemble that multitude of files, you still have a valid header, see?

could someone who has some of these files post links to a few more gifs and jpegs? I saw the one gif posted earlier. That gif seemed to be identifiable as not being a valid image.

cheers

http://www.greenapple.verisexy.net/wezgina/
http://www.greenapple.verisexy.net/salamba/


Are these the type of files you guys are talking about? Looks, so weird by the way they are names and no sponsors or anything =/ I will trying to find more and post them when I do so people can see examples.

If you need a bit of help, let me know...

That looks exactly like a recent user on my freehost. Exactly. Different file names, and the pics were asian, but the "Welcome to xxxxxx's Web Site", followed by the centered line of small images. Deleted his account after reading this thread.

why would they use host for warez in the first place? isnt it easier to just get it off kazaa?

You can't move ZIPs and RARs around on KaZaA (i think), two, that would require someone volunteering to dedicate their bandwidth to little .jp punks downloading warez all day, haven't you ever heard of freeloading at someone else's cost?

User agent is spoofed from a list of user agents, and the referer is spoofed depending on the client, some do the actual HTML file, others do "index.html", others just use the server host, so blocking by those two variables are not really solutions.

AdultWire - proftpd can use mod_autorun to run a script after upload...

Warez in files - Yeah... I've been noticing these bad boys for a while... my gif/jpg detection scripts aren't able to nail em either, grumble grumble...

Ahhh jesus.. freehosting just isn't as fun as it was back in 99 ;)

-- kevin
-- adultserver.com

Shit.. I'm in meetings all day today.. so again, not much time to work on this.. But if anyone knows how to code, I think GIF's should be relatively easy to do without losing any data. You need to read the header for the size of the image, and then unpack lzw blocks until you've filled up the bitmap area with data. Then, discard any remaining blocks and write a header.

At least, this should work for 87a's, but probably not 89a's.

With jpeg's, if you don't mind a slightly lossy method, nconvert will do the trick.

The whole shebang can be glued together with mod_autorun, and that should be a working solution.

I hope I get some time this evening and maybe we can fill in the rest of the blanks.

if mod_autorun requests a script every time a file is uploaded, it may become quiet resource consuming.... there should be some other solution like custom log, and scheduled script execution

for as much as people who don't use kazaa seem to think it's a freeloaders dream and is an eazy way out for surfers, it's really not that great. i would say 70% of transfers you try and initiate don't actually work, most of them sit on 'remotely queued' and 'connecting' and whatever forever. most of the files you do actually manage to transfer are slooooow. if you get a 20KB/s transfer you're praising jesus. this goes for any type of file, movies, mp3s, pdfs (e-books), etc. it really sucks. same way with the gnutella clients. nothing has come even close to touching napster since it croaked. i'm sure we'll see more and more stuff liek this as peoples options dwindle: http://www.vnunet.com/News/1132038 (kazaa going OOB)

mod_autorun is hardly resource consuming... I get what you're saying, but trust me when I tell you the load difference from this to a scheduled event will be negligable, and this solves the problem before it can become one.

Heh, hey kev you may want to change your signature line. :1orglaugh

Spanky, yes that helps a lot; thank you.

All: As Spanky referenced, there is a utility called STEGDETECT that may help us out. I just downloaded it and did a preliminary test, with positive results.

STEGDETECT tests if information has been embedded in files using jsteg/outguess/jphide/invisible secrets(wtf?)/F5/camouflage/or appendX. Here are my initial findings:

STEGDETECT run on a normal JPG file that has not been embedded:

[root@SERVER_3 munimuni]# stegdetect 11544r033.jpg
11544r033.jpg : negative

STEGDETECT run on a JPG file that has been embedded

[root@SERVER_3 munimuni]# stegdetect Kurumi01_3204.jpg
Kurumi01_3204.jpg : jphide(***) appended(560)<[random][data][7|FG7.w.P=...x.J]>

thanks for the jpegs, I'd like to see more of the gifs is anybody has some. do they do this with PNG files as well? what about movie files or encoded text/html files?

cheers

helo, just a couple of thoughts about all of this.

It would seem like detection would be preferable to processing every file.

The gif file that I got had an invalid header but it's probably safe to say that most do have valid headers.

The gif licensing is pretty restrictive so processing every file may not even be an option. At a company I used to work for we decided not to compress gifs after we spoke to unisys about the licensing costs. Decompressing gifs is ok, compressing with unisys algorithms is not without a license.

Couldn't the files be identified by establishing a 'normal' ratio of bytes/pixel for each file type and testing the abnormal bytes/pixel ratios of these warez files against this normal? So far it looks like the jpeg headers contain the height and width of the valid image data with the rest of the crap tagged on to the end.

Let's that the r=b/(h*w) where r is our ratio, b is the number of bytes and h & w is the height and width reported in the header.

I grabbed 5 of the files posted earlier:
AGF2nd811.jpg 146618/(100 x 118) = 12.4252542372881
AGF2nd812.jpg 163365/(200 x 160) = 5.10515625
AGF2nd813.jpg 161336/(126 x 176) = 7.27525252525253
AGF2nd814.jpg 104842/(116 x 150) = 6.02540229885058
AGF2nd815.jpg 121379/(93 x 160) = 8.15719086021505

Compared with 5 (non warez) files reasonably compressed:
minnkim-035.jpg 34981/(600 x 457) = 0.127574762946754
minnkim-036.jpg 35846/(600 x 457) = 0.13072939460248
minnkim-037.jpg 30728/(600 x 457) = 0.112064186725018
minnkim-138.jpg 46571/(457 x 600) = 0.169843180160467

The ratios of bytes/pixel are severely out of whack with these warez files.

It would seem as though some c or asm code could read the headers of all new files on the system (either through hooking into ftp events or cron jobs on log files, doesn't really matter), compute the ratio of bytes per pixel, compare that against the normal and flag any abnormal ratios for delete or human examination.

any thoughts or comments on this?

cheers

The problem arises with animated gif's (which could be any number of frames, and could also be severly out of wack based on the imagesize:filesize ratio)

All one really needs to do is get a piece of code that unpacks a gif -- unpack the gif, and see if there is more than a few bytes of trailing data. If there is, the file is warez.

Most code that does this is written in C, and my C coding has become very rusty over the years.. haven't really done any hardcore coding of that nature since the 68000Assembly demo days on the Amiga.

unfortunately nobody has posted links to gifs other than the first one which was obviously not a gif so I haven't been able to look at any of these files as gifs. the bytes/pixel ratio should hold true for animated gifs by reading the framecount and dividing the total bytes by the number of frames reported r= (b/n)/(w*h) ... once again, I just have not seen any of the gif files though.

"All one really needs to do is get a piece of code that unpacks a gif -- unpack the gif, and see if there is more than a few bytes of trailing data. If there is, the file is warez. "

I agree, this should work. Personally I am more interested in low impact detection so if anyone has some of these gifs could you let me know?

cheers

Someone wanted a example of gifs. Here they are


http://www.greenapple.verisexy.net/t...bicross14.html

thanks, I grabbed a few and I'll take a peek. at first glance these gifs don't seem nearly as sophisticated as the jpegs, I wonder if it's the same program used to generate them both?

pretty clever really, they even include a checksum file. not as clever as it could be, but pretty clever.

cheers

if you ban .jp it will cut your warez problems down 70% :o)

here is the code I wrote, put this in a script and run it at upload time it checks for correct start of GIF file and correct header and footer of a JPEG

it uses sysread to be more efficient instead of usual perl file shit

# example - $pathfile = "/path/to/uploaded/file.gif";

if ($pathfile =~ /.gif$/gi)
{
open (TEST,"<$pathfile");
sysread (TEST,$check_header,4);
close (TEST);

$header = "GIF8";

unlink ("$pathfile") if ($check_header ne "$header");
}
elsif ($pathfile =~ /.jpg$/gi)
{
open (TEST,"<$pathfile");
sysread (TEST,$check_header,4);
sysseek (TEST,-2,2);
sysread (TEST,$check_footer,2);
close (TEST);

$header = "ÿØÿà";
$footer = "ÿÙ";

unlink ("$pathfile") if ($check_header ne "$header");
unlink ("$pathfile") if ($check_footer ne "$footer");
}


don't diss da jungle

Ok, heres some warez file names that I found on the server. Please share them =) Wait till traffic builds up and then delete the files and send the warez downloaders to popup hell.

*AGF1*
*AGF2*
*AGF3*
*AGF4*
*0425_do*
*kyohaku*
*elfan*
*wind_disc*
*eroken*
*dojin*
*ouma_disk*
*nsmdvd*
*kuraemon*
*mikoava*
*kango*
*zeroone*
*ayu*
*BB1WA*
*sp10*
*arisapv*
*ign0*
*ign1*
*ign2*
*ign3*
*ign4*
*minid*
*kiwoku*
*cubicross*
*afgan*
*megami0*
*megami1*
*megami2*
*ove0*
*ove1*
*ove2*
*ove3*
*ove4*
*IBM0*
*eva0*
*ds1*
*ds2*
*kojin*