GFY EDUCATIONAL SERIES: How to prevent Piracy - A new way. - Page 4 - GoFuckYourself.com

borked · 10-08-2010, 08:40 AM

Quote:

Originally Posted by Nautilus

Too slow unfortunately. To make it practical, it should do at least a billion comparisons/hour.

well, to be fair, this is running on a dev server that is about 5 years old. Shit, it takes me about 15 minutes to compile ffmpeg whereas on a new quad core, it's compiled in ~15 secs.

Plus, there could be many instances of the app running - no need for a single fork to be doing all the work...

And of course, you would only search for comparisons after searching for pages that flag your keywords... you wouldn't have to search ALL photos on ALL sites

--edit

ok, I see what you mean - pairwise comparison of a single photo with all your photos.

Can you send me a zip with some photos in (more the merrier, but try to keep the zip <5MB) and a test image to compare against? The script will run *much* faster comparing 1 photo to many than doing 1on1 comparisons...

borked · 10-08-2010, 08:47 AM

Quote:

Originally Posted by ottopottomouse

Nearly everything I have tried doing has been with an aim of keeping the image as close to the original as possible and get it to pass as NO PIRATE.

Just think that to make your comparison test as good as possible I need to to be trying to think of a way I could get a whole photoset to pass while still having it acceptable to the human eye.

Similar to what I said above, you could lower the level to whatever is satisfactory to *not* let false negatives through, which would allow more false positives to show up. However, I suppose this stuff is with the view to scan loads of images per hour and to flag ones that require further intervention by a human eye.

Automated scripts will always cause false negatives and positives, but false negatives are much more costly. Much better to let a human brain search through the results flagged as pirated and toss away the false positives, than it is to allow false negatives slip through.

But yes, you're right - if a photoset contains 15 images - it only takes 1 to be flagged as pirated to raise the alarm bell on the photoset.

Nautilus · 10-08-2010, 08:49 AM

Quote:

Originally Posted by borked

well, to be fair, this is running on a dev server that is about 5 years old. Shit, it takes me about 15 minutes to compile ffmpeg whereas on a new quad core, it's compiled in ~15 secs.

Plus, there could be many instances of the app running - no need for a single fork to be doing all the work...

And of course, you would only search for comparisons after searching for pages that flag your keywords... you wouldn't have to search ALL photos on ALL sites

Is is possible to estimate it's maximum productivity on a good modern server, and take into the account both server and script optimization (multi threaded mode or some other tricks)?

I understand that you need to optimize where you search and what finds you're going to compare, but still - we have a database of about one million pictures to protect, camparing against this db by 1Kpics/hour is kinda not going to work. Even 100K/hour not going to work.

Relentless · 10-08-2010, 08:51 AM

A good post and an excellent read. Thanks.

The only problem with it is the part where you wrote "The only way to "rip" your stream is to have a screen capture program record full playback of your move. Impossible to prevent that!" That's a very weak Achilles heel to the whole 'solution.' It means essentially, YES your videos can still be pirated. And once *anyone* does it, they have a pirated version of the video which they can use to propagate tubes, torrents and other p2p media exactly the same way they would have if they had downloaded it. The basic problem is that you only need 1 competent thief and all the incompetent ones get access to whatever he stole exactly the same way as if they stole it.

brandonstills · 10-08-2010, 08:55 AM

Quote:

Originally Posted by borked

b - if we didn't permit downloads, but made sure the movies you like were always available, in full, for 1 year even after you cancelled your membership at some point in the future, would you consider cancelling your membership (15% said they would consider cancelling)

This allows you the opportunity to upsell and try to re-convert them as welll.

borked · 10-08-2010, 09:29 AM

Quote:

Originally Posted by brandonstills

This allows you the opportunity to upsell and try to re-convert them as welll.

this is exactly what has been implemented - great return rates on those coming back and trying to view a movie they are not entitled to view...

Blocking users from logging in simply because their pass expired is dumb... give them the members area but on trying to view something that they are not entitled to view... upsell, like you said

borked · 10-08-2010, 09:32 AM

Quote:

Originally Posted by Nautilus

Is is possible to estimate it's maximum productivity on a good modern server, and take into the account both server and script optimization (multi threaded mode or some other tricks)?

I understand that you need to optimize where you search and what finds you're going to compare, but still - we have a database of about one million pictures to protect, camparing against this db by 1Kpics/hour is kinda not going to work. Even 100K/hour not going to work.

Yes, I think I have a way around that... pre-calculate all your images before hand and dump the floating hash. That way, the app simply has to calculate from the single image and compare it for likeness to all the pre-calculated ones.

Let me see if that's possible - that should speed shit up *enormously* (well, except for inital calculations on you image stock). I will put that on "to do" for next week as I'm a bit loaded up atm due to time I wasted in doing this app

borked · 10-08-2010, 09:35 AM

Quote:

Originally Posted by Relentless

A good post and an excellent read. Thanks.

The only problem with it is the part where you wrote "The only way to "rip" your stream is to have a screen capture program record full playback of your move. Impossible to prevent that!" That's a very weak Achilles heel to the whole 'solution.' It means essentially, YES your videos can still be pirated. And once *anyone* does it, they have a pirated version of the video which they can use to propagate tubes, torrents and other p2p media exactly the same way they would have if they had downloaded it. The basic problem is that you only need 1 competent thief and all the incompetent ones get access to whatever he stole exactly the same way as if they stole it.

then watermark your streams with the user's username, IP etc - this was discussed in the thread. It's impossible to stop a screen grabber from ripping your streams, but you can deter it perhaps

DWB · 10-08-2010, 10:01 AM

Quote:

Originally Posted by borked

Timely bump for people to digitally fingerprint their own content easily...

I just heard the FSC, Mansef and Top Bucks just put a hit out on you.

Quote:

Originally Posted by borked

I really do find it extremely amazing that there has been only one person to date that has actually contacted me to discuss wanting to implement things discussed in this thread on their sites.

I only see a handful of people interacting in a serious thread that is highlighting how to combat it

You know I'm waiting patiently.

Just sent you mail BTW about your dog pic.

DWB · 10-08-2010, 10:04 AM

Quote:

Originally Posted by borked

this is exactly what has been implemented - great return rates on those coming back and trying to view a movie they are not entitled to view...

Blocking users from logging in simply because their pass expired is dumb... give them the members area but on trying to view something that they are not entitled to view... upsell, like you said

How do you implement this with 3rd party billing? At least for an upgrade back into the site after they cancel, I don't think that is possible unless you have their date and your own merchant account, correct?

Don't have to go into details here, can shoot me an email.

PXN · 10-08-2010, 10:48 AM

Clearly a way better and cheaper solution that FSC/APAP fingerprinting approach.

Great post borked!

borked · 10-08-2010, 11:26 AM

Quote:

Originally Posted by PXN

Clearly a way better and cheaper solution that FSC/APAP fingerprinting approach.

Great post borked!

well, to be honest, the solutions are not really comparable...

The OP was designed at *preventing* content ever being able to get uploaded. Then it evolved to involve injecting use data to track the pirate down, then it evolved to finding copyrighted content to help automate the process in finding copyrighted content already out there.

The FSC solution is to prevent content getting onto those tube sites that are a part of the circle, which I'm not too hot about since that leaves the others free to do as they please.

Let's just say they are complementary approaches to the same goal.

My goal is to evolve the thread to be able to detect movies, but that I fear may never happen outside of a "proof of concept" sandbox due to the bandwidth involved, but hey, I can surprise myself sometime

All this knowledge btw is already out there in forms of scientific papers who have shown ways to detect identical/similar images/videos. The hard part is implementing it in a real-world cost-effective situation.

borked · 10-08-2010, 11:28 AM

Quote:

Originally Posted by DirtyWhiteBoy

You know I'm waiting patiently.

shit, I knew someone would pick up on that post!
I just posted that in sheer frustration when I was tired and thinking to myself "why am I even bothering with all this frikken complicated shit?"

damn time-sensitive edit gfy button!

chronig · 10-08-2010, 12:45 PM

Quote:

Originally Posted by Relentless

A good post and an excellent read. Thanks.

The only problem with it is the part where you wrote "The only way to "rip" your stream is to have a screen capture program record full playback of your move. Impossible to prevent that!" That's a very weak Achilles heel to the whole 'solution.' It means essentially, YES your videos can still be pirated. And once *anyone* does it, they have a pirated version of the video which they can use to propagate tubes, torrents and other p2p media exactly the same way they would have if they had downloaded it. The basic problem is that you only need 1 competent thief and all the incompetent ones get access to whatever he stole exactly the same way as if they stole it.

Hopefully some vid quality will be lost in the doing --- and watermark the vidz and persue the original capturer later

borked · 10-09-2010, 03:46 PM

Quote:

Originally Posted by Nautilus

Is is possible to estimate it's maximum productivity on a good modern server, and take into the account both server and script optimization (multi threaded mode or some other tricks)?

I understand that you need to optimize where you search and what finds you're going to compare, but still - we have a database of about one million pictures to protect, camparing against this db by 1Kpics/hour is kinda not going to work. Even 100K/hour not going to work.

brandonstills gave me an idea in another thread...

and yes, I've sped things up by 1000x

this is having pre-hashed all your images (which is kinda slow at ~0.1/sec, but new images can be easily added to the db, just the initial compile is going to be slow) and then comparing 1 image against this hash db:

Quote:

time taken to compare 55 pre-hashed images: 0.120398044586 seconds (~1644544 images/hr)

And that is on my crappy PowerEdge 1850

About as good as it's going to get

--edit
if someone wants to verify that calculation of images/hr based on
intval( 3600 / ( (1 / 55) * $time )
cos my brain is really fried!

borked · 10-09-2010, 03:58 PM

to give an idea of size, hash db is 8.0KB for a directory of 16MB images (55 in total). Can't get better than that with how I'm hashing things...

Nautilus · 10-09-2010, 06:14 PM

Quote:

Originally Posted by borked

and yes, I've sped things up by 1000x

That's much better but still not good enough.

If on a much better server your script can go up to comparing 10mil pics/hour, that means you can run 10 comparisons/hour against a database of one million pics (pretty average db, many programs have that much and often more). That means ~250 comparisons/day - practically, that means you can check about 1 thread/day at pornbb or similar major boards. Or, in terms of posts, that's about 100 posts/ day (an average post has more than one preview attached).

To give you some idea of the size of the task, major boards like pornbb or saff boast 5-10K posts/day. Granted, not all of them contain any pictures to analyze (most of them are just "thanks"), but it is safe to assume that no less than 1K posts/day will contain some graphics to compare with the database.

That requires 10x more computing power than we have according to our best estimation. And you need to control at least a dozen of the major boards to make sure your stuff is not easy to find, hence 100x faster script is necessary. Which could do about 1 billion comparisons/hour.

I'm talking boards only because that's where most of the picture piracy takes place. Tubes and torrents mostly steal videos (although photo content is not uncommon for torrents too, boards host much more of it).

Nautilus · 10-09-2010, 06:27 PM

You can optimize your search of course, and only analyze threads/posts that contain some specific keywords. You can also narrow your database and compare with either your most recent or most pirated pictures (sets).

But that raises new issues/questions:

1. That requires much more sophisticated spidering software that filters what you need.

2. Keyword filters are bound to be innacurate, especially for mainstream/hardcore/babe sites.

3. People will still be able to post your stuff that's not in the db.

All in all, simply brute forcing all new posts and comparing then against your entire database, is the REAL solution; while everything else is bound to be less accurate and miss a lot of your stuff posted illegally.

Thus, please find us some way to make 1 billion comparisons/hour

While search optimization (for those who still want to go that route) will be better spent on adding more boards to the pool.

Ron Bennett · 10-09-2010, 08:38 PM

Quote:

Originally Posted by chronig

Hopefully some vid quality will be lost in the doing --- and watermark the vidz and persue the original capturer later

Not likely, and furthermore, in some instances, screen cap video quality will be better than what many paid members will get.

Reason is that many users have mediocre / saturated (ie. other computers sharing / torrents / malware / bots, etc) slow internet connections, which will adversely affect streaming, while many pirates have good connections and fast computers optimized for video capturing.

On a related note regarding video quality ... many surfers plain do not care as long as it's good enough. Heck, even the worst quality capture will likely still be far clearer and better than the average poor quality VHS pirated copies that people used to widely trade.

In regards to embedded user member info - what good does that really do? Presumably many members have little assets to go after. And more to the point, as was highlighted in another thread, much of the pirating is being done by others within the industry.

Anyways, member info in files is little more than a roadbump to the pros ... throwaway credit cards, anonymized IP addresses, VPNs, botnets, etc make it easy to get around all that - leaving a dead-end trail / pinning the blame on unsuspecting users.

Ron

borked · 10-10-2010, 12:07 AM

Quote:

Originally Posted by Nautilus

Thus, please find us some way to make 1 billion comparisons/hour

While search optimization (for those who still want to go that route) will be better spent on adding more boards to the pool.

first it was a mill/hr, then 10mill, now a billion/hr

Well, I can't test this accurately as I don't have a large enough dataset. I only have 55 pics...
I did ask for a dataset - if you can email me a link where I can download a zipped dataset, I can give it a try. Testing something that is taking 0.1 sec is not accurate enough to project up to large datasets.

Plus, the app can load in > 1 image at a time so multi-instance comparisons can be achieved. I just need a decent enough dataset to be able to test - I will trash the dataset when finished testing.

ottopottomouse · 10-10-2010, 03:09 AM

Quote:

Originally Posted by borked

I just need a decent enough dataset to be able to test - I will trash the dataset when finished testing.

Apparently the full Lightspeed archive is available on a torrent...

The larger numbers just come from putting a bit more thought into real application of the script. What size of image are you comparing against? as I would have thought if it can reliably recognise thumbs it would whizz through them compared to an 800x600 image or bigger and a lot of board images are a thumb preview.

borked · 10-10-2010, 03:18 AM

Once the hash db has been created, comparing thumbs against it would be faster, yes. And quality of matching wouldn't decrease, just because it's a thumb. Best though to create the original hash db against full size, to get the best quality plot of the pixels involved.

If someone finds a forum or something with their content on it, drop me an email giving me permission to download the content for testing and I'll use that as a base - the more images the better!

btw, script tested and runs 8x faster on a quad core as opposed to a dual core.

Nautilus · 10-10-2010, 04:06 AM

Quote:

Originally Posted by borked

first it was a mill/hr, then 10mill, now a billion/hr

Well, I can't test this accurately as I don't have a large enough dataset. I only have 55 pics...

Well actually it was a billion from the very beginning

Quote:

Originally Posted by Nautilus

Too slow unfortunately. To make it practical, it should do at least a billion comparisons/hour.

I'll upload some pics for you, check your e-mail.

Nathan · 10-10-2010, 05:06 AM

Quote:

Originally Posted by borked

For those of you actually reading this thread - it means you are actually interested in piracy prevention and proper detection....

The "real world" is light years ahead on all this as visual hashing is a massive research area (especially in videos given transcoding, youtube, copyright etc)

For one example (warning scientific paper, but well written and good visuals!) - see this PDF manuscript. There are hundreds more like it (but more mathematically detailed!)

The real world is moving forward a lot, which is why large companies can easily send take down notices to google because things are automated.

When it's so easy for some to send out mass emails to end-users of pirated content, why the need for all this piracy prevention?

I really do find it extremely amazing that there has been only one person to date that has actually contacted me to discuss wanting to implement things discussed in this thread on their sites. (4 days on the front page is like 8 years in real human terms)

If piracy is such a major problem in this industry, I only see a handful of people interacting in a serious thread that is highlighting how to combat it

borked,

tagging the content is an interesting concept in order to sue the people that distribute it, that's for sure.

Your ideas on first page saying only stream and protect the stream, however, are not good. The income lost because of unhappy members is far bigger than "fixing" piracy will ever return.

Nathan · 10-10-2010, 05:15 AM

borked,

sorry for my ignorance.. but how exactly does hashing images help you find stolen videos? Or am I missing the point of all of this?

What images are you hashing and comparing to? Places like PornBB will never have an image on a post which comes cleanly from the members area. 99% of the time it will be a video thumbnail compilation image.

ottopottomouse · 10-10-2010, 05:30 AM

The hashing images is for photosets.

borked · 10-10-2010, 09:41 AM

Quote:

Originally Posted by Nathan

borked,

tagging the content is an interesting concept in order to sue the people that distribute it, that's for sure.

Your ideas on first page saying only stream and protect the stream, however, are not good. The income lost because of unhappy members is far bigger than "fixing" piracy will ever return.

Yes, the thread evolved to allow for tagging videos with member details, so that if/when pirated content was found, the original member could be removed from the members area and if needed followed up with prosecution with "direct proof" of pirating user content.

However, there are quite a number of sites that I know of that only allow streams - don't discount an idea since it isn't what "the big sites" use...

Quote:

Originally Posted by Nathan

borked,

sorry for my ignorance.. but how exactly does hashing images help you find stolen videos? Or am I missing the point of all of this?

What images are you hashing and comparing to? Places like PornBB will never have an image on a post which comes cleanly from the members area. 99% of the time it will be a video thumbnail compilation image.

Otto is correct - another thread evolution into detecting pirated photosets, which is equally as damaging to content owners as videos. Still rife. The app is looking pretty good atm and the algorithm used in phase 2 to do the same to detect videos (more difficult as you know, but still very do-able)

It's detection by comparison, not by fingerprinting. Maybe not as water tight, but certainly useful enough in an automated setting to flag certain content for verification by a human eye. Makes finding a needle in a haystack easier when you have a giant magnet to help you look

borked · 10-10-2010, 09:45 AM

Quote:

Originally Posted by Nautilus

Well actually it was a billion from the very beginning

ah! in all honesty, there's no way this app is going to be able to do 278,000 analyses per second...

--edit, if you read the post above using the hashed db, you'll see where the problem lies - it takes ~0.1 sec to generate a pixel hash of an image - the comparison of that hash against a hashdb is lightening fast. I will see if I can reduce this time to generate the hash, but in the order of 10^5 faster - no ways...

Quote:

Originally Posted by Nautilus

I'll upload some pics for you, check your e-mail.

didn't receive anything...

signupdamnit · 10-10-2010, 09:48 AM

Quote:

Originally Posted by Nathan

borked,

tagging the content is an interesting concept in order to sue the people that distribute it, that's for sure.

It definitely is. If tagged content for instance ended up on a pirate tube and further investigation showed that the uploader was employed by the pirate tube in some way then I imagine a lot could potentially be done legally.

Nautilus · 10-10-2010, 10:43 AM

Quote:

Originally Posted by borked

ah! in all honesty, there's no way this app is going to be able to do 278,000 analyses per second...

--edit, if you read the post above using the hashed db, you'll see where the problem lies - it takes ~0.1 sec to generate a pixel hash of an image - the comparison of that hash against a hashdb is lightening fast. I will see if I can reduce this time to generate the hash, but in the order of 10^5 faster - no ways...

Yes I see where the problem is now... Maybe it is possible to turn it into some kind of distributed operation? I mean, volunteers will be running some apps on their PCs (or install them on their servers, to run whenever there's some computing power to spare), create hushes and then send them back to a central database.

Quote:

didn't receive anything...

Took awile to upload, you can download it now - ~10K pics, should be enough for tests.

Nathan · 10-10-2010, 12:32 PM

Quote:

Originally Posted by borked

Yes, the thread evolved to allow for tagging videos with member details, so that if/when pirated content was found, the original member could be removed from the members area and if needed followed up with prosecution with "direct proof" of pirating user content.

However, there are quite a number of sites that I know of that only allow streams - don't discount an idea since it isn't what "the big sites" use...

We are one of those "big sites" and we tested everything under the moon, and continue to test everything under the moon.. Allowing downloads increases member retention.

borked · 10-10-2010, 12:47 PM

Quote:

Originally Posted by Nathan

We are one of those "big sites" and we tested everything under the moon, and continue to test everything under the moon.. Allowing downloads increases member retention.

I should have used "just because" instead of "it since"

don't discount something just because the big sites use it.

If downloads were so important, tubes wouldn't exist.

Anyway, this isn't the thread for discussing this kind of stuff - it's about putting the options and possibilities on the table so each producer/site owner can make their own decisions - I'm all for giving/having the choice.

borked · 10-10-2010, 12:49 PM

Quote:

Originally Posted by Nautilus

Took awile to upload, you can download it now - ~10K pics, should be enough for tests.

Thanks - I'll keep you updated how the testing goes. I downloaded it to my dev server which got the best speed (1.5M/s). Now taking it from there over to the quad core since may as well test on the best server I have. Speed though is lousy as it's a transatlantic hope from dev to europe. eta 5hrs

looking forward to testing them out

borked · 10-10-2010, 11:40 PM

OK, on a database hash of 10,315 images, I took a dataset of 281 images and asked to find the matches - it took 38.52 seconds. The vast majority of that time (36.53 secs) was taken up by creating the hashes of the 281 images (which was expected)

That's 2,898,515 hash-against-hash calculations in 2.01 seconds (5.19 billion calculations per hour).

Nathan · 10-10-2010, 11:55 PM

borked, very nice...

now, if we could only setup our own industry wide video fingerprinting so we do not have to pay some company to run it, that would be lovely... q is, can we without using some patent that possibly exists?

Your current hashes, they change if the image is resized (preserving aspect) or lowered in quality, right? So how can we build hashes which are still accurate enough but do not care about resizing or quality loss?

Any thoughts on that?

I am wondering if changing resolution of an image to a very low number, like 50x50 or so, if the colors would get close enough together regardless of how the image is cut or changed in quality?

IE, take a square part of the inside of an image of around 1000x1000, re-size it to 50x50 using a standard re-size technique which interpolates the colors.

Then use this on two versions of the same image, jpeg at 100 and jpeg at 50% quality.. and see what happens to the outcome, compare it visually...

borked · 10-10-2010, 11:58 PM

better still...
I created 120x90 thumbs of all the 281 query images (which would also distort them all as it doesn't preserve aspect ratio) and used the thumbs to query the db...

it took 3.69 seconds

borked · 10-11-2010, 12:04 AM

Quote:

Originally Posted by Nathan

borked, very nice...

now, if we could only setup our own industry wide video fingerprinting so we do not have to pay some company to run it, that would be lovely... q is, can we without using some patent that possibly exists?

Your current hashes, they change if the image is resized (preserving aspect) or lowered in quality, right? So how can we build hashes which are still accurate enough but do not care about resizing or quality loss?

Any thoughts on that?

I am wondering if changing resolution of an image to a very low number, like 50x50 or so, if the colors would get close enough together regardless of how the image is cut or changed in quality?

IE, take a square part of the inside of an image of around 1000x1000, re-size it to 50x50 using a standard re-size technique which interpolates the colors.

Then use this on two versions of the same image, jpeg at 100 and jpeg at 50% quality.. and see what happens to the outcome, compare it visually...

Nathan - no, the hash db is constant as that is the reference data set. The hashes for the queries are created on the fly (these aren't one-way ie md5 hashes by the way but floating reference point hashes) and used to query the hash db.

See my point above for distorting images - I used the imagemagick command
mogrify -format jpg -define jpeg:size=240x180 -thumbnail 120x90 '*.jpg'

to make thumbs of all the images in the query set and all came back with the original image found in the db. 2 false negatives slipped through.

This is only for images, not videos and no patents were violated in its creation.

borked · 10-11-2010, 12:15 AM

an eg of original image that had been pre-hashed and a thumb that found its original image when queried

You can see that the query image has lost a lot of information, but still finds its original. So much faster and about as efficient to do this on thumbs, so long as the reference db was created on full-size images.

Robbie · 10-11-2010, 12:18 AM

Nice read borked

This is what I implemented over 2 years ago on our videos. And you are 100% on the money.

As for the folks convinced they will lose members or it will be a "miserable experience": Claudia-Marie's site is kicking ass on both new sales and rebills. The high quality stream can not be downloaded, and since it's h264 compression I'm able to keep the bit rate down to 1.2 to 1.5 Mbps which doesn't put a lot of strain on the customers computer.

I've actually done this for a little more than 2 years. And since I do all the support for the site AND we have a "wall" on the main page of the members area for members to write anything they like...I can tell you that it's a non-issue with our members.

The one thing I did to completely stop any complaints was I actually DO give them a downloadable version. But the DL version is only 480 x 272 and is a much lower bit rate. Just enough for an honest member to be able to watch it when he's not online...but just crappy enough to not be the kind of quality that most would want to upload to a pirate site. Every once in a while they do...but removeyourcontent comes along and gets those down in a timely manner.

And that simple thing eliminated all complaints. Surfing her members area myself...I would never even bother with the download. The player in the members area is so much better. Big high def stream that starts instantly, and I can click anywhere in the timeline and start streaming from that point instantly. So if I don't want to watch the blowjob, I can go straight to the anal.

It's truly a MUCH better experience than my original .wmv set up. Looks better, works better, and gives them an experience that they are used to from YouTube (only mine streams better lol )

And yeah...the high res stream displays their username and IP address for a couple of seconds at random times in random places on the vid. Nobody even really notices it because it's small and unobtrusive. But IF they ever screen record one of my high res streams...I'm going to know EXACTLY who did it...right down to their real name and address straight out of NATS data base. And then? They would be getting a letter from our lawyer and paying a settlement.

I can report to all of you that "yes" it does indeed work.

And yes it has made money hand over fist for me since I first implemented it. Of course we offer a lot in our members area which is important too. Lots of interaction, a "community" with profiles, etc. and Claudia-Marie does her webcam show every week for them as well.

I personally communicate with our members in the members area (unlike most of the "big" paysite owners) and I know exactly what the concerns of our members are. And it ain't whether or not they can download the streams. Our members are more interested in putting in requests for what they want to see CM do in her next scene.

It's a remarkable concept...the members are actually interested in the PORN and not the delivery. That's what happens when you give an innovative and high quality delivery system and combine it with a product that they want.

Nautilus · 10-11-2010, 08:56 AM

Quote:

Originally Posted by borked

OK, on a database hash of 10,315 images, I took a dataset of 281 images and asked to find the matches - it took 38.52 seconds. The vast majority of that time (36.53 secs) was taken up by creating the hashes of the 281 images (which was expected)

That's 2,898,515 hash-against-hash calculations in 2.01 seconds (5.19 billion calculations per hour).

Quote:

Originally Posted by borked

better still...
I created 120x90 thumbs of all the 281 query images (which would also distort them all as it doesn't preserve aspect ratio) and used the thumbs to query the db...

it took 3.69 seconds

Now that's workable

If my calculations are correct, at that speed you can compare ~ 6500 images/day with a database of 1 million pics (hashes), and if using thumbnails only that goes up to ~ 65 000 images/day - more than enough to keep ALL boards and torrents under control image piracy wise.

incredibleworkethic · 10-11-2010, 09:08 AM

If I ran my own pay site, I would be upset if people stole content, however I'd be even more pissed at the people MASS DISTRIBUTING it. Targeting torrent sites / tube sites with law or some kind of brute force would kill the mother feeding the children.

Nathan · 10-11-2010, 09:14 AM

borked, nice.. what happends if you lower the jpg quality of the thumbnail to 50 or even 25? does it still find the right match?

What if you crop the thumbnail? What if you do both crop and lower quality?

Nautilus · 10-11-2010, 09:22 AM

Quote:

Originally Posted by borked

an eg of original image that had been pre-hashed and a thumb that found its original image when queried

You can see that the query image has lost a lot of information, but still finds its original. So much faster and about as efficient to do this on thumbs, so long as the reference db was created on full-size images.

Very impressive.

With such an accuracy, and with some additional programming we can even detect multi-thumbnail previews that are popular at surfer boards too.

borked · 10-11-2010, 11:07 AM

Quote:

Originally Posted by Nathan

borked, nice.. what happends if you lower the jpg quality of the thumbnail to 50 or even 25? does it still find the right match?

What if you crop the thumbnail? What if you do both crop and lower quality?

I don't want to go into the ins and outs of how cropping affects calculations on a public board, but it can be detected - see below

I've taken a single master image and made the following "attacks"

lks_g004_007.jpg - original image
lks_g004_007_o.jpg - another orig image
lks_g004_007_t.jpg - 90x120 thumb of orig
lks_g004_007_t_10.jpg - 90x120 thumb with jpeg quality of 10
lks_g004_007_c_60.jpg - crop of orig image at jpeg 60
lks_g004_007_c_t.jpg - crop of c_60
lks_g004_007_c_t_10.jpg as c_t but jpeg 10

all passed bar lks_g004_007_c_t_10.jpg. c_t brought up 2 false negatives.

all images in the test query set can be found here

output is:

Code:

./hashdbQ images/test-queryattacked hashes.mvp 
using hash database: test-pics-hashes
using query image directory: images/test-queryattacked for query files
number of query files: 7
number of database hashes: 10315

query[0]: images/test-queryattacked/lks_g004_007.jpg e99fa2c502d26d3c
 1 files found
 0  images/test-pics/lks_g004_007.jpg confidence = 100.000000
query[1]: images/test-queryattacked/lks_g004_007_t.jpg e99fa2c506c26d3c
 1 files found
 0  images/test-pics/lks_g004_007.jpg confidence = 98.000000
query[2]: images/test-queryattacked/lks_g004_007_c_t_10.jpg e11be64c86874bbc
 0 files found
query[3]: images/test-queryattacked/lks_g004_007_o.jpg e99ea2c502db2d3c
 1 files found
 0  images/test-pics/lks_g004_007.jpg confidence = 96.000000
query[4]: images/test-queryattacked/lks_g004_007_c_t.jpg e11be64c86874bbc
 3 files found
 0  images/test-pics/boyslovematures_g5026_018.jpg confidence = 79.000000
 1  images/test-pics/lks_g004_007.jpg confidence = 79.000000
 2  images/test-pics/licksonic_g5115_008.jpg confidence = 77.000000
query[5]: images/test-queryattacked/lks_g004_007_c_60.jpg e11be6cc06874bbc
 0 files found
 1  images/test-pics/lks_g004_007.jpg confidence = 83.000000
query[6]: images/test-queryattacked/lks_g004_007_t_10.jpg e99fa2c506c26d3c
 1 files found
 0  images/test-pics/lks_g004_007.jpg confidence = 98.000000

borked · 10-11-2010, 11:14 AM

Quote:

Originally Posted by Nautilus

Now that's workable

If my calculations are correct, at that speed you can compare ~ 6500 images/day with a database of 1 million pics (hashes), and if using thumbnails only that goes up to ~ 65 000 images/day - more than enough to keep ALL boards and torrents under control image piracy wise.

yup - something around there. Querying the db once the hashes are made is really quite rapid by comparison. *However* the script searching for the images has to be well-targetted since don't forget the images to be used to query the db include bw usage and time, which adds to the overall time (the most deciding factor I'd say)

borked · 10-11-2010, 11:49 AM

I need to merge the code that compares 1 image against 1 image with the newer code of comparing against a hashdb, since if you load the orig image and cropped thumb 10% jpeg, it gets matched at 82% confidence, whereas the hash scanning misses it.

CopyMotion · 10-30-2010, 04:27 PM

Just something you may want to add to your educational series entry.

CopyMotion - adult video copyright protection service.

We have recently launched a service that requires no watermarking or alterations of any kind to find and remove your video content from tube sites. Our proprietary technology is computer vision based, just like the FSC Anti Piracy / Vobile program. The tube videos can be severely altered and they will still be found. This is explained in detail in the Video Matching Algorithm section of our website.

What's really unique is that we bring this to the table at a much lower price point than our competition. We believe for this technology to have a real impact in helping the adult industry it must be affordable to all content producers, not just the mega studios.

Cheers.

Hentaikid · 10-31-2010, 05:04 AM

This is all fascinating stuff, but one question, are pirates really hiding at all? I get the impression threads posting pirated stuff just use the name of the video on the title or post... there's no need to detect the image automatically when they're just calling it "Torrent of Video_name"...

borked · 11-03-2010, 04:00 PM

Quote:

Originally Posted by Hentaikid

This is all fascinating stuff, but one question, are pirates really hiding at all? I get the impression threads posting pirated stuff just use the name of the video on the title or post... there's no need to detect the image automatically when they're just calling it "Torrent of Video_name"...

well the thread was originally about "piracy prevention" to stop your content getting onto the tube sites in the first place.

Then it evolved into detection. One of the main reasons for automated detection would be to save you the hassle of finding it to help in the sending out of take down notices...

ottopottomouse · 11-04-2010, 04:17 AM

Sure there must be a use for dating sites to auto-nuke all the scamming Raven Riley etc profiles.

10-08-2010, 08:51 AM	#154
Relentless www.EngineFood.com Industry Role: Join Date: Aug 2006 Posts: 5,697	A good post and an excellent read. Thanks. The only problem with it is the part where you wrote "The only way to "rip" your stream is to have a screen capture program record full playback of your move. Impossible to prevent that!" That's a very weak Achilles heel to the whole 'solution.' It means essentially, YES your videos can still be pirated. And once anyone does it, they have a pirated version of the video which they can use to propagate tubes, torrents and other p2p media exactly the same way they would have if they had downloaded it. The basic problem is that you only need 1 competent thief and all the incompetent ones get access to whatever he stole exactly the same way as if they stole it. __________________ Website Secure \| Engine Food ICQ# 266-942-896 Last edited by Relentless; 10-08-2010 at 08:52 AM..

10-09-2010, 03:58 PM	#166
borked Totally Borked Industry Role: Join Date: Feb 2005 Posts: 6,284	to give an idea of size, hash db is 8.0KB for a directory of 16MB images (55 in total). Can't get better than that with how I'm hashing things... __________________ For coding work - hit me up on andy // borkedcoder // com (consider figuring out the email as test #1) All models are wrong, but some are useful. George E.P. Box. p202

10-09-2010, 06:27 PM	#168
Nautilus Confirmed User Industry Role: Join Date: Sep 2002 Posts: 1,631	You can optimize your search of course, and only analyze threads/posts that contain some specific keywords. You can also narrow your database and compare with either your most recent or most pirated pictures (sets). But that raises new issues/questions: 1. That requires much more sophisticated spidering software that filters what you need. 2. Keyword filters are bound to be innacurate, especially for mainstream/hardcore/babe sites. 3. People will still be able to post your stuff that's not in the db. All in all, simply brute forcing all new posts and comparing then against your entire database, is the REAL solution; while everything else is bound to be less accurate and miss a lot of your stuff posted illegally. Thus, please find us some way to make 1 billion comparisons/hour While search optimization (for those who still want to go that route) will be better spent on adding more boards to the pool. __________________ . . FerroCash - 50+ quality niche paysites to promote \| 100K+ FHGs \| Check recently added galleries New sites \| Pantyhose \| Nylon \| Shemale \| Strapon \| Lesbian \| Mature/MILF \| Anal \| Old&Young \| Gay \| Feet Morphing RSS feeds - check them at the Official blog\| Page Peels (Sample 1 : Sample 2) Wish to review or evaluate our sites before promoting them? Contact me for free password. ICQ: 38.89.22.76 e-mail: support AT ferrocash.com

10-10-2010, 03:18 AM	#172
borked Totally Borked Industry Role: Join Date: Feb 2005 Posts: 6,284	Once the hash db has been created, comparing thumbs against it would be faster, yes. And quality of matching wouldn't decrease, just because it's a thumb. Best though to create the original hash db against full size, to get the best quality plot of the pixels involved. If someone finds a forum or something with their content on it, drop me an email giving me permission to download the content for testing and I'll use that as a base - the more images the better! btw, script tested and runs 8x faster on a quad core as opposed to a dual core. __________________ For coding work - hit me up on andy // borkedcoder // com (consider figuring out the email as test #1) All models are wrong, but some are useful. George E.P. Box. p202

10-10-2010, 05:15 AM	#175
Nathan Confirmed User Industry Role: Join Date: Jul 2003 Posts: 3,108	borked, sorry for my ignorance.. but how exactly does hashing images help you find stolen videos? Or am I missing the point of all of this? What images are you hashing and comparing to? Places like PornBB will never have an image on a post which comes cleanly from the members area. 99% of the time it will be a video thumbnail compilation image. __________________ "Think about it a little more and you'll agree with me, because you're smart and I'm right." - Charlie Munger

10-08-2010, 10:48 AM	#161
PXN Confirmed User Join Date: Jun 2008 Posts: 1,548	Clearly a way better and cheaper solution that FSC/APAP fingerprinting approach. Great post borked!

10-10-2010, 05:30 AM	#176
ottopottomouse She is ugly, bad luck. Industry Role: Join Date: Jan 2010 Posts: 13,177	The hashing images is for photosets. __________________ ↑ see post ↑ 13101

10-10-2010, 11:40 PM	#184
borked Totally Borked Industry Role: Join Date: Feb 2005 Posts: 6,284	OK, on a database hash of 10,315 images, I took a dataset of 281 images and asked to find the matches - it took 38.52 seconds. The vast majority of that time (36.53 secs) was taken up by creating the hashes of the 281 images (which was expected) That's 2,898,515 hash-against-hash calculations in 2.01 seconds (5.19 billion calculations per hour). __________________ For coding work - hit me up on andy // borkedcoder // com (consider figuring out the email as test #1) All models are wrong, but some are useful. George E.P. Box. p202

10-10-2010, 11:55 PM	#185
Nathan Confirmed User Industry Role: Join Date: Jul 2003 Posts: 3,108	borked, very nice... now, if we could only setup our own industry wide video fingerprinting so we do not have to pay some company to run it, that would be lovely... q is, can we without using some patent that possibly exists? Your current hashes, they change if the image is resized (preserving aspect) or lowered in quality, right? So how can we build hashes which are still accurate enough but do not care about resizing or quality loss? Any thoughts on that? I am wondering if changing resolution of an image to a very low number, like 50x50 or so, if the colors would get close enough together regardless of how the image is cut or changed in quality? IE, take a square part of the inside of an image of around 1000x1000, re-size it to 50x50 using a standard re-size technique which interpolates the colors. Then use this on two versions of the same image, jpeg at 100 and jpeg at 50% quality.. and see what happens to the outcome, compare it visually... __________________ "Think about it a little more and you'll agree with me, because you're smart and I'm right." - Charlie Munger

10-10-2010, 11:58 PM	#186
borked Totally Borked Industry Role: Join Date: Feb 2005 Posts: 6,284	better still... I created 120x90 thumbs of all the 281 query images (which would also distort them all as it doesn't preserve aspect ratio) and used the thumbs to query the db... it took 3.69 seconds __________________ For coding work - hit me up on andy // borkedcoder // com (consider figuring out the email as test #1) All models are wrong, but some are useful. George E.P. Box. p202

10-11-2010, 12:15 AM	#188
borked Totally Borked Industry Role: Join Date: Feb 2005 Posts: 6,284	an eg of original image that had been pre-hashed and a thumb that found its original image when queried You can see that the query image has lost a lot of information, but still finds its original. So much faster and about as efficient to do this on thumbs, so long as the reference db was created on full-size images. __________________ For coding work - hit me up on andy // borkedcoder // com (consider figuring out the email as test #1) All models are wrong, but some are useful. George E.P. Box. p202

10-11-2010, 12:18 AM	#189
Robbie Leaner, Meaner, Faster Industry Role: Join Date: Aug 2002 Location: Vegas Posts: 20,959	Nice read borked This is what I implemented over 2 years ago on our videos. And you are 100% on the money. As for the folks convinced they will lose members or it will be a "miserable experience": Claudia-Marie's site is kicking ass on both new sales and rebills. The high quality stream can not be downloaded, and since it's h264 compression I'm able to keep the bit rate down to 1.2 to 1.5 Mbps which doesn't put a lot of strain on the customers computer. I've actually done this for a little more than 2 years. And since I do all the support for the site AND we have a "wall" on the main page of the members area for members to write anything they like...I can tell you that it's a non-issue with our members. The one thing I did to completely stop any complaints was I actually DO give them a downloadable version. But the DL version is only 480 x 272 and is a much lower bit rate. Just enough for an honest member to be able to watch it when he's not online...but just crappy enough to not be the kind of quality that most would want to upload to a pirate site. Every once in a while they do...but removeyourcontent comes along and gets those down in a timely manner. And that simple thing eliminated all complaints. Surfing her members area myself...I would never even bother with the download. The player in the members area is so much better. Big high def stream that starts instantly, and I can click anywhere in the timeline and start streaming from that point instantly. So if I don't want to watch the blowjob, I can go straight to the anal. It's truly a MUCH better experience than my original .wmv set up. Looks better, works better, and gives them an experience that they are used to from YouTube (only mine streams better lol ) And yeah...the high res stream displays their username and IP address for a couple of seconds at random times in random places on the vid. Nobody even really notices it because it's small and unobtrusive. But IF they ever screen record one of my high res streams...I'm going to know EXACTLY who did it...right down to their real name and address straight out of NATS data base. And then? They would be getting a letter from our lawyer and paying a settlement. I can report to all of you that "yes" it does indeed work. And yes it has made money hand over fist for me since I first implemented it. Of course we offer a lot in our members area which is important too. Lots of interaction, a "community" with profiles, etc. and Claudia-Marie does her webcam show every week for them as well. I personally communicate with our members in the members area (unlike most of the "big" paysite owners) and I know exactly what the concerns of our members are. And it ain't whether or not they can download the streams. Our members are more interested in putting in requests for what they want to see CM do in her next scene. It's a remarkable concept...the members are actually interested in the PORN and not the delivery. That's what happens when you give an innovative and high quality delivery system and combine it with a product that they want. __________________ -Robbie ClaudiaMarie.Com Last edited by Robbie; 10-11-2010 at 12:26 AM..

10-11-2010, 09:08 AM	#191
incredibleworkethic Confirmed User Industry Role: Join Date: Sep 2009 Posts: 2,302	If I ran my own pay site, I would be upset if people stole content, however I'd be even more pissed at the people MASS DISTRIBUTING it. Targeting torrent sites / tube sites with law or some kind of brute force would kill the mother feeding the children.

10-11-2010, 09:14 AM	#192
Nathan Confirmed User Industry Role: Join Date: Jul 2003 Posts: 3,108	borked, nice.. what happends if you lower the jpg quality of the thumbnail to 50 or even 25? does it still find the right match? What if you crop the thumbnail? What if you do both crop and lower quality? __________________ "Think about it a little more and you'll agree with me, because you're smart and I'm right." - Charlie Munger

10-11-2010, 11:49 AM	#196
borked Totally Borked Industry Role: Join Date: Feb 2005 Posts: 6,284	I need to merge the code that compares 1 image against 1 image with the newer code of comparing against a hashdb, since if you load the orig image and cropped thumb 10% jpeg, it gets matched at 82% confidence, whereas the hash scanning misses it. __________________ For coding work - hit me up on andy // borkedcoder // com (consider figuring out the email as test #1) All models are wrong, but some are useful. George E.P. Box. p202

10-30-2010, 04:27 PM	#197
CopyMotion Registered User Industry Role: Join Date: Aug 2010 Location: Copyright Protection Posts: 10	Automated adult video copyright protection Just something you may want to add to your educational series entry. CopyMotion - adult video copyright protection service. We have recently launched a service that requires no watermarking or alterations of any kind to find and remove your video content from tube sites. Our proprietary technology is computer vision based, just like the FSC Anti Piracy / Vobile program. The tube videos can be severely altered and they will still be found. This is explained in detail in the Video Matching Algorithm section of our website. What's really unique is that we bring this to the table at a much lower price point than our competition. We believe for this technology to have a real impact in helping the adult industry it must be affordable to all content producers, not just the mega studios. Cheers. __________________ CopyMotion, the most advanced copyright protection available in the adult industry. High-tech computer vision monitoring technology protecting your videos from unauthorized distribution. http://www.copymotion.com

10-31-2010, 05:04 AM	#198
Hentaikid Confirmed User Join Date: Nov 2002 Posts: 1,249	This is all fascinating stuff, but one question, are pirates really hiding at all? I get the impression threads posting pirated stuff just use the name of the video on the title or post... there's no need to detect the image automatically when they're just calling it "Torrent of Video_name"...

11-04-2010, 04:17 AM	#200
ottopottomouse She is ugly, bad luck. Industry Role: Join Date: Jan 2010 Posts: 13,177	Sure there must be a use for dating sites to auto-nuke all the scamming Raven Riley etc profiles. __________________ ↑ see post ↑ 13101