Fellow coders I need some advice - GoFuckYourself.com - Adult Webmaster Forum

X

ClickBuster

Confirmed User

Join Date: May 2004

Posts: 210
Share

Post
#1

Fellow coders I need some advice

12-29-2007, 06:18 AM

I want to make a script that restricts the member area bandwidth usage per member per hour. My question is how do I check the used bandwidth without having to pass all content through scripts. So far the only whatso usable solution to this problem is using iptables for bandwidth info per IP. If anybody can give me any advice, it'd be greatly appreciated.

Thanks in advance,
Andrew

P.S.
Hope you all have a Happy New Year

-- ClickBuster
-- ICQ# 263653704
-- Email: clickbuster1 [-at-] gmail [-dot-] com
Tags: None
pip

So Fucking Banned

Join Date: Nov 2005

Posts: 468
Share

Post
#2

12-29-2007, 06:34 AM

Think ftvcash does this how much you can allow per day, stops people from downloading everything
Comment
ClickBuster

Confirmed User

Join Date: May 2004

Posts: 210
Share

Post
#3

12-29-2007, 07:31 AM

Many sites have that actually

-- ClickBuster
-- ICQ# 263653704
-- Email: clickbuster1 [-at-] gmail [-dot-] com
Comment
sortie

Confirmed User

Join Date: Mar 2007

Posts: 7771
Share

Post
#4

12-29-2007, 07:45 AM

Build a table with the content size of each piece of media.

Use a redirect script (like traffic trading: /cgi-bin/media.cgi?url=/themedia.mpg) to print "location: /themedia.mpg\n\n"

The redirect script reads the content size table based on the url and adds to a cookie and IP file the total amount of bandwidth and redirects to an error page if bandwidth exceeded.

There's several other ways but this is probably the easiest.

Using htaccess to hide the redirect script can help make it harder to defeat.
Comment
FightThisPatent

Confirmed User

Join Date: Aug 2003

Posts: 4090
Share

Post
#5

12-29-2007, 07:52 AM

Originally posted by sortie

Build a table with the content size of each piece of media.

Use a redirect script (like traffic trading: /cgi-bin/media.cgi?url=/themedia.mpg) to print "location: /themedia.mpg\n\n"

The redirect script reads the content size table based on the url and adds to a cookie and IP file the total amount of bandwidth and redirects to an error page if bandwidth exceeded.

There's several other ways but this is probably the easiest.

Using htaccess to hide the redirect script can help make it harder to defeat.

good idea, but this assumes that the person viewed the entire file... so if the mpg file is 400meg, and they got their satisfaction around 100meg point, then they got "over charged" (assuming content was streaming vs. downlaoded as a whole clip).

you can have a background cron job that runs every minute that does b/w stat checks by IP (as you were thinking) to fill in a table, and your PHP script checks that table based on IP. (this method would help with streamed content)

Most ppl have the same IP during their session with you... AOL users can have IP change in the middle of their time on your site...

Fight the .02!

Last edited by FightThisPatent; 12-29-2007, 07:53 AM.

http://www.t3report.com (where's the traffic?) v5.0 is out! |
http://www.FightThePatent.com | ICQ 52741957
Comment
MMarko

Confirmed User

Join Date: Jun 2007

Posts: 160
Share

Post
#6

12-29-2007, 07:54 AM

afaik, there are some robust production grade scripts available for this... I think guy who made strongbox has script for this too (it's actualy an apache module)

dlXer - web design, developing, managed hosting, website optimizations
Comment
FightThisPatent

Confirmed User

Join Date: Aug 2003

Posts: 4090
Share

Post
#7

12-29-2007, 08:22 AM

Originally posted by MMarko

afaik, there are some robust production grade scripts available for this... I think guy who made strongbox has script for this too (it's actualy an apache module)

ah, yes, apache modules... found this:
http://digg.com/linux_unix/Manage_Ap...With_mod_cband

Fight the 411!

http://www.t3report.com (where's the traffic?) v5.0 is out! |
http://www.FightThePatent.com | ICQ 52741957
Comment
ClickBuster

Confirmed User

Join Date: May 2004

Posts: 210
Share

Post
#8

12-29-2007, 08:35 AM

Originally posted by FightThisPatent

ah, yes, apache modules... found this:
http://digg.com/linux_unix/Manage_Ap...With_mod_cband

Fight the 411!

Thats not it To be honest, so far the iptables version seems to be the smartest solution, but I still have too hook with a cron job to count the used bandwidth...

-- ClickBuster
-- ICQ# 263653704
-- Email: clickbuster1 [-at-] gmail [-dot-] com
Comment
sortie

Confirmed User

Join Date: Mar 2007

Posts: 7771
Share

Post
#9

12-29-2007, 08:48 AM

Originally posted by ClickBuster

Thats not it To be honest, so far the iptables version seems to be the smartest solution, but I still have too hook with a cron job to count the used bandwidth...

Cron? We don't need no stinking cron!!

$n="dontstop";
while ($n eq "dontstop") {
&manage_bandwidth;
sleep(60);
if (-e "killme.txt") {exit;}
}
Comment
ultimatebbwdotcom

Confirmed User

Join Date: Mar 2006

Posts: 591
Share

Post
#10

12-29-2007, 08:52 AM

Doesnt Throttlebox do this?

Ultimatebbw.com
Dangerouscurvesdesign.com
Comment
ClickBuster

Confirmed User

Join Date: May 2004

Posts: 210
Share

Post
#11

12-29-2007, 10:31 AM

Originally posted by sortie

Cron? We don't need no stinking cron!!

$n="dontstop";
while ($n eq "dontstop") {
&manage_bandwidth;
sleep(60);
if (-e "killme.txt") {exit;}
}

:D 1337 but I need a bit more explanation to your prop...

-- ClickBuster
-- ICQ# 263653704
-- Email: clickbuster1 [-at-] gmail [-dot-] com
Comment
Why

MFBA

Join Date: Mar 2003

Posts: 7230
Share

Post
#12

12-29-2007, 10:31 AM

there are numerous apache modules that do this.
Comment
Why

MFBA

Join Date: Mar 2003

Posts: 7230
Share

Post
#13

12-29-2007, 10:36 AM

Originally posted by ClickBuster

:D 1337 but I need a bit more explanation to your prop...

its a script that runs an endless loop every 60 seconds because $n never changes it just keeps looping with a 60 second sleep(pause) in the middle. the second to last line checks for the existence of a file, which could be used to turn it on or off. make sense?
Comment
ClickBuster

Confirmed User

Join Date: May 2004

Posts: 210
Share

Post
#14

12-29-2007, 10:40 AM

Originally posted by Why

its a script that runs an endless loop every 60 seconds because $n never changes it just keeps looping with a 60 second sleep(pause) in the middle. the second to last line checks for the existence of a file, which could be used to turn it on or off. make sense?

What if Apache timesout? Also, you can perform only JS redirects after the output. Any specific Apache module suggestions?

-- ClickBuster
-- ICQ# 263653704
-- Email: clickbuster1 [-at-] gmail [-dot-] com
Comment
sortie

Confirmed User

Join Date: Mar 2007

Posts: 7771
Share

Post
#15

12-29-2007, 11:43 AM

Originally posted by Why

its a script that runs an endless loop every 60 seconds because $n never changes it just keeps looping with a 60 second sleep(pause) in the middle. the second to last line checks for the existence of a file, which could be used to turn it on or off. make sense?

Exactamundo!

The "&manage_bandwidth" is a subroutine to do whatever you want.

If you have dedicated server then this is easy way to do some things but maybe some host is paranoid on shared hosting and don't like it.

It's just a choice other than cron; not better in anyway.
Comment
rowan

Too lazy to set a custom title

Join Date: Mar 2002

Posts: 17393
Share

Post
#16

12-29-2007, 11:46 AM

Apache logs show both (a) the authenticated member name and (b) the size of the object it has delivered. It shouldn't be hard to write a script that continuously "tails" the logs and keeps stats on how much each member has downloaded. You could then set up something to modify/rewrite htaccess to redirect any further requests from that member to a page explaining that they've exceeded their quota.

Are you actually having a problem with excessive downloads or are you just being careful? Sometimes pissing off members may cost you more than savings in bw.
Comment
sortie

Confirmed User

Join Date: Mar 2007

Posts: 7771
Share

Post
#17

12-29-2007, 11:55 AM

NOTE: to my last post.

The "killme.txt" file can only turn it off!
I has to be executed again to restart.
Comment
k0nr4d

Confirmed User

Join Date: Aug 2006

Posts: 9231
Share

Post
#18

12-29-2007, 02:14 PM

cron is still better to use then because if you reboot the box it will still do what needs to be done.

as for the issue at hand, I can't think of any way you could limit this outside of an apache module.

Mechanical Bunny Media
Mechbunny Tube Script | Mechbunny Webcam Aggregator Script | Custom Web Development
Comment
ClickBuster

Confirmed User

Join Date: May 2004

Posts: 210
Share

Post
#19

12-29-2007, 05:34 PM

Originally posted by rowan

Apache logs show both (a) the authenticated member name and (b) the size of the object it has delivered. It shouldn't be hard to write a script that continuously "tails" the logs and keeps stats on how much each member has downloaded. You could then set up something to modify/rewrite htaccess to redirect any further requests from that member to a page explaining that they've exceeded their quota.

Are you actually having a problem with excessive downloads or are you just being careful? Sometimes pissing off members may cost you more than savings in bw.

I like this one the best so you say apache logs... mhm I wonder if there's a separate log for authenticated users or if Apache can be modified in anyway to log this data separately? I need this coded for various monitoring reasons, including extensive account usage, account sharing, etc.

-- ClickBuster
-- ICQ# 263653704
-- Email: clickbuster1 [-at-] gmail [-dot-] com
Comment
rowan

Too lazy to set a custom title

Join Date: Mar 2002

Posts: 17393
Share

Post
#20

12-29-2007, 05:40 PM

Originally posted by ClickBuster

I like this one the best so you say apache logs... mhm I wonder if there's a separate log for authenticated users or if Apache can be modified in anyway to log this data separately? I need this coded for various monitoring reasons, including extensive account usage, account sharing, etc.

Yep, look up the LogFormat directive to define a custom log format, then the CustomLog directive (inside <VirtualHost...> )to write it out to a separate file. I'm not 100% sure you can write to more than one log per vhost but I'd guess it's likely.
Comment
raymor

Confirmed User

Join Date: Oct 2002

Posts: 3745
Share

Post
#21

12-29-2007, 07:48 PM

Originally posted by MMarko

afaik, there are some robust production grade scripts available for this... I think guy who made strongbox has script for this too (it's actualy an apache module)

Originally posted by ultimatebbwdotcom

Doesnt Throttlebox do this?

Yep, Throttlebox, from the makers of Strongbox, does this, but does it in an
ITELLIGENT way. Just counting bandwidth without more intelligence than that
seems to cause more problems than it solves.

I wouldn't want to redirect every hit, or even count everything through iptables,
the overhead seems to high to me. Instead we track bandwidth only when it
actually matters, and we do it using a custom Apache module. This is something
we worked on for a few years, so you may find that it makes much more
sense to take advantage of all of the time that we spent on this, all of the
research, the discussions we had with university professors about declining
averages, etc. rather than either doing all of that work over again yourself
or skipping the hard part and building something that doesn't work so well.

For historical display only. This information is not current:
support@bettercgi.com ICQ 7208627
Strongbox - The next generation in site security
Throttlebox - The next generation in bandwidth control
Clonebox - Backup and disaster recovery on steroids
Comment
ClickBuster

Confirmed User

Join Date: May 2004

Posts: 210
Share

Post
#22

12-30-2007, 02:31 AM

Originally posted by raymor

Yep, Throttlebox, from the makers of Strongbox, does this, but does it in an
ITELLIGENT way. Just counting bandwidth without more intelligence than that
seems to cause more problems than it solves.

I wouldn't want to redirect every hit, or even count everything through iptables,
the overhead seems to high to me. Instead we track bandwidth only when it
actually matters, and we do it using a custom Apache module. This is something
we worked on for a few years, so you may find that it makes much more
sense to take advantage of all of the time that we spent on this, all of the
research, the discussions we had with university professors about declining
averages, etc. rather than either doing all of that work over again yourself
or skipping the hard part and building something that doesn't work so well.

Not looking for a ready solution, but thanks for the input.

-- ClickBuster
-- ICQ# 263653704
-- Email: clickbuster1 [-at-] gmail [-dot-] com
Comment
k0nr4d

Confirmed User

Join Date: Aug 2006

Posts: 9231
Share

Post
#23

12-30-2007, 03:16 AM

Give mod_leech a shot,
http://toroid.org/ams/mod_leech

Mechanical Bunny Media
Mechbunny Tube Script | Mechbunny Webcam Aggregator Script | Custom Web Development
Comment
ClickBuster

Confirmed User

Join Date: May 2004

Posts: 210
Share

Post
#24

12-30-2007, 04:38 AM

Originally posted by k0nr4d

Give mod_leech a shot,
http://toroid.org/ams/mod_leech

I like this one and gonna try it for sure. The hahahahahaha story sucks too...

Edit: ROFL the shady hosting company name got changed to hahahahahaha

-- ClickBuster
-- ICQ# 263653704
-- Email: clickbuster1 [-at-] gmail [-dot-] com
Comment
raymor

Confirmed User

Join Date: Oct 2002

Posts: 3745
Share

Post
#25

12-30-2007, 10:04 AM

Originally posted by ClickBuster

Not looking for a ready solution, but thanks for the input.

Ok, good luck with it. Be forwarned to get something to actually work well and
not create problems is a bigger job than it first appears.

For historical display only. This information is not current:
support@bettercgi.com ICQ 7208627
Strongbox - The next generation in site security
Throttlebox - The next generation in bandwidth control
Clonebox - Backup and disaster recovery on steroids
Comment

Previous template Next

Working...