quick question for server gurus - GoFuckYourself.com

latinasojourn · 05-03-2006, 11:37 PM

i think this issue was discussed before on GFY but i can't find the thread.

here's the problem.

let's say you have a dedicated box which has tons of video galleries on it.

and you have guys with very high speed connections clicking on like 20 of the vids all for simultaneous dowload, then they go have breakfast.

and maybe you have 5 guys at the same time do this stuff.

this degrades the server performance.

what is the fix in apache to limit concurrent downloads from a single IP# to like maybe 4 or 5 files?

anyone know, or anyone know of a good fix for this scenario? this is getting more common as surfers get ultra high speed connections.

nestle · 05-03-2006, 11:39 PM

I'd recommend mod_throttle for apache:
http://freshmeat.net/projects/mod_throttle/

here's more info on it:
http://www.snert.com/Software/mod_throttle/index.shtml

teksonline · 05-04-2006, 12:43 AM

nice try but he would need concurrent which is not supported via clientIP or Remote User

Concurrent
Impose a limit on the number of concurrent requests at any one time. The period specifies how long data is accumulated before the counters are reset. This policy cannot be used with either ThrottleClientIP or ThrottleRemoteUser.

fact of the matter is, they are downloading 2 files at a time max, if they are bypassing this rule, then they are breaking the RFC specs

what you need is RAMMAGE

ilbb · 05-04-2006, 12:58 AM

I recommend placing all Your content to thttpd server. Apache is too slow.

teksonline · 05-04-2006, 01:04 AM

dont think you can secure videos on a thttpd server...

nestle · 05-04-2006, 02:17 AM

If mod_throttle won't work for your needs then try mod_choke:

Quote:

mod_choke is a module for Apache that limits usage by greedy clients. It's two main functions are to limit the number of concurrent connections per IP, and to limit the rate at which apache sends data to the client.

mod_choke:
http://os.cyberheatinc.com/mod_choke.php

teksonline, mind letting know where I can find some info about 'rammage' ?

skinnay · 05-04-2006, 02:35 AM

may want to look into mod_limitipconn.c

chaze · 05-04-2006, 02:47 AM

We have a custom mod that limits the megs per user per day.

Other then that your server shouldn't slow down unless it can't handle the load. You may just need to upgrade your server load handling ability.

Also make sure and block download bots from stealing your content.

http://www.dwhs.com/hostingforum/viewtopic.php?t=1114

And password sharing obviously will drain your server as well so blocking that and hotlinking will make a big difference.

darksoul · 05-04-2006, 02:51 AM

freebsd has the nice feature of doing this directly from firewall

Code:

ipfw add 100 allow tcp from any to any 80 in limit src-addr 5

that will limit the number of connections per ip to 5
iptables has also a patch that can do this.

latinasojourn · 05-14-2006, 05:24 PM

Quote:

Originally Posted by darksoul

freebsd has the nice feature of doing this directly from firewall

Code:

ipfw add 100 allow tcp from any to any 80 in limit src-addr 5

that will limit the number of connections per ip to 5
iptables has also a patch that can do this.

tip of the hat to "darksoul"

this tweak is now working, the guy throws a download utility on a movie page and can only get 5 at a time.

NICE big BW saver!

05-03-2006, 11:37 PM	#1
latinasojourn Confirmed User Join Date: Oct 2003 Posts: 3,191	quick question for server gurus i think this issue was discussed before on GFY but i can't find the thread. here's the problem. let's say you have a dedicated box which has tons of video galleries on it. and you have guys with very high speed connections clicking on like 20 of the vids all for simultaneous dowload, then they go have breakfast. and maybe you have 5 guys at the same time do this stuff. this degrades the server performance. what is the fix in apache to limit concurrent downloads from a single IP# to like maybe 4 or 5 files? anyone know, or anyone know of a good fix for this scenario? this is getting more common as surfers get ultra high speed connections.

05-04-2006, 02:35 AM	#7
skinnay Confirmed User Join Date: Apr 2004 Location: NEW YORK CITY Posts: 2,274	may want to look into mod_limitipconn.c __________________ Make Real Green with ORGANIC SEO \| Blog post exchange \| Non-index page trades \| A-B C-D Trades [icq: 194-215-962] [mail: [email protected]]

05-04-2006, 02:51 AM	#9
darksoul Confirmed User Join Date: Apr 2002 Location: /root/ Posts: 4,997	freebsd has the nice feature of doing this directly from firewall Code: ipfw add 100 allow tcp from any to any 80 in limit src-addr 5 that will limit the number of connections per ip to 5 iptables has also a patch that can do this. __________________ 1337 5y54\|)m1n: 157717888 BM-2cUBw4B2fgiYAfjkE7JvWaJMiUXD96n9tN Cambooth

05-03-2006, 11:39 PM	#2
nestle Confirmed User Industry Role: Join Date: Apr 2006 Posts: 647	I'd recommend mod_throttle for apache: http://freshmeat.net/projects/mod_throttle/ here's more info on it: http://www.snert.com/Software/mod_throttle/index.shtml

05-04-2006, 12:43 AM	#3
teksonline So Fucking Banned Join Date: Jan 2005 Location: At My Desk Posts: 2,904	nice try but he would need concurrent which is not supported via clientIP or Remote User Concurrent Impose a limit on the number of concurrent requests at any one time. The period specifies how long data is accumulated before the counters are reset. This policy cannot be used with either ThrottleClientIP or ThrottleRemoteUser. fact of the matter is, they are downloading 2 files at a time max, if they are bypassing this rule, then they are breaking the RFC specs what you need is RAMMAGE

05-04-2006, 12:58 AM	#4
ilbb Confirmed User Industry Role: Join Date: May 2005 Location: EU - Czech republic Posts: 3,025	I recommend placing all Your content to thttpd server. Apache is too slow.

05-04-2006, 01:04 AM	#5
teksonline So Fucking Banned Join Date: Jan 2005 Location: At My Desk Posts: 2,904	dont think you can secure videos on a thttpd server...

05-04-2006, 02:47 AM	#8
chaze Confirmed User Industry Role: Join Date: Aug 2002 Posts: 9,752	We have a custom mod that limits the megs per user per day. Other then that your server shouldn't slow down unless it can't handle the load. You may just need to upgrade your server load handling ability. Also make sure and block download bots from stealing your content. http://www.dwhs.com/hostingforum/viewtopic.php?t=1114 And password sharing obviously will drain your server as well so blocking that and hotlinking will make a big difference.