GoFuckYourself.com - Adult Webmaster Forum - View Single Post - Offline Browsers spidering and crashing apache.

XXXManager · 02-15-2003, 06:56 PM

Hi. I was called here by Mr Fiction

so here I am...
> 7156 www 18 0 446M 371M 56 R 0 63.6 42.2 12:41 apache:
OUCH. 446M 371M? Goddamn.
Anyway.
Hel0 raised some interesting options but the possible list is long. Much longer than listed here.
More info could certainly help identify the problem.

First.. if what you want to stop access from specific remote hosts or IPs or something like that tell me.
Second.. I'll try to respond to some points raised here:
mod_gzip: Not likely the problem
HTTP/1.1: Not likely the problem. I assume the one who mentioned it meant the keep-alive issue. Well - that is usually a helpful thing when answering requests from the same host. So its not the immediate cause in your case.
Mod_throttle: Most likely wont help.
strace: you will need to install it (not default on many/most/all? platforms) and know what it is. Don't go there if you don't know what it is.
MaxRequestsPerChild: Probably has no meaning in your case even regarding HTTP/1.1, unless there is a gradual leak, which could be but I cant say how likely. Change this value to 10 just for the test. MaxRequestsPerChild has a crucial effect mainly if you use resident modules like mod_perl on a persistent connection base. if not you can set it to 1 or 10 like I suggested - for a while to see the effect.
Actually I recommend turning off the keep-alive (state "KeepAlive Off" in httpd.conf) if you dont have graphics rich site. Again - you can do it for testing..
Restarting Apache every 15mins: Not nice.

Question:
1. what is your MaxClients in httpd.conf?
2. assuming you still use keep-alive - what is your KeepAliveTimeout?
3. what is your MaxSpareServers?

Since what you identified as "cause" of the crash can only be an irrelevant syptom - I suggest debugging the server..

Suggestion:
(If you want to fix this for long term)
1. log the top into a file (top -b > /path/to/log/top.log)
2. log the memory usage and swap activity every 10 seconds(vmstat -n 10 > /path/to/log/vmstats.log)
3. Enable the apache logs so you can see the last requests before the hang (I assume you already did that by the info you gave)

If anyone want/need to know anything I mentioned, ask.

02-15-2003, 06:56 PM
XXXManager So Fucking Banned Join Date: Mar 2002 Location: Far out in the uncharted backwaters of the unfashionable end of the Western Spiral arm of the Galaxy Posts: 893	Hi. I was called here by Mr Fiction so here I am... > 7156 www 18 0 446M 371M 56 R 0 63.6 42.2 12:41 apache: OUCH. 446M 371M? Goddamn. Anyway. Hel0 raised some interesting options but the possible list is long. Much longer than listed here. More info could certainly help identify the problem. First.. if what you want to stop access from specific remote hosts or IPs or something like that tell me. Second.. I'll try to respond to some points raised here: mod_gzip: Not likely the problem HTTP/1.1: Not likely the problem. I assume the one who mentioned it meant the keep-alive issue. Well - that is usually a helpful thing when answering requests from the same host. So its not the immediate cause in your case. Mod_throttle: Most likely wont help. strace: you will need to install it (not default on many/most/all? platforms) and know what it is. Don't go there if you don't know what it is. MaxRequestsPerChild: Probably has no meaning in your case even regarding HTTP/1.1, unless there is a gradual leak, which could be but I cant say how likely. Change this value to 10 just for the test. MaxRequestsPerChild has a crucial effect mainly if you use resident modules like mod_perl on a persistent connection base. if not you can set it to 1 or 10 like I suggested - for a while to see the effect. Actually I recommend turning off the keep-alive (state "KeepAlive Off" in httpd.conf) if you dont have graphics rich site. Again - you can do it for testing.. Restarting Apache every 15mins: Not nice. Question: 1. what is your MaxClients in httpd.conf? 2. assuming you still use keep-alive - what is your KeepAliveTimeout? 3. what is your MaxSpareServers? Since what you identified as "cause" of the crash can only be an irrelevant syptom - I suggest debugging the server.. Suggestion: (If you want to fix this for long term) 1. log the top into a file (top -b > /path/to/log/top.log) 2. log the memory usage and swap activity every 10 seconds(vmstat -n 10 > /path/to/log/vmstats.log) 3. Enable the apache logs so you can see the last requests before the hang (I assume you already did that by the info you gave) If anyone want/need to know anything I mentioned, ask.