|
Offline Browsers spidering and crashing apache.
I've got a few sites that get spidered frequently and an occasion rogue apache process will hang.. consuming cpu and memory, crashing mysql and shooting the server load up. I'll usually get 1 ip bounce all over the server grabbing pages from numerous domains.
Example Top output:
5 root 14 0 0 0 0 RW 0 68.7 0.0 63:48 kswapd
7156 www 18 0 446M 371M 56 R 0 63.6 42.2 12:41 apache
7158 www 14 0 434M 382M 56 R 0 58.2 43.5 12:26 apache
Is there anything that will automatically monitor for those apache processes at least and kill them before they reach this point ?
What about a mod_throttle solution or is that going overboard ?
Suggestions ?
|