Linux Servers - Important Kernel Update
 

I am posting this because there are alot of people suffering from this
hard to identify problem. So I researched it and came up with the solution.
My gift to the universe.

Is your server eating up memory for apparantly no reason?
Does your memory usage continually grow in 4096 byte increments (approx.)
and fill up all available RAM then being using swap space until it's full
as well? Is your servers load average like 900 or some crazy high number?

The culprit of your problem may be as simple as the kernel version you
are using.

Everyone who had this problem that I helped was using kernel version:
2.6.14, however it might affect other releases as well.

In my research, I have found that this kernel release had a big bad memory
leak related to buggy ext3 filesystem code.

Your server will appear ok until you start up apache or mysql or anything that uses the hard drive. The problem appears to manifest itself as a
side afect of apache (because it shows up typically only when apache is running, which is misleading). Techs will immediatly jump to apache and try solving the problem by tuning apache's MaxClients and Keep Alives and MaxRequestsPerChild.... to no avail. This is because there are settings in
Apache to combat "memory leaks", which are actually not for linux systems
and won't help in this situation.

The buggy ext3 filesystem code in the 2.6.14 Kernel eats up memory
during disk usage. On a lightly used server, the problem may take
weeks to show up (do you have to reboot your server every week or so?).

On a heavily used server, it may take all but 30 minutes of usage. The rule of thumb is the more you use the hard drives, the quicker it eats up the memory, and the quicker your load goes out of control due to Virtual Memory Swap Thrashing.

Your load goes high because of the swap space thrashing and swapping pages continually in and out of memory. This is what causes the load, not apache or mysql.

The solution? Upgrade to kernel 2.6.16, reboot and enjoy the new relaxed attitude your server will have.

Happy Memorial Day everyone!
-SpLiTiNfInItY-

Not much attention to this as I posted it over the holiday... bumping just for the sake of giving everyone the opprotunity to read. :-)

Cheers.

Thanks!
Bump for this

Thanks.. using 2.6.14 on my servers it looks like.

I've had a memory leak problem with Red Hat for ages... kjournald seems to be taking up all the memory

You're such a nerd. :upsidedow Great info.

2.6.8 for me. This kernel is nearly 2 years old! Fuck upgrading when it works just fine. Too many times I've had machines not come back up properly when trying to upgrade the kernel remotely.

thnx
allready recognized the problem :)

Thanks very much for this post...We have had some weird server problems and this may be it.

Will have the Admin look in the a.m.

Thanks for taking your time to post this info and share it with other webmasters.

Love GFY when its useful, Hate GFY when its "would u hit it" posts...


THANKS!

You are welcome.

Anyone got some other odd problems I can help solve?
:-)

Note: I can't help with the wart on your penis. :-) LOL

can you tell me why the linux autofs code sucks as bad as it does?

"can you tell me why the linux autofs code sucks as bad as it does?"


The autofs code poorly spent its $440.00 on this school of trade, and
is therefore why it sucks bad.

http://www.themoscowtimes.com/storie...06/11/013.html

autofs (the userland part) is a little dodgy. It can get upset pretty easily with NFS mounts as can amd (the alternative, written by CSG at Imperial) is only a little better. Both are pretty huge programs for doing a simple task. I think I'll write a replacement over the weekend.

who the heck uses linux for a webserver? :)

Run FreeBSD n00bs

Now there is a great post by a hosting company.

Thanks RobbieRye.

Nice info Chris.

That's why there are so much hacked servers around.
"why update if it's working?" looks the same as "why drive a ferrari when your old car still running" :1orglaugh

Thanks for the heads up Chris. :)

"That's why there are so much hacked servers around.
"why update if it's working?" looks the same as "why drive a ferrari when your old car still running" "


So true. :-)

thanks bro, ill keep this in mind