I've been trying to understand why one of my Linux servers keeps getting out of memory errors. While Googling around, I found this useful article which talks about measuring Linux memory usage.
It sums up the problem like so:
The moral of this story is that process memory usage on Linux is a complex matter; you can't just run ps and know what is going on. This is especially true when you deal with programs that create a lot of identical children processes, like Apache. ps might report that each Apache process uses 10 megabytes of memory, when the reality might be that the marginal cost of each Apache process is 1 megabyte of memory.
Although the problem of measuring memory on Linux is tricky, the article does give quite a bit of explanation and even introduced me to a new tool - pmap. It's a good place to start for understanding what makes your system go.
Update: Want to see the current top 10 processes that have the largest resident set size? Just run:
ps -e -o user,pid,%cpu,%mem,rss,cmd --sort=-rss | head
ah the joys of linux memory... try on a mac... I think apple has taken the idea of memory confusion and taken it to a new level... i'm not saying that the memory management is better or worse on mac, just I think it is near impossible to determine what is going on.
ReplyDeleteanyways I'll have to remember pmap next time I run into serious memory errors. Also if the error is in 'your' code, see if valgrind is available. It will help you find memory leaks.
"just I think it is near impossible to determine what is going on."
ReplyDeleteSo well put!
I'm also starting to appreciate vmstat more, specifically watching the io wait time.
What I really need to do is just install all my apps on a server with like 16GB of ram, and forget about it.