Expected memory usage?

[subBlue]

We're half way through the 30 day trial of LS Enterprise and have 20 sites up and running using our Rails based CMS. One thing we have noticed is an increasing memory requirement over a few days. From a clean reboot the memory usage starts about 550Mb then over the course of the first day will increase to about 1.3Gb. By the end of the second day it is at 2.5Gb. I had wondered if there is a memory leak in our CMS, but a ps uax trace shows something like this:

USER       PID %CPU %MEM   VSZ  RSS TTY      STAT START   TIME COMMAND
apache   15050  0.0  0.8 39580 35596 ?       SN   Oct19   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   24868  0.0  0.8 38472 34448 ?       SN   Oct19   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   31944  0.0  0.9 41124 37428 ?       SN   Oct19   0:16 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   32649  0.0  0.8 40116 36216 ?       SN   Oct19   0:06 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
admin     2584  0.0  0.6 29704 26356 ?       SNs  07:18   0:03 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache    2623  0.0  0.6 29984 26640 ?       SNs  07:19   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
admin    10346  0.0  0.6 29704 26356 ?       SNs  08:59   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   13587  0.1  0.6 29984 26640 ?       SNs  09:31   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   15510  0.3  0.5 24740 21856 ?       SNs  09:51   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   15825  0.4  0.5 24612 21708 ?       SNs  09:53   0:01 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   15838  0.5  0.5 24740 21856 ?       SNs  09:53   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   16148  1.0  0.5 24608 21604 ?       SNs  09:57   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
admin    16247  1.6  0.7 35380 31284 ?       SN   09:58   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   16263  4.1  0.8 39796 35700 ?       SN   09:58   0:05 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
admin    16284 10.5  0.8 40928 36832 ?       SN   09:58   0:13 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   16376  2.8  0.5 24472 21588 ?       SNs  09:59   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
admin    16383  7.8  0.8 40508 36412 ?       SN   09:59   0:06 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   16444  3.5  0.5 24472 21588 ?       SNs  09:59   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   16453  6.2  0.8 40604 36856 ?       SN   09:59   0:03 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   16455  4.6  0.6 29968 26676 ?       SNs  09:59   0:02 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   16462  0.0  0.6 29968 26680 ?       SN   09:59   0:00 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   16493  3.7  1.1 56516 47544 ?       SN   09:59   0:01 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
apache   16500  1.4  0.6 28716 24964 ?       SN   09:59   0:00 ruby /usr/local/lsws/fcgi-bin/RailsRunner.rb
root      2544  0.0  0.0  4668 2516 ?        S    07:18   0:01 lshttpd
apache    2546  0.0  0.0  5816 3840 ?        S    07:18   0:01 lshttpd
apache    2547  0.1  0.1  8728 6688 ?        S    07:18   0:10 lshttpd

There is nothing else listed in the ps trace to suggest another app hogging the memory. Our rails apps are setup using the default Rails VH template and we are running on a Dual Xeon 2.8GHz with 4Gb RAM and Centos 4.2 From the Real-time server log we only have 4-5 of the sites active most of the time as the others are still in development and the busiest site gets about 2500 visitors a day.

The server is usually complete quiet between 2 and 7am so I'm wondering if the memory usage should be dropping back down each morning? Should I be worried about the amount of memory usage on this server, or is this something that you would expect to see?

many thanks for any suggestions/thoughts.

[xing]

From what you posted, LiteSpeed processes are using 14MB of real ram total, which is normal.

Linux/CentOS will agressively cache file read requests in the file buffer. So If all your processes leave 3G of ram "free", the kernel will use all that ram to cache file i/o. The kernel will use whatever "free/unused" process memory to cache disk requests and relinguish those ram to real processes when the need comes.

What this means that even if LiteSpeed and all your Ruby processes are only using 256MB total, the linux kernel is still free to use and display up to max memory usage.

This is normal and expected behavior.

The most important item is to check "swap" value. It should not be increasing and should hold at a very steady value, only up or down very infrequently.

Use vmstat to see some memory related stats.

[xing]

Forgot to one more thing.

There would be a cause of concern for potential memory leak only if:

Swap value keeps going up with constant activity despite the fact you don't see any process should be using that much memory.

We also can't say about your CMS app and the Ruby app in general. Just make sure you are running the latest Ruby binary and not the not the one that came with CentOS 4.2.

[subBlue]

Thanks for your feedback, you have put my mind at ease I didn't realise that the kernal will using the remaining ram to cache i/o. Our swap is consistantly empty so all seems fine.

[dreamscape]

If you want to see what amount of RAM is actually available to applications, you can run `free -m`. The first line will show used & free without the cache included. But the second line will read "-/+ buffers/cache:" and show how much is actually being used and how much is actually free to applications if you take the cache & buffers into account. The cache and buffers are also reported in top.

If you just read the reported "free" physical RAM, it doesn't take long for a Linux box to appear to have eaten a ton of RAM for apparently no reason. I used to get thrown off by this all the time as well until I learned that cached RAM was actually available for applications to use if they needed it.