cannot allocate memory error

Had a weird memory error my Linux servers at work today. These were heavy duty machines running  heavy jobs throughout the day. /var/log/messages kept reporting that the kernel had no more memory to process new jobs. Most of the errors read ” cannot allocate memory ”

Some digging around later we found the fix. Kernel 2.6 has a new parameter “min_free_kbytes” which allows it to reserve a dedicated amount of memory for itself to process jobs. This kept the kernel from choking up when the servers were faced with sudden spikes in heavy jobs.

I set my server’s  “min_free_kbytes” to “4096″ kbytes which was the recommended value. It’s more of a trial and error configuration so I’ll have to monitor the server over a period of time and increase the value if needed till I hit the sweet spot.

How to set it?

To have the new value take effect immediately, edit the “/proc/sys/vm/min_free_kbytes” file. Remember!, reboot and the changes will be forgotten.

echo "4096" > /proc/sys/vm/min_free_kbytes

To have it permanent, add “vm.min_free_kbytes=65536″ to the /etc/sysctl.conf file.

echo "vm.min_free_kbytes=4096" >> /etc/sysctl.conf

One Response to “cannot allocate memory error”

Author comments are in a darker gray color for you to easily identify the posts author in the comments

1. Raja says:

   June 22, 2009 at 9:32 pm

   Dang, you got a bazillion articles here man.

   An alternative way is to edit your sysctl.conf, then run ‘sysctl -p’.

   This will apply the changes immediately.

   Reply