Error Codes Wiki

Linux OOM Killer Terminated Process — Out of Memory and Memory Leak Detection

Criticalsystem

Overview

Fix the Linux OOM Killer terminating processes due to memory exhaustion, identify memory leaks, and configure memory limits to prevent OOM kills.

Key Details

  • The OOM (Out of Memory) Killer is a Linux kernel mechanism that terminates processes when memory is exhausted
  • OOM kills are logged in dmesg and kernel logs with the message 'Out of memory: Kill process'
  • The kernel selects which process to kill based on the oom_score (higher score = more likely to be killed)
  • Memory overcommit (vm.overcommit_memory) allows allocating more memory than physically available
  • cgroups memory limits can prevent specific services from consuming all available memory

Common Causes

  • Application memory leak gradually consuming all available RAM
  • Insufficient physical RAM for the workload (under-provisioned server)
  • No swap space configured as a safety buffer for memory pressure
  • Multiple memory-hungry processes competing for limited RAM

Steps

  1. 1Check OOM events: 'dmesg -T | grep -i oom' or 'journalctl -k | grep -i oom'
  2. 2Monitor memory usage in real-time: 'htop' or 'watch -n 1 free -h'
  3. 3Find memory-hungry processes: 'ps aux --sort=-%mem | head -20'
  4. 4Set memory limits with systemd: add MemoryMax=2G in the service file [Service] section
  5. 5Add swap as safety buffer: 'fallocate -l 4G /swapfile && mkswap /swapfile && swapon /swapfile'

Tags

oom-killermemoryleakout-of-memorycgroups

More in System

Frequently Asked Questions

It calculates an oom_score for each process based on memory usage, runtime, and privilege level. The process with the highest oom_score gets killed. You can protect critical processes by setting oom_score_adj to -1000 via /proc/PID/oom_score_adj.