Linux journald Disk Space Exhausted — What It Means & How to Fix It
About Linux journald Disk Space Exhausted
Fix systemd-journald consuming excessive disk space with system logs filling the /var partition and causing service failures. This guide covers everything you need to know about this topic, including common causes, step-by-step solutions, and answers to frequently asked questions.
Here are the key things to understand: systemd-journald stores system logs in binary format in /var/log/journal/ by default. Without size limits, journal logs can grow to consume all available space in /var. When /var fills up, services may fail to start, databases may crash, and the system becomes unstable. By default, journald limits logs to 10% of the partition or 4GB (whichever is smaller), but this may not be configured. Understanding these fundamentals will help you diagnose and resolve this issue more effectively.
The most common reasons this occurs include: Verbose service logging (debug mode left on) generating gigabytes of log data. Log rotation not configured or not aggressive enough for the disk size. Application crash loop generating repeated error messages filling the journal. SystemMaxUse not set in journald.conf, allowing logs to grow unconstrained. Identifying the root cause is the first step toward finding the right solution.
To resolve this, follow these recommended steps: Check journal disk usage: journalctl --disk-usage. Immediately free space: sudo journalctl --vacuum-size=500M to keep only the last 500MB of logs. Set permanent limits in /etc/systemd/journald.conf: SystemMaxUse=500M and RuntimeMaxUse=100M. Restart journald: sudo systemctl restart systemd-journald for the new limits to take effect. If these steps do not resolve the issue, consider consulting additional resources or a qualified professional.
This article is part of our Linux Error Codes collection on Error Codes Wiki. We provide comprehensive, up-to-date information to help you find solutions quickly.
Quick Answer
Will vacuuming delete important logs?
Vacuuming removes the oldest log entries first. Recent logs are preserved. If you need to keep old logs, export them first: journalctl --since '7 days ago' --output=export > backup.journal
Overview
Fix systemd-journald consuming excessive disk space with system logs filling the /var partition and causing service failures.
Key Details
- systemd-journald stores system logs in binary format in /var/log/journal/ by default
- Without size limits, journal logs can grow to consume all available space in /var
- When /var fills up, services may fail to start, databases may crash, and the system becomes unstable
- By default, journald limits logs to 10% of the partition or 4GB (whichever is smaller), but this may not be configured
Common Causes
- Verbose service logging (debug mode left on) generating gigabytes of log data
- Log rotation not configured or not aggressive enough for the disk size
- Application crash loop generating repeated error messages filling the journal
- SystemMaxUse not set in journald.conf, allowing logs to grow unconstrained
Steps
- 1Check journal disk usage: journalctl --disk-usage
- 2Immediately free space: sudo journalctl --vacuum-size=500M to keep only the last 500MB of logs
- 3Set permanent limits in /etc/systemd/journald.conf: SystemMaxUse=500M and RuntimeMaxUse=100M
- 4Restart journald: sudo systemctl restart systemd-journald for the new limits to take effect