The OOM Killer Ate Your Agent: Memory Limits for Self-Hosted AI Boxes

Your agent was running fine at midnight. In the morning, the brief never arrived, the webhook listener is gone, and the logs just stop. No error, no stack trace, no “shutting down.” The process simply is not there anymore.

On a small home agent box, one of the most common reasons is the Linux out-of-memory killer, usually called the OOM killer. When the system runs out of memory, the kernel picks a process and kills it on the spot so the machine as a whole can keep running. It does not ask, it does not let the process clean up, and it does not send you a message. It writes one line to the kernel log and moves on.

The process it picks is the one using the most memory, adjusted by a few rules. On an agent box, that is very often the agent runtime itself, a headless browser, or a local model server. In other words, the thing you actually needed.

This post walks through how to confirm the OOM killer was the cause, how to find out what was eating the memory, and how to set limits so the right process dies first and comes back on its own.

Why agent boxes run out of memory

A Raspberry Pi or a cheap mini PC with 4 or 8 GB of RAM handles one agent comfortably. Trouble starts as the box picks up more jobs, which it always does:

  • Headless browsers. A browser automation step can use several hundred megabytes per tab and often more. A job that forgets to close the browser leaves it running, and the next run opens another one.
  • Local models. A local model server loads the model into memory. A 7B model can take several gigabytes on its own, which is most of a small box.
  • Build steps. Static site builds, image processing, and dependency installs spike memory for a few minutes and then drop. If one lands at the same time as everything else, the box tips over.
  • Slow leaks. A long-running Node or Python process that grows a little every hour is fine for a week and dead by the second.
  • Overlapping runs. A job that runs every 15 minutes but sometimes takes 20 stacks copies of itself. If you have not read it yet, the stale lock file guide covers how to stop overlaps without creating a new silent failure.

Swap is supposed to soften this, but many small boxes have very little of it, or none. Some Raspberry Pi OS versions ship with a small swap file, and some use compressed RAM swap instead. Either way, once both RAM and swap are gone, the OOM killer is the next step.

Step 1: Confirm it was the OOM killer

Before you change anything, prove the cause. Over SSH, check the kernel log for the current boot:

journalctl -k --no-pager | grep -iE 'out of memory|killed process|oom-kill'

If the box has rebooted since, look at the previous boot as well:

journalctl -k -b -1 --no-pager | grep -iE 'out of memory|killed process|oom-kill'

A hit looks something like Out of memory: Killed process 1234 (node) with a memory total next to it. The name in parentheses is the process that died. The timestamp tells you when.

If your agent runs as a systemd service, systemd also records why it stopped:

systemctl show -p Result -p NRestarts your-agent.service

Result=oom-kill is a direct answer. A climbing NRestarts count means it has been happening more than once and something was quietly restarting it.

If none of these show an OOM event, the OOM killer is not your problem, and you can move on to other causes. That is a useful result too.

Step 2: Find out who is eating the memory

Look at the big picture first:

free -h
swapon --show

free shows total, used, and available memory. The number that matters is available, not free. swapon --show tells you whether there is any swap at all and how big it is.

Then list the top memory users:

ps -eo pid,rss,etime,comm --sort=-rss | head -15

rss is resident memory in kilobytes. etime is how long the process has been running. A browser process that has been alive for three days when the job that started it runs for two minutes is a leak you can fix today.

One snapshot is not enough, because the spike that killed your agent might only last a few minutes. A tiny cron entry that logs the top five memory users every five minutes gives you a record to look back at:

*/5 * * * * (date -u; ps -eo rss,comm --sort=-rss | head -6) >> /var/log/mem-top.log 2>&1

Let it run for a day or two, then line its timestamps up against the OOM events from step 1. The culprit is usually obvious: a job that starts at the same time every night, or a process whose number only ever goes up. Remember to rotate or delete that log once you are done, so it does not become its own disk problem.

Step 3: Decide what should die first

The OOM killer will kill something. Your job is to make sure it kills the thing that matters least and is easiest to restart.

A simple ranking for most agent boxes:

  1. Keep alive: the agent runtime, the scheduler, SSH, and anything that receives webhooks or payments.
  2. Okay to kill: headless browsers, one-off build steps, image processing, a local model server you can reload.

Systemd lets you express this with OOMScoreAdjust. Lower numbers make a process less likely to be picked, higher numbers make it more likely. In the agent’s service file:

[Service]
OOMScoreAdjust=-500

And for a heavy, disposable job:

[Service]
OOMScoreAdjust=500

Do not set the agent to -1000. That makes it effectively unkillable, and if the agent itself is the one leaking, the kernel will kill everything else on the box to keep it alive, including SSH.

Step 4: Put a ceiling on the heavy jobs

Ranking helps, but a cap is better. A cap stops one job from taking the whole box in the first place. Systemd can limit memory per service:

[Service]
MemoryHigh=1G
MemoryMax=1500M

MemoryHigh is a soft limit. When the service goes over it, the kernel slows it down and reclaims memory from it aggressively. MemoryMax is the hard limit. If the service goes past that, only that service gets killed, not something random elsewhere on the box.

If you run jobs in Docker, the same idea is a flag on the container:

docker run --memory=1g --memory-swap=1g your-job-image

For a Node-based job, you can also tell Node itself to stay smaller, so it collects garbage before it hits your cap:

node --max-old-space-size=768 job.js

Start with caps that are roughly double what the job normally uses according to your log from step 2. Tighten later if you need to.

Step 5: Make sure the survivor comes back

Even with good limits, something will get killed eventually. The final piece is making sure it restarts on its own and that you hear about it.

In the agent’s service file:

[Service]
Restart=on-failure
RestartSec=10

That brings the agent back after an OOM kill. On its own, though, it hides the problem: the agent dies and restarts every night and you never find out. Add one line to your daily heartbeat or shift report that includes the restart count:

systemctl show -p NRestarts --value your-agent.service

If that number moves, you want to know. The stalled automation alarm post covers how to alert from outside the box when a job stops producing output, which also catches the case where the restart itself fails.

Then do one real test. Run the reboot test after you add the limits, so you know the new service settings load cleanly from a cold start.

Add swap, but do not lean on it

A modest amount of swap gives the box breathing room for short spikes, like a build step that peaks for 90 seconds. It is worth having. On an SD card, though, heavy swapping is slow and wears the card, so treat swap as a cushion, not extra RAM. If your mem-top log shows swap in constant use, the real fix is fewer concurrent jobs or a bigger box, not a bigger swap file.

That is also the moment to stop adding work. If the box keeps running near the edge, the load governor guide covers how to decide what runs when before you add another agent.

The short version

  1. Check journalctl -k for Killed process lines to confirm the OOM killer did it.
  2. Use free -h, swapon --show, and ps --sort=-rss to see who is using the memory, and log it every five minutes for a day.
  3. Rank processes: protect the agent with a negative OOMScoreAdjust, mark disposable jobs with a positive one, and never use -1000.
  4. Cap heavy jobs with MemoryHigh and MemoryMax, Docker --memory, or Node’s heap flag.
  5. Set Restart=on-failure on the agent and report the restart count in your daily heartbeat.
  6. Keep a small swap cushion and stop adding jobs when the box lives near the edge.

None of this is exciting. It just turns “the agent vanished overnight” into “the browser job hit its cap at 2:14 a.m. and the agent kept working,” which is the kind of morning you want.

Have your agent check its own memory

If you want your agent to run this check weekly and only speak up when it finds an OOM event or a climbing restart count, start with MarketMai’s free AI prompt pack and adapt one into a “check the kernel log for OOM kills, list the top memory users, and report restart counts” task. When you are ready to run more of your business on agents, the MarketMai Ultimate Bundle puts 13 prompt, automation, and agent-operations products in one $49 download.

More from the build log

Suggested

Want the full MarketMai stack?

Get the core MarketMai guides and operator playbooks in one premium bundle for $49.

View Bundle