What happens when a container hits its CPU limit, and what happens when it hits its memory limit?

Two limits, two unrelated failure modes

Both limits are cgroup settings, but they behave nothing alike, and the single most useful thing you can say in an interview is why. CPU is a renewable resource measured over time, so a limit can be enforced by making you wait. Memory is not: once it is allocated there is nothing to wait for, so the only enforcement available is refusal or killing. Everything else follows from that asymmetry — CPU pressure makes a service slow, memory pressure makes it disappear.

CPU is a quota inside a repeating window

Under cgroup v2 the control is cpu.max, holding a quota and a period in microseconds. A period of 100000 with a quota of 200000 means the cgroup may consume 200 ms of CPU time in every 100 ms of wall clock, which is what "two CPUs" means. docker run --cpus=1.5 writes 150000 100000.

# Inside a cgroup v2 container.
cat /sys/fs/cgroup/cpu.max     # e.g. "150000 100000"
cat /sys/fs/cgroup/cpu.stat    # nr_throttled and throttled_usec live here

When the cgroup exhausts its quota, every runnable task in it is descheduled until the period rolls over, regardless of how idle the machine is. This is why a service can show modest average CPU utilisation and still be throttled badly: a request that needs 40 ms of CPU across four threads consumes 160 ms of quota, so with a 100 ms period and a one-CPU limit it stalls partway through and resumes only after the window resets. Average utilisation hides that entirely, which is why nr_throttled and throttled_usec from cpu.stat are the metrics that matter, not CPU percentage.

Note that --cpu-shares, or cpu.weight in cgroup v2, is a different thing: a relative weighting that only takes effect when the CPU is contended, with no ceiling. Confusing a weight with a limit is a common slip.

Memory has no brake, only a cliff

The limit is memory.max. When a charge to the cgroup would exceed it, the kernel first tries to reclaim: writing back and dropping clean page cache, swapping if swap is available to that cgroup. If reclaim cannot free enough, the cgroup's OOM killer selects a process within the cgroup and sends SIGKILL. There is no signal you can catch and no chance to shut down cleanly, which is why "handle out-of-memory gracefully" is not a container-level option.

What you observe afterwards is exit code 137, which is 128 plus 9 for SIGKILL, and an OOMKilled reason recorded by the runtime. It is worth saying explicitly that 137 does not always mean the memory limit: any SIGKILL produces it, including a shutdown timeout being exceeded. Check the recorded reason and the kernel log rather than inferring from the code alone.

Cgroup v2 does offer memory.high, a throttling threshold that stalls the allocator to apply back pressure before the hard limit is reached. It is genuinely useful and most orchestrators do not expose it, so in practice you get the cliff.

Page cache counts, and this is where the numbers mislead

Every page the container reads or writes through the filesystem is charged to its cgroup as page cache. A service streaming a large file will show memory usage climbing towards its limit and then sitting there, because the kernel has no reason to reclaim cache until it needs the room. That looks exactly like a leak on a graph and is not one.

The consequence for monitoring is concrete: alert on the working set — usage minus the reclaimable inactive file cache, which is what container_memory_working_set_bytes reports — rather than on total usage. Alerting on total usage in a cache-heavy service produces a permanent false alarm, and teams then raise the threshold until the alert can no longer catch a real leak either.

The runtime that does not know where it is

A limit written into a cgroup file is not automatically visible to a process that asks the operating system how big the machine is, and this is where a lot of production capacity gets misconfigured.

The JVM handles it. Container awareness is on by default from JDK 10 and was backported to 8u191, so the JVM reads the cgroup memory limit when sizing the heap and reads the CPU limit when reporting availableProcessors. Use -XX:MaxRAMPercentage rather than a fixed -Xmx so the heap tracks the limit, and leave real headroom, because the heap is only part of the process — metaspace, thread stacks, code cache and direct buffers are all charged to the same cgroup.

Elsewhere it is patchier. nproc and libuv's uv_available_parallelism, which backs Node's os.availableParallelism(), respect the CPU affinity mask but not the quota, so a container limited to one CPU on a 64-core host still reports the number of CPUs it is allowed to be scheduled on rather than the fraction it may use. Pinning with cpuset changes that answer; a cpu.max quota does not. Go's runtime set GOMAXPROCS from NumCPU for the same reason, until Go 1.25 made it cgroup-aware; before that version the automaxprocs library was the standard workaround. A thread pool sized from a host-wide CPU count inside a quota-limited container is a throttling machine.

Choosing the numbers

Derive them, and say so. Take the observed working set at peak over a representative period, add headroom for the parts of the process that are not the heap, and set the memory limit there — a limit below peak working set is an outage on a schedule. For CPU, size the request from observed steady-state usage so the scheduler places the workload honestly, then decide on a limit separately: for a latency-sensitive service, a limit close to the request buys predictability at the cost of throttling on every burst, which is often the wrong trade. Setting no CPU limit at all is a defensible choice when requests are set correctly and the node is not oversubscribed.