NodeLoadHigh

The host’s long-term load average has been far above the number of CPUs it has, so work is queuing up.


Severity	warning
Source	node_exporter 1.x, `loadavg` and `cpu` collectors
Key metrics	`node_load15`, `node_cpu_seconds_total` (used to count CPUs)

What it means

On Linux, load average counts tasks that are running, waiting for a CPU, or in uninterruptible sleep (usually waiting for disk or NFS). The alert compares the 15-minute load to the host’s CPU count and fires when load has been a clear multiple of it for a long period, so a short spike will not page you.

High load is a symptom, not a diagnosis. It can mean “not enough CPU” or “everything is stuck waiting on storage”, and the fixes are different.

Common causes

CPU-bound workload larger than the host: traffic growth, a runaway process, a busy-looping thread.
Tasks stuck in D state on a slow disk or hung NFS mount.
Noisy neighbours or CPU steal on an oversubscribed VM.
Heavy swapping due to memory pressure.
Kubernetes node packed with pods that have no CPU limits.

First checks

Compare short and long load to see if it is getting better or worse:

node_load1{instance="<instance>"}
node_load15{instance="<instance>"}
count by (instance) (node_cpu_seconds_total{mode="idle", instance="<instance>"})

Break CPU time down by mode. High user/system means CPU contention; high iowait means blocked I/O; high steal means the hypervisor:
```
sum by (mode) (rate(node_cpu_seconds_total{instance="<instance>", mode!="idle"}[5m]))
```
On the host, find the top consumers:
```
top -b -n1 -o %CPU | head -20
```
Count processes in uninterruptible sleep and what they wait on:
```
ps -eo state,pid,wchan:32,cmd | awk '$1=="D"'
```

Check run queue and swap activity:

vmstat 2 5   # r = runnable, b = blocked, si/so = swap

Fixing it

For CPU contention, stop or throttle the runaway process, scale out, or move to a larger instance; on Kubernetes, set CPU requests and limits and rebalance pods. For D-state tasks, fix the underlying storage or remount the hung filesystem. For steal, move the VM or change instance type. For swapping, address memory first.

NodeHighCPUUsage: confirms the load is real CPU work.
NodeCPUHighIOWait: load driven by tasks waiting on I/O.
NodeDiskIOSaturation: the disk those tasks are waiting on.