KubeVersionMismatch
Kubernetes components in this cluster report more than one minor version.
| Severity | warning |
| Source | /metrics of API server, kubelets, scheduler, controller manager |
| Key metric | kubernetes_build_info (label git_version) |
What it means
Every Kubernetes binary exposes its version. The alert groups them by minor version (patch differences are ignored) and fires when more than one minor version has been running for a while.
During an upgrade this is expected. Left in place, it is a risk: Kubernetes only supports a limited version skew (kubelets may lag the API server by a few minors but must never be newer), and the next upgrade can push old nodes outside the supported range.
Common causes
- An upgrade is in progress or was abandoned halfway.
- A node pool or autoscaling group still uses an old node image or launch template.
- Nodes that were cordoned and forgotten during a previous upgrade.
- A managed control plane auto-upgraded while self-managed node groups did not.
First checks
- List versions per component and instance:
count by (git_version, job) (kubernetes_build_info) - Find the nodes that are behind:
kubectl get nodes -o custom-columns=NAME:.metadata.name,KUBELET:.status.nodeInfo.kubeletVersion --sort-by=.status.nodeInfo.kubeletVersion - Confirm the API server version:
kubectl version - Check control plane static pods on each control plane node:
kubectl -n kube-system get pods -l tier=control-plane -o custom-columns=NAME:.metadata.name,IMAGE:.spec.containers[0].image
Fixing it
Finish the upgrade in the supported order: control plane first, then nodes. For node pools, update the image or template and roll nodes with drain and replace. With kubeadm, run kubeadm upgrade node and upgrade the kubelet package on the remaining nodes. If an upgrade is deliberately staged, silence the alert with an expiry rather than ignoring it.
Related alerts
- KubeSchedulerDown: a failed control plane upgrade can leave components down.
- KubeControllerManagerDown: same risk during upgrades.
- KubeNodeCordoned: nodes left cordoned mid-upgrade.