Fractal Techware

Alert runbooks /

KubePodDisruptionBudgetBlocksEviction

A PodDisruptionBudget has allowed zero voluntary disruptions for a long time, so any kubectl drain, node upgrade or cluster autoscaler scale-down touching its pods will stall.

   
Severity info
Source kube-state-metrics v2.x
Key metrics kube_poddisruptionbudget_status_pod_disruptions_allowed, kube_poddisruptionbudget_status_expected_pods

What it means

disruptionsAllowed is how many matching pods may be evicted right now. When it stays at zero for a PDB that actually covers pods, the eviction API refuses every request with a “would violate the pod’s disruption budget” error. The alert is informational: nothing is down, but maintenance is blocked.

It tends to surface at the worst time, during a node pool upgrade that hangs for hours.

Common causes

First checks

  1. List PDBs with no disruptions allowed:
    kubectl get pdb -A | awk 'NR==1 || $5==0'
    
    kube_poddisruptionbudget_status_pod_disruptions_allowed == 0
    
  2. Compare the budget with the replica count:
    kubectl -n <ns> get pdb <pdb> -o yaml | grep -E 'minAvailable|maxUnavailable|currentHealthy|desiredHealthy|expectedPods'
    kubectl -n <ns> get deploy,statefulset -l <selector>
    
  3. Check whether a drain is already stuck on it:
    kubectl get nodes | grep SchedulingDisabled
    
  4. Look for unhealthy pods consuming the budget:
    kubectl -n <ns> get pods -l <selector>
    

Fixing it

Give the workload room: run at least two replicas, or switch to maxUnavailable: 1. If a pod is unhealthy, fix it first. On Kubernetes 1.31+ (beta since 1.27), unhealthyPodEvictionPolicy: AlwaysAllow lets drains evict pods that are already not Ready. For a truly single-instance workload, accept the PDB and plan a manual failover before draining, rather than deleting the PDB mid-upgrade.