Fractal Techware

Alert runbooks /

KubeAggregatedAPIDown

An API group served by an extension API server (for example metrics.k8s.io) is failing its availability checks.

   
Severity warning
Source kube-apiserver /metrics (aggregation layer)
Key metric aggregator_unavailable_apiservice (labels name, namespace)

What it means

The API server proxies some API groups to other services, registered as APIService objects. It continuously checks whether each backend is reachable. The alert fires when a given APIService has been unavailable for a meaningful part of the recent window, not just a single blip.

Impact depends on the API: a down metrics.k8s.io stops kubectl top and every CPU/memory based HPA. It can also make discovery fail, so kubectl prints “unable to retrieve the complete list of server APIs”, and namespace deletion hangs in Terminating.

Common causes

First checks

  1. Find unavailable APIServices and the reason:
    kubectl get apiservices | grep -v True
    kubectl describe apiservice <name>
    
  2. See which ones are flapping over time:
    max by (name, namespace) (aggregator_unavailable_apiservice) == 1
    
  3. Check the backing Service and pods:
    kubectl -n <namespace> get svc,endpoints <service>
    kubectl -n <namespace> get pods -o wide
    kubectl -n <namespace> logs deploy/<backend> --tail=100
    
  4. Look at the API server’s view of the proxy errors:
    kubectl -n kube-system logs -l component=kube-apiserver --tail=300 | grep -i "<name>"
    

Fixing it

Restart or fix the backend deployment and make sure its Service has endpoints. If the control plane cannot reach the pod network (private GKE, EKS with restrictive security groups), open the webhook/aggregation port from the control plane. Delete APIServices left behind by removed software. Fix the caBundle if TLS verification fails.