Fractal Techware

Alert runbooks /

CoreDNSDown

Prometheus has not seen a single healthy CoreDNS target for a sustained period.

   
Severity critical
Source Prometheus scrape of CoreDNS (prometheus plugin, port 9153)
Key metric up for the CoreDNS job

What it means

The alert fires when there is no CoreDNS target reporting up == 1, either because every target is failing or because the targets have vanished from service discovery entirely. The second case matters: a mislabelled Service makes the targets disappear rather than go down.

If CoreDNS is really down, almost everything in the cluster breaks: service discovery, image pulls from in-cluster registries, and any outbound call by hostname. If only scraping is broken, DNS works but you have lost visibility into it.

Common causes

First checks

  1. Check pod status:
    kubectl -n kube-system get pods -l k8s-app=kube-dns -o wide
    kubectl -n kube-system logs -l k8s-app=kube-dns --tail=30
    
  2. Test DNS from a throwaway pod to see if resolution actually works:
    kubectl run dnstest --rm -it --restart=Never --image=busybox:1.36 -- nslookup kubernetes.default.svc.cluster.local
    
  3. Check that the metrics port is configured and exposed:
    kubectl -n kube-system get configmap coredns -o yaml | grep prometheus
    kubectl -n kube-system get endpoints kube-dns -o yaml | grep -A2 9153
    
  4. Look at what Prometheus discovers:
    up{job=~".*dns.*"}
    

    If the job name differs from what the alert expects, the rule needs adjusting rather than CoreDNS.

Fixing it

If pods are failing, read the logs: fix the Corefile in the coredns ConfigMap, or break the loop by pointing forward at a real upstream instead of a resolver that points back at CoreDNS. Then kubectl -n kube-system rollout restart deployment coredns. If only scraping is broken, restore port 9153 on the Service and fix the ServiceMonitor selector.