Running Kubernetes in production is less about the initial kubectl apply and more about the dozens of decisions that determine whether your cluster survives its first real incident. This checklist collects the configuration that separates a demo cluster from one that carries revenue.
Treat it as a baseline, not gospel. Every item below has a reason, and every reason has an exception. Read the reason, then decide.
Reliability and resilience
The first job of a production cluster is to stay up when things go wrong — and something always goes wrong.
- Set resource requests and limits on every workload. Unbounded pods are the most common cause of noisy-neighbor incidents.
- Configure PodDisruptionBudgets so voluntary disruptions (node drains, upgrades) never take your last healthy replica.
- Spread replicas across zones with topology spread constraints.
A cluster that has never been drained under load has not been tested. Drain a node during a load test before you trust your PDBs.
Health probes
Liveness and readiness probes are not optional, and they are not the same thing. A readiness probe controls traffic; a liveness probe controls restarts. Conflating them causes restart storms.
livenessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 10
periodSeconds: 10
readinessProbe:
httpGet:
path: /ready
port: 8080
periodSeconds: 5Security
Default Kubernetes is not secure Kubernetes. The gap between them is configuration you have to add.
- Enforce a Pod Security Standard (restricted) via admission control.
- Drop all Linux capabilities by default and run as non-root.
- Scope every ServiceAccount with least-privilege RBAC. The
cluster-adminbinding is not a starting point.
Observability
You cannot operate what you cannot see. Ship metrics, logs and traces before you need them, not during the incident.
| Signal | Tool | Retention baseline |
|---|---|---|
| Metrics | Prometheus | 15 days hot |
| Logs | Loki / OpenSearch | 30 days |
| Traces | OpenTelemetry + Tempo | 7 days |
Cost
Cost is a reliability concern in disguise: the cheapest cluster is one that is right-sized, and right-sizing requires the observability you set up above. Start with requests based on real usage, not guesses.
Work through the rest of the list before your next launch. The items you skip are the postmortems you will write later.