A Kubernetes liveness probe continuously checks whether an application still works correctly and lets Kubernetes restart the container once the check keeps failing. A readiness probe, on the other hand, does not decide on a restart at all, it decides whether a Pod is allowed to receive traffic from its Service. Together they form the basis for self healing in Kubernetes, so nobody has to be woken up at night to restart a Pod by hand.
I run Kubernetes clusters in production and gave probes their own section in chapter eight of my Kubernetes book published by Rheinwerk Computing. Misconfigured probes are one of the most common causes of outages I have seen in client clusters, usually because a probe checks too much or checks too early. You can find all my Kubernetes articles collected on the Kubernetes page.
Kubernetes liveness probe, readiness probe and startup probe: three questions to your Pod
The three probe types answer different questions and trigger different actions. The liveness probe asks: is the application still working? If it fails over a defined period, Kubernetes restarts the container, governed by the Pod's restart policy.
The readiness probe asks: is the application ready to handle requests right now? If it fails, the matching Service removes the Pod from load balancing, without restarting the container. That is useful not only during startup but also when a running application is briefly overloaded and would rather not receive new requests for a while.
The startup probe asks: has the container even finished starting up? Until it succeeds, the liveness and readiness probes are disabled. That protects applications with a long startup from being flagged as stuck and restarted too early. In my experience it is used almost exclusively for legacy applications with a startup that takes several minutes, modern applications rarely need it.
Mechanisms: HTTP, TCP, exec and gRPC
Every probe needs a way to ask your application about its state. The most common is the HTTP probe: Kubernetes sends a GET request to a port and path you choose, and any status code between 200 and 399 counts as success. That gives you the most freedom, because you design the health endpoint yourself.
If your application cannot speak HTTP, such as a database or a queue, a TCP probe simply checks whether a socket can be opened on the given port. An exec probe instead runs a command inside the container and evaluates its exit code, handy for applications that only expose their state through a local file or a command line tool. More recent Kubernetes releases also offer a gRPC probe, for services that already speak gRPC anyway.
The parameters that decide between real alerts and false alarms
A probe is only as good as its timing values. Five parameters determine how strict and how patient Kubernetes is when checking:
| Parameter |
Meaning |
Default |
| initialDelaySeconds |
delay before the first probe runs at all |
0 |
| periodSeconds |
interval between two checks |
10 |
| timeoutSeconds |
how long a single check waits for a response |
1 |
| failureThreshold |
how many consecutive failures are needed before the probe counts as failed |
3 |
| successThreshold |
how many consecutive successes are needed before a readiness probe counts as successful again (fixed at 1 for liveness and startup) |
1 |
A timeoutSeconds value that is set too low is one of the most common causes of a readiness probe timeout, especially when your application responds a bit slower than usual under load. It has proven useful to point liveness and readiness probes at the same endpoint, but run the readiness probe with a shorter period or a smaller failureThreshold: that way a Pod switches to not ready and gets removed from load balancing before the stricter liveness probe restarts it. That is practice, not a Kubernetes rule, and there is one case where separate endpoints are clearly safer: as soon as the endpoint also checks dependencies such as the database, the liveness probe must not use it. Otherwise a database that is briefly unreachable makes Kubernetes restart every Pod at once even though the application itself is healthy. In that case the liveness probe gets an endpoint that only checks its own process, and the readiness probe gets the one with the dependencies.
Building health endpoints the right way
The biggest design mistake with probes is checking too much. A liveness probe should only answer whether the process itself still works, not whether a database or an external service is reachable. If your liveness probe also checks a dependency that is having problems of its own, Kubernetes restarts a whole row of otherwise healthy containers and turns one outage into a cascade.
The opposite is true for the readiness probe: here you may and should check everything your application needs to process requests, such as an open database connection or a warmed up cache. If you can build your application so it exits with an error code when it hits a problem it cannot recover from, you may not even need a liveness probe at that point and can leave the reaction to the Pod's restart policy.
Here is a simple manifest with all three probe types on different endpoints:
readinessProbe:
httpGet:
path: /readyz
port: 8080
initialDelaySeconds: 5
periodSeconds: 5
timeoutSeconds: 2
livenessProbe:
httpGet:
path: /healthz
port: 8080
initialDelaySeconds: 15
periodSeconds: 10
failureThreshold: 3
startupProbe:
httpGet:
path: /healthz
port: 8080
failureThreshold: 30
periodSeconds: 10
Here the startup probe allows up to five minutes for startup before the liveness probe even starts counting. /readyz checks the application's dependencies, /healthz checks only the process itself, a deliberately simple endpoint with no outside calls.
The three mistakes that cause outages
The first and most common mistake is a liveness probe that is too strict: short network hiccups trigger restarts even though the application is actually healthy. This often shows up as "Liveness probe failed: connection refused" in the events, while the application itself was still running but could not respond in time under brief load. A higher failureThreshold or a larger periodSeconds eases that without making the probe useless.
The second mistake is a liveness probe that checks dependencies along the way, as described in the section on health endpoints. The third mistake is a missing startup probe on applications with a slow start: without one, the regular liveness probe already counts during initialization and restarts the container before it has even finished starting up, leaving the Pod stuck in a restart loop.
Frequently asked questions
What is the difference between a liveness and a readiness probe?
The liveness probe decides whether Kubernetes restarts a container. The readiness probe decides whether a Pod receives traffic from the Service, without anything being restarted. Both can run against the same endpoint, but with different strictness.
When do I need a startup probe?
Mainly for applications with a long, unpredictable startup, such as legacy software that takes several minutes to come up. Modern, fast starting applications usually get by without a startup probe.
Why do I get "Liveness probe failed: connection refused"?
Usually because the application briefly does not respond at the moment of the check, for example due to a network hiccup, a short garbage collection pause, or because it has not fully started yet. A initialDelaySeconds that is too low or a failureThreshold that is too strict makes the problem worse.
What do I do about a readiness probe timeout?
Check timeoutSeconds first: if your application responds slower than this value under load, the probe already counts as failed even though the application is simply busy. A more realistic timeout, measured under real load, usually fixes this without any other changes.
Is the liveness probe allowed to check the database?
No, not directly. If the liveness probe checks a dependency that is having problems of its own, Kubernetes restarts healthy containers without fixing anything. Dependencies belong in the readiness probe, the liveness probe should only check the process itself.
Where to go next
Probes are closely tied to the resources you give your Pod: a throttled container produces the same symptoms as a misconfigured probe, and how to find the right values is covered in Kubernetes Requests and Limits Explained. If a Pod ends up in a restart loop despite correctly configured probes, the systematic troubleshooting in CrashLoopBackOff: Debugging Pods will help. And how probes decide the moment traffic switches over during a rollout is covered in Kubernetes Deployment: Rollouts Explained.
My suggestion for today: open one of your Deployments and check whether the liveness and readiness probes share the same endpoint with a different failureThreshold, or whether a probe accidentally checks dependencies it should not. The full parameter list is in the Kubernetes documentation on probes.
In full detail, with a sample application, a Docker image and a test setup, this is covered in chapter 8 of my Kubernetes Practical Guide (Rheinwerk Computing).
