A Kubernetes Pod is the smallest unit Kubernetes can schedule and deploy. A Pod groups one or more containers that land on the same node together and share an IP address and, where needed, volumes. Scheduling, creation and deletion always apply to the whole Pod, never to a single container inside it.
Once the Pod makes sense to you, the rest of Kubernetes gets a lot easier. Deployments, Jobs, DaemonSets and StatefulSets are, in the end, just different ways of creating Pods and keeping them alive.
One sentence up front, because it shortens every later debugging session: a Pod's phase and the state of its containers are two different things. Running means the Pod is running, not that the application inside it is healthy.
I run Kubernetes clusters in production and gave the Pod its own section in chapter 3 of my Kubernetes book published by Rheinwerk. Much of that is condensed here, together with the mistakes I have run into in my own operations and in client projects. You can find all my Kubernetes articles collected on the Kubernetes page.
Kubernetes Pod vs. container: what is the difference?
A container is a running process with its own file system, isolated by the namespaces and cgroups of the Linux kernel. A Pod is the wrapper Kubernetes puts around one or more of these containers. You cannot start a container directly in Kubernetes, you always describe a Pod.
The difference becomes tangible when you look at what the containers in a Pod have in common. They share one IP address and one network namespace, they reach each other via localhost, and for that reason they cannot bind the same port. They can mount the same volumes and exchange files through them. And they are scheduled to a node together and deleted together with the Pod. That does not put the individual container out of play: if one fails, the kubelet restarts it inside the same Pod according to the restart policy, leaving the others untouched (Kubernetes documentation on the Pod lifecycle).
Technically, this is made possible by a container that kubectl never shows you: the pause container. It starts first, holds the Pod's IP address and Linux namespaces, and lives as long as the Pod itself. Your application containers attach to those namespaces. This has a pleasant side effect: if your application container crashes and restarts, the Pod keeps its IP address.
|
Container |
Pod |
| What it is |
an isolated process from an image |
a group of one or more containers |
| Who manages it |
the container runtime (containerd, CRI-O) |
Kubernetes via the kubelet |
| Networking |
its own network settings per container |
one IP address for all containers in the Pod |
| Storage |
its own file system |
shared volumes possible |
| Placement |
wherever you start it |
all containers on the same node |
Whether containerd or CRI-O does the work underneath the Pod hardly matters to you as a developer: the kubelet talks to the runtime through the Container Runtime Interface (CRI), and the runtime handles image, file system and start. How the components fit together is covered in Kubernetes architecture: the components.
One container per Pod or several? Sidecar, ambassador, adapter
Most Pods I see in production contain exactly one application container. That is the right default: one container per Pod can be scaled independently, rolled out independently and kept cleanly decoupled. Several containers belong in one Pod only when they make no sense without each other.
Three questions help with the decision. First: do the containers have to share a resource, such as a file system one of them writes to and the other reads from? Then they belong together. Second: do they have different scaling needs? A web server and its database never scale in lockstep, so they get separate Pods. Third: could the containers run on different machines without any trouble? If yes, separate them.
For the helpers that really do belong in the same Pod, three patterns have become established:
| Pattern |
Job |
Typical case |
| Sidecar |
adds a function to the main application |
log shipper, metrics exporter |
| Ambassador |
proxy to the outside, the application only talks to localhost |
authentication against external APIs, TLS connections |
| Adapter |
reshapes incoming data to fit |
protocol or format conversion for legacy applications |
An example from my clusters: Prometheus expects metrics in its own format. If an application does not provide them, an exporter sits as a sidecar in the same Pod and translates. The application never notices, and the exporter is rolled out and stopped together with it. As soon as a helper limits the scaling of the main application, though, it belongs in a Pod of its own.
One thing matters for the design: all regular containers in a Pod start at the same time. You cannot rely on the sidecar being ready before the application. If you need a fixed order, you use init containers, more on those in a moment.
Kubernetes Pod YAML: your first manifest
Like every Kubernetes object, a Pod is described declaratively. The following manifest starts an Nginx web server and, next to it, a small sidecar that reads the web server's access log. Both containers mount the same volume, an emptyDir that is created with the Pod and disappears with it.
apiVersion: v1
kind: Pod
metadata:
name: my-nginx
labels:
app: my-nginx
spec:
containers:
- name: web
image: nginx
ports:
- containerPort: 80
volumeMounts:
- name: log-volume
mountPath: /var/log/nginx
- name: log-collector
image: busybox
command:
- sh
- -c
- "until [ -f /var/log/nginx/access.log ]; do sleep 1; done; tail -n +1 -f /var/log/nginx/access.log"
volumeMounts:
- name: log-volume
mountPath: /var/log/nginx
volumes:
- name: log-volume
emptyDir: {}
The fields that matter: apiVersion and kind tell Kubernetes which object you mean. metadata.name is the Pod's name within its namespace, and you will need the labels later for Services and Deployments. spec.containers lists the containers with image, ports and volume mounts, and spec.volumes defines the volumes the containers may mount. The sidecar waits until Nginx has created the log file and then streams it to its standard output. One detail about the Nginx image: it normally redirects its logs to standard output. Because the volume covers /var/log/nginx, a real file appears there again, and that is what the sidecar reads.
Save the manifest as my-nginx.yaml and apply it with kubectl:
kubectl apply -f my-nginx.yaml
kubectl get pod my-nginx
kubectl port-forward pod/my-nginx 8080:80
kubectl logs my-nginx -c log-collector
After the apply, get pod shows the columns READY, STATUS and RESTARTS. If you see 2/2 and Running, both containers are up. With the port forward you reach the web server at localhost:8080, and every request then shows up in the sidecar's log. More everyday commands are collected in kubectl commands: the most important ones for daily work.
An honest note: in production you almost never create Pods this directly. A single Pod is not replaced when its node fails and cannot be updated without downtime. That is what the Deployment is for, which creates and replaces Pods on your behalf. For learning, the bare Pod manifest is still the right starting point, because a Deployment contains exactly the same Pod specification inside.
Init containers: preparation before the start
The more prerequisites an application has, the more painful its startup becomes: the database is not reachable yet, a config file is missing, a directory has the wrong permissions. Instead of building these checks into the application, you use an init container.
Init containers run before the regular containers of the Pod, strictly one after another. Each one has to finish successfully before the next one starts, and only when all of them are done does your application start. Typical jobs are waiting for a database, preparing the file system, or a step that needs tools which have no business being in the application image. An init container can also get its own secrets and permissions that the main application does not need. That is least privilege in practice.
This is what the beginning of the spec looks like if you put an init container in front of the Pod above that creates the log file up front:
spec:
initContainers:
- name: init-logs
image: busybox
command: ["sh", "-c", "touch /var/log/nginx/access.log"]
volumeMounts:
- name: log-volume
mountPath: /var/log/nginx
containers:
- name: web
image: nginx
If an init container fails, Kubernetes restarts it by default until it succeeds. During that time the Pod stays in the Pending phase, and kubectl get pod shows you in the STATUS column which init container it is stuck on. In my projects, init containers are the tool of choice for waiting on database migrations before the application starts, instead of stuffing the application itself with retry logic.
Kubernetes Pod status: reading phases and container states
The STATUS column of kubectl get pod is your first look at the health of a Pod. Behind it are two layers: the phase of the Pod and the state of each container inside it. If you can tell the two apart, you find problems much faster.
| Phase |
Meaning |
| Pending |
The Pod exists, but not all containers are running yet. Kubernetes is looking for a node, pulling images, mounting volumes or waiting for init containers. |
| Running |
The Pod is bound to a node, all containers have been created and at least one is running or starting. |
| Succeeded |
All containers have terminated successfully and will not be restarted. Typical for Jobs. |
| Failed |
All containers have terminated and at least one of them failed, meaning it exited with a non-zero code. |
| Unknown |
Kubernetes cannot determine the state, usually because the node is no longer reachable. |
The phase alone does not tell the whole story: a Pod in Pending may have been failing to pull an image for minutes, a Pod in Running may be stuck in a loop of crashing and restarting. That is why the container states matter: Waiting (the container is waiting, for example for an image or a secret), Running (it is running) and Terminated (it has ended, successfully or not). What you see in the STATUS column as CrashLoopBackOff, ImagePullBackOff or Terminating are not phases but reasons derived from these states.
The most important command for troubleshooting is kubectl describe pod <name>. At the bottom of the output you find the events, further up the current state and the last state with exit code for each container. That last state is often the only thing that reveals why a container keeps dying. The full description of the lifecycle is in the Kubernetes documentation on the Pod lifecycle.
Stopping a Pod: restart policy and graceful shutdown
How Kubernetes deals with a terminated container is controlled by the restartPolicy at Pod level. It applies to all containers in the Pod, including init containers. Always is the default and restarts every terminated container, no matter why it stopped. OnFailure restarts only on a non-zero exit code, which suits batch work. Never never restarts and gives you full control.
In Kubernetes, Pods are stopped and recreated all the time: when scaling, when rolling out a new version, when moving to another node. Your application therefore has to shut down cleanly. To make that happen, the kubelet sends a SIGTERM to the process with PID 1 in each container and waits for the grace period, 30 seconds by default. If a preStop hook is defined, it runs before the signal. If the application does not respond in time, a SIGKILL follows and the container is stopped the hard way.
Two things your application has to guarantee for this: it runs as PID 1 in the container, and it catches the signal to finish open transactions, close database connections and exit with code 0. If it needs more than 30 seconds, raise terminationGracePeriodSeconds in the Pod spec. With kubectl delete pod <name> --grace-period=0 --force you can skip the wait in an emergency, but you lose any guarantee of a clean shutdown. The hooks are described in the documentation on container lifecycle hooks.
Frequently asked questions
What is a Pod in Kubernetes, in one sentence?
A Pod is the smallest unit Kubernetes can schedule and deploy: one or more containers that share an IP address and volumes and run together on one node. Everything else in Kubernetes, from the Deployment to the Job, ultimately creates and manages Pods.
Why does Kubernetes manage Pods rather than individual containers?
Because containers that belong together need shared resources: the same IP address, the same volumes, the same node. The Pod provides that shared environment and can be scheduled, scaled and replaced as a whole. An application with a sidecar, an init container and a volume would be nearly impossible to control as a loose collection of individual containers.
How many containers belong in one Pod?
Normally one. Several containers belong in the same Pod only if they have to share a resource, scale identically and could not sensibly run on different machines. A log shipper or a metrics exporter next to the application meets that bar, a database next to the web server does not.
What do I do when a Pod is stuck in Pending?
Run kubectl describe pod <name> first and read the events at the bottom. The reason is usually right there: no node with enough resources, an image that cannot be pulled, a volume that cannot be mounted, or an init container that never finishes. The Pending phase is only the symptom, the events show the cause.
Where to go next
The next logical step is the Deployment: it wraps your Pod specification, keeps the desired number of Pods alive and rolls out new versions without downtime. How that works is explained in Kubernetes Deployment: rollouts explained. For daily work with Pods, logs and shells inside containers, the overview in kubectl commands: the most important ones for daily work will help.
My suggestion for today: take the manifest above, swap the Nginx image for an application of your own and watch with kubectl describe how the Pod moves through its phases. The official concept chapter on Pods in the Kubernetes documentation adds the details there was no room for here.
In full detail with all examples, from the Dockerfile through the log collector to the restart policy, this is covered in chapter 3 of my book "Kubernetes: Practical Guide for Developers and DevOps Teams" (Rheinwerk Computing). You can find all the information about the book on my Kubernetes page.
