The Horizontal Pod Autoscaler, or HPA, is the built-in mechanism Kubernetes uses to adjust how many Pods a deployment runs based on current load. If CPU usage crosses a threshold, the HPA creates more Pods, and once load drops again, it removes them. Kubernetes autoscaling usually means exactly this, horizontal scaling through the HPA, not adding more compute power to a single Pod.
I run Kubernetes clusters in production and wrote the Kubernetes Practical Guide published by Rheinwerk Computing (2024). The HPA is standard equipment in almost every one of those clusters, mostly because it works without any cloud specific autoscaler and behaves the same on Hetzner as it would on a managed Kubernetes service. You can find all my articles on running Kubernetes clusters on the Kubernetes page.
The HPA needs two things before it does anything at all: a metrics source in the cluster and resource requests set on the container. Miss either one and it stays quiet, which is exactly where most setups fail.
What the Kubernetes HPA does
The HPA is a control loop that runs on a fixed interval, 15 seconds by default, comparing one or more metrics of a deployment against a target value. If measured CPU usage sits above the target, the HPA calculates the right new Pod count and raises the deployment's spec.replicas accordingly. When load drops, it lowers the count again, though more cautiously and with a built in delay, so it does not flip up and down on every short spike.
One thing matters for understanding this: the HPA never creates Pods directly. It only changes the desired count on the deployment, and the deployment does what it always does, using its ReplicaSet to create or remove Pods to match. That is why the HPA only works with objects that have a replica count, deployments, ReplicaSets and StatefulSets, not with individual Pods or DaemonSets.
Only what is built to scale can actually scale. An application that keeps state in memory or ties sessions to a specific Pod behaves unpredictably once there are several replicas. The HPA is therefore best suited to stateless services: APIs, web frontends, workers that process messages off a queue.
Prerequisite: metrics-server and requests set on containers
Before the HPA can scale anything, it needs a data source for current usage. That job belongs to metrics-server, a lightweight service that pulls resource usage per Pod from the kubelet and exposes it through the metrics API. Without a running metrics-server, kubectl top pods shows nothing, and the HPA cannot act at all.
The second prerequisite gets overlooked often: the HPA calculates CPU utilization as a percentage of the container's configured resources.requests.cpu. Without that value set, the HPA cannot form a meaningful percentage and reports the target as unknown. I cover how to set requests and limits properly in Kubernetes Requests and Limits Explained, which is the foundation every HPA setup builds on.
A first HPA definition based on CPU
Say you run a deployment called shop-api with CPU requests already set. The HPA in the manifest below keeps the application between two and eight replicas, targeting an average CPU utilization of 60 percent of the configured requests.
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
name: shop-api
namespace: shop
spec:
scaleTargetRef:
apiVersion: apps/v1
kind: Deployment
name: shop-api
minReplicas: 2
maxReplicas: 8
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
The scaleTargetRef field points at the deployment to scale. minReplicas and maxReplicas set the range the HPA is allowed to work in, preventing both a total outage under low load and unbounded growth if a load spike is actually a bug. The metrics section holds the actual rule, here the average CPU utilization relative to the request, though you can also target an absolute value in millicores.
Roll the manifest out with kubectl apply -f hpa.yaml and check its state with kubectl get hpa shop-api, and if something looks off, kubectl describe hpa shop-api. The events at the bottom of that output almost always tell you whether the HPA scaled successfully or why it did not, for example because the target metric is missing.
Multiple metrics and calmer scaling behavior
A single CPU metric covers plenty of cases, but the current HPA version supports much more. You can specify several metrics at once, say CPU and memory, and the HPA computes the required replica count for each one and picks the maximum. That way it always reacts to whichever bottleneck is more pressing. On top of that, the behavior block lets you control how aggressively or cautiously the HPA scales up and down.
metrics:
- type: Resource
resource:
name: cpu
target:
type: Utilization
averageUtilization: 60
- type: Resource
resource:
name: memory
target:
type: Utilization
averageUtilization: 75
behavior:
scaleDown:
stabilizationWindowSeconds: 300
scaleUp:
stabilizationWindowSeconds: 0
The stabilizationWindowSeconds setting on scale down makes the HPA look back over the last five minutes of measurements and go with the highest demand seen in that window, instead of reacting instantly to a brief dip. Those two values in the example are also the Kubernetes defaults, 300 seconds on scale down and 0 on scale up; writing the block out simply makes them visible so you can change them deliberately afterwards.
The block only gets interesting once you deliberately move away from those defaults. Out of the box, the Kubernetes documentation on the HPA allows removing up to 100 percent of the running replicas every 15 seconds on scale down, and on scale up it additionally allows four Pods every 15 seconds. The example below slows the teardown considerably and caps the ramp up at the same time:
behavior:
scaleDown:
stabilizationWindowSeconds: 600
policies:
- type: Percent
value: 50
periodSeconds: 60
scaleUp:
stabilizationWindowSeconds: 0
policies:
- type: Pods
value: 2
periodSeconds: 30
selectPolicy: Max
On scale down the HPA now looks back ten minutes and may remove at most half of the currently running Pods per minute. On scale up it still reacts without delay, but it climbs in steps of two Pods every 30 seconds, which gives an application with a longer startup time room to breathe. That direction is usually the right one: react fast when load genuinely rises, and back off cautiously. This fine tuning is the difference between an HPA that behaves calmly and one that adds and removes Pods on every small fluctuation.
Why the HPA does not scale: the most common mistakes
The same three causes keep showing up in my projects whenever an HPA appears to do nothing. First, metrics-server is missing or serving stale data, which shows up as <unknown> in the TARGETS column of kubectl get hpa. Second, the container has no requests set, so the HPA cannot compute a percentage even when metrics-server works fine. Third, maxReplicas is set too low, so the cluster wants to scale but hits its own ceiling, which is not obvious at first glance.
A fourth, subtler problem is flapping: the HPA scales up and down every minute or so because the threshold sits too close to normal load variation. A stabilization window that is too short, or an overly tight target like 90 percent utilization, both encourage this. Set the target conservatively instead, around 60 to 70 percent, and watch the replica count over a few days to confirm it stays calm.
HPA, VPA and Cluster Autoscaler compared
All three mechanisms solve different scaling problems, and in practice they get mixed up or combined unnecessarily. The table below sums up what each one is actually for.
| Trait |
Horizontal Pod Autoscaler |
Vertical Pod Autoscaler |
Cluster Autoscaler |
| What it scales |
number of Pods |
resources (CPU, memory) per Pod |
number of nodes |
| Reacts to |
load crossing a metric |
requests that are permanently sized wrong |
Pods that cannot be scheduled |
| Requires |
metrics-server, requests set |
its own controller and CRDs |
a cloud API or node provisioning |
| Typical use |
stateless services with variable load |
applications with unclear resource needs |
clusters with strongly variable total load |
I have used the Vertical Pod Autoscaler myself on a Prometheus setup, to adjust resource requests automatically without having to keep several replicas in sync by hand. What I found critical about it was that requests and limits in the Git repository no longer matched what actually ran in the cluster. For most applications I still prefer the HPA: it only changes the Pod count, which stays visible in the manifest, instead of quietly adjusting resources on individual Pods behind the scenes. You only need the Cluster Autoscaler on top of that if your cluster itself can add or remove nodes, which on a self operated Hetzner cluster typically happens manually or through infrastructure as code rather than automatically.
What comes next: scaling on your own metrics
CPU and memory are not always the right basis for a scaling decision. A worker that processes messages off a queue should scale on queue length rather than CPU usage, which stays low while messages pile up because the process is mostly waiting. For cases like that, the HPA supports so called custom metrics and external metrics, typically mirrored into the Kubernetes metrics API from Prometheus through an adapter.
A growing share of that job is now handled by KEDA, a project that ships ready made scalers for queues, topics and other external systems, and creates and manages a perfectly ordinary HPA underneath. I keep an eye on KEDA in my own projects without running it everywhere in production yet, mostly because plain CPU and memory scaling has been enough for most of my workloads so far. If you need to scale on your own metrics, though, it is worth a serious look before you build a custom adapter yourself.
Frequently asked questions
What is the difference between HPA and VPA?
The HPA changes how many Pods a deployment runs, the VPA changes the resource requests of individual Pods. The two can technically be combined, but in practice that easily leads to conflicting adjustments, so I usually pick one mechanism per application.
Why is my HPA not scaling?
The most common reasons are a missing metrics-server, containers without requests set, or maxReplicas already reached. Checking kubectl describe hpa almost always shows the exact reason in the events, faster than any guess.
Can the HPA scale on custom metrics?
Yes, through custom metrics and external metrics, usually fed in from Prometheus via an adapter, or through projects like KEDA that ship this integration ready to use. For plain CPU and memory scaling you do not need it, for queue based workers it is close to a requirement.
How fast does the HPA react to a load spike?
The control loop runs every 15 seconds by default, on top of whatever stabilization window you configure before it actually scales. For workloads that need sub second reaction times, the HPA is too slow, but for most web and API traffic that response time is plenty.
Do I also need a Cluster Autoscaler?
Only if you want your nodes themselves to grow automatically. On a self operated cluster without a cloud autoscaler, the node count stays fixed, and the HPA distributes load within existing capacity until the nodes are full.
Where to go next
An HPA is only as good as the requests it calculates against: read Kubernetes Requests and Limits Explained if you have not nailed that part down yet. To see what your Pods actually consume before you set thresholds, Kubernetes Monitoring with Prometheus will help. And if you want to know what a cluster with automatic scaling ends up costing, that is covered in Kubernetes Costs: What a Cluster Costs.
My suggestion: set up the HPA first with a single CPU metric and a conservative target, watch its scaling behavior for a week, and only add memory or custom metrics after that. The official walkthrough is in the Kubernetes documentation on the HPA, and details on custom metrics are in the documentation on the HPA with custom metrics.
In full detail, including the VPA and the Cluster Autoscaler, this is covered in chapter 8.3 of my Kubernetes Practical Guide (Rheinwerk Computing).
