Kubernetes HPA: Production Autoscaling Guide

Treat Kubernetes HPA as a delayed feedback loop

A HorizontalPodAutoscaler periodically reads metrics, calculates a desired replica count, and updates a scalable target such as a Deployment or StatefulSet. The workload controller creates or removes Pods; the scheduler then has to place them; the application must start, become ready, and receive traffic. The current Kubernetes HPA documentation describes the controller, supported metrics APIs, and calculation details.

At its simplest, the controller uses a ratio:

desiredReplicas = ceil(
  currentReplicas × currentMetricValue / desiredMetricValue
)

The real algorithm is deliberately more conservative. It accounts for tolerance, missing metrics, unready Pods, multiple metrics, and recent recommendations. This means a dashboard calculation may not match the final decision exactly. Diagnose the HPA from its status, conditions, events, and input metrics rather than assuming the ratio is the entire system.

Use horizontal scaling only when replicas add capacity

HPA works best for stateless HTTP services, independent consumers, and workers whose load can be divided across interchangeable replicas. It cannot make a serialized critical section parallel, increase a database connection ceiling, partition one hot key, or repair an application that slows down because a dependency is failing.

Before adding HPA, run a fixed-replica load test. Confirm that throughput rises as replicas increase and that tail latency, error rate, database saturation, and queue age improve. If doubling Pods barely changes useful work, remove the bottleneck before automating replica count.

Choose a metric that represents the constrained work

CPU is a useful first signal when CPU consumption grows with request or job load. With a utilization target, Kubernetes divides measured CPU usage by each container's CPU request. A target of 65% is therefore relative to declared requests, not 65% of a node and not a universal performance threshold.

Memory is often a poor reactive signal for runtimes that retain heaps or caches after traffic falls. Scaling out may duplicate that memory rather than relieve it. Measure whether per-Pod memory actually responds to load and returns after the load leaves before using it for scale-down.

The Kubernetes resource metrics pipeline exposes the basic CPU and memory data used for autoscaling, commonly through Metrics Server. Custom and external metrics require an adapter and a separately operated monitoring pipeline. Use low-cardinality selectors, document units, and verify what happens when data is late, duplicated, or unavailable.

Understand multiple-metric behavior

With autoscaling/v2, the controller calculates a proposal for each configured metric and uses the largest desired replica count. If one metric cannot be converted and the available metrics recommend only scaling down, scale-down is skipped. This bias protects capacity, but a broken metric can quietly hold a fleet large. Alert on HPA conditions and metric freshness, not only replica count.

Calibrate resource requests before CPU autoscaling

For CPU utilization, the request is part of the control equation. If relevant containers lack CPU requests, utilization for the Pod is undefined and the HPA cannot act on that metric. If requests are much too low, ordinary traffic looks permanently saturated; if too high, the application may suffer before the target is reached.

1. Profile one Pod. Record throughput, latency, errors, CPU, memory, event-loop or runtime saturation, and dependency usage across a gradual load ramp.
2. Find the safe envelope. Identify the operating region before latency or errors rise sharply, with headroom for normal variance.
3. Set requests from evidence. Requests influence scheduling as well as utilization. Re-test after runtime, instance type, sidecar, or application changes.
4. Separate containers when needed. A busy sidecar can distort Pod-level resource utilization. Consider a ContainerResource metric only after checking cluster support and ensuring the named application container exists consistently across revisions.

Do not tune HPA and resource requests independently. Reducing a CPU request can trigger more replicas without changing demand; raising it can suppress scaling while reserving more node capacity. Review both changes as one capacity-policy update.

Build an autoscaling/v2 manifest with bounded behavior

This example combines a Deployment contract with an HPA. The numbers are illustrative, not defaults to copy. Replace them with values derived from load tests, startup measurements, failure-domain requirements, cluster capacity, and dependency budgets. Use an immutable image digest produced by your own delivery system.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: orders-api
spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0
      maxSurge: 1
  selector:
    matchLabels:
      app: orders-api
  template:
    metadata:
      labels:
        app: orders-api
    spec:
      terminationGracePeriodSeconds: 45
      containers:
        - name: api
          image: registry.example.com/orders-api@sha256:REPLACE_ME
          ports:
            - name: http
              containerPort: 3000
          resources:
            requests:
              cpu: 500m
              memory: 512Mi
            limits:
              memory: 1Gi
          startupProbe:
            httpGet:
              path: /startup
              port: http
            periodSeconds: 5
            failureThreshold: 24
          readinessProbe:
            httpGet:
              path: /ready
              port: http
            periodSeconds: 5
            timeoutSeconds: 2
            failureThreshold: 2
---
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: orders-api
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: orders-api
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 65
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies:
        - type: Percent
          value: 100
          periodSeconds: 60
        - type: Pods
          value: 4
          periodSeconds: 60
      selectPolicy: Max
    scaleDown:
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 25
          periodSeconds: 60
      selectPolicy: Max

The Deployment deliberately omits a lasting spec.replicas declaration so a GitOps or repeated apply workflow does not continually fight the HPA. Decide how your deployment tool transfers replica ownership, especially during initial installation and rollback. The HPA then maintains a floor of three and a ceiling of thirty while targeting average CPU utilization.

A startup probe shields a slow initialization phase from ordinary readiness and liveness evaluation. A readiness probe prevents traffic from reaching a Pod before it can serve. Kubernetes' current probe documentation explains these distinct semantics. Application-side health contracts are covered in our readiness and liveness guide.

Example external queue metric

For a worker, CPU may stay low while backlog grows, especially when jobs wait on remote I/O. An external metric can express a desired backlog per replica:

metrics:
  - type: External
    external:
      metric:
        name: queue_messages_ready
        selector:
          matchLabels:
            queue: "email-delivery"
      target:
        type: AverageValue
        averageValue: "20"

This fragment requires an external metrics adapter that exposes exactly that name and selector. The value of twenty is only a syntax example. Derive a target from job arrival rate, service time, concurrency, and the maximum acceptable queue age. Secure access to the external metrics API and keep tenant or user identifiers out of selectors.

Scale up for recovery; scale down for stability

The behavior block applies velocity policies after the metric calculation. In the example, scale-up may add four Pods or double the fleet during a policy period, whichever allows the larger change. Scale-down removes no more than 25% during its policy period and considers recent recommendations for five minutes before reducing capacity.

The current HorizontalPodAutoscaler v2 API reference defines policies, selection, and stabilization fields. A stabilization window does not delay every action by a fixed timer; it selects a safer recommendation from the recorded window. Test observed behavior on the actual cluster version and controller configuration.

• Use faster scale-up when overload is expensive and dependencies can accept the extra concurrency.
• Use slower scale-down when traffic is bursty, Pod startup is costly, or cache warm-up matters.
• Raise minReplicas when recovery would otherwise begin from too little warm capacity.
• Set maxReplicas from a proven system limit, then alert when ScalingLimited shows the ceiling is active.

A rollout changes the measured population while HPA changes its desired size. Confirm surge capacity, readiness behavior, and mixed-version performance together. Our Kubernetes deployment strategies guide covers rolling, blue-green, and canary release trade-offs.

Budget the systems behind every new Pod

HPA changes desired replicas; it does not guarantee nodes exist to schedule them. Pending Pods provide no capacity. Combine workload autoscaling with deliberate node capacity, quotas, topology rules, image distribution, and—where appropriate—node autoscaling. Kubernetes documents how workload and node autoscaling interact: new unschedulable Pods can prompt node provisioning, which adds another delay to the response path.

Calculate worst-case aggregate demand at maxReplicas:

• database connections and concurrent queries;
• cache, message broker, and object-store connections;
• third-party API concurrency and rate limits;
• egress bandwidth, load balancer targets, addresses, and volumes;
• node CPU, memory, ephemeral storage, and topology capacity;
• logging, tracing, and metrics volume during an incident.

For example, a thirty-Pod maximum with an unreviewed per-process database pool can turn a recoverable traffic spike into database saturation. Set a global connection budget, reserve capacity for migrations and operations, and divide the remainder across the maximum application fleet. Our Node.js connection-pooling guide explains that calculation.

Validate the complete demand-to-capacity path

Apply manifests in a representative environment, confirm the metrics API, and watch the controller, Pods, and resource usage together:

kubectl get --raw /apis/metrics.k8s.io/v1beta1/nodes
kubectl get hpa orders-api --watch
kubectl describe hpa orders-api
kubectl get pods -l app=orders-api -w
kubectl top pods -l app=orders-api

The raw metrics endpoint path can vary with the API version available in a cluster; discover registered APIs rather than hard-coding it into automation. kubectl describe hpa exposes conditions such as AbleToScale, ScalingActive, and ScalingLimited, along with recent events.

Then test failure paths: missing metrics, an unschedulable Pod, a slow image pull, failing readiness, node provisioning delay, database saturation, and a deployment during high traffic. Scale-down terminates real processes; handlers need graceful shutdown and sufficient termination time so requests and claimed jobs are not abandoned. See our graceful shutdown guide for the application lifecycle.

Record the highest safe throughput per ready Pod, time to first new ready Pod, time to reach useful capacity, maximum customer-visible latency and error rate during the transition, pending-Pod duration, metric age, scale events, and downstream saturation. These observations justify policy values more reliably than a target copied from another service.

Troubleshoot symptoms from metric to serving capacity

Avoid fixing every symptom by increasing maxReplicas. That changes the blast radius. Identify whether the delay sits in observation, HPA calculation, scheduling, startup, readiness, traffic distribution, or a downstream dependency, then change the control that owns that phase.

Kubernetes HPA production checklist

Frequently asked questions

What CPU target should a Kubernetes HPA use?

There is no universal target. Profile the application, set realistic CPU requests, find the utilization region that preserves latency and error objectives, and leave headroom for metric and startup delay. Validate the target with repeatable load tests.

Should HPA scale on CPU or memory?

Use the signal that changes with useful load and predicts the constrained resource. CPU often works for compute-bound services. Memory works only when per-Pod memory responds predictably and can fall after demand falls; retained heaps and caches frequently violate that assumption.

Does HPA add Kubernetes nodes?

No. HPA changes workload replica count. A separate node autoscaling system can add nodes when new Pods cannot be scheduled, subject to its own configuration and provisioning delay.

Can HPA use several metrics?

Yes. With autoscaling/v2, HPA evaluates each metric and chooses the largest replica recommendation. Operate every metrics pipeline as production infrastructure because a missing metric can prevent a recommended scale-down.