Kubernetes Autoscaling: HPA, VPA, and Cluster Autoscaler Boundaries

How to make autoscaling predictable: right requests, sane HPA behavior, VPA recommendations, and capacity-aware cluster scaling.

Autoscaling is a system of systems. If it’s unstable, it’s rarely “HPA is bad”—it’s usually that the inputs are wrong or multiple controllers are fighting each other.

This page focuses on concrete autoscaling that behaves well in production.

The three layers

HPA (Horizontal Pod Autoscaler): changes replica count based on metrics.
VPA (Vertical Pod Autoscaler): changes (or recommends) requests/limits.
Cluster Autoscaler: adds/removes nodes when the cluster lacks capacity.

If you want stability, you must align all three.

Step 0: Requests are the foundation

Many autoscaling behaviors depend on requests:

HPA CPU utilization is often computed as: current CPU usage / CPU requests
Scheduler placement uses requests
Cluster autoscaler reacts to unschedulable Pods (often caused by requests)

If requests are wrong:

HPA scales at the wrong time
Pods pack poorly
scaling becomes either too aggressive or too slow

Before tuning HPA, confirm every container has reasonable requests.

HPA basics (CPU utilization example)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api
  namespace: app
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60

Interpretation:

if average CPU usage across Pods is ~60% of request, HPA tries to hold steady
above 60% it scales up, below it scales down

Avoid oscillation: stabilization and scaling policies

HPA v2 supports behavior tuning:

stabilization windows (avoid rapid scale down/up)
scaling policies (cap how fast it changes)

Example:

behavior:
  scaleUp:
    stabilizationWindowSeconds: 0
    policies:
      - type: Percent
        value: 100
        periodSeconds: 60
  scaleDown:
    stabilizationWindowSeconds: 300
    policies:
      - type: Percent
        value: 20
        periodSeconds: 60

Why this works:

scale up quickly (handle bursts)
scale down slowly (avoid flapping when traffic dips briefly)

Pick the right metric (CPU is not always correct)

CPU utilization is a good default for:

stateless web services

But it’s often wrong for:

queue workers (better: queue depth / lag)
memory-bound workloads (CPU stays low while memory grows)
I/O bound services

Consider:

custom metrics (queue length, request latency, RPS)
external metrics (cloud load balancer metrics)

If you scale on latency, make sure you avoid feedback loops (latency is affected by many factors).

Cold start, warmups, and readiness

Autoscaling adds Pods, but Pods need time to become useful:

image pull time
startup time (JVM warmup, migrations)
readiness gating

If your service has cold starts, you may need:

higher minReplicas
faster scale up policies
pre-pulled images (node cache)
startupProbe/readiness tuned to avoid serving too early

Cluster Autoscaler: the capacity layer

HPA can request more replicas, but if the cluster has no room, Pods go Pending. That is where Cluster Autoscaler should add nodes.

Common reasons cluster autoscaler does not help:

Pods have node selectors/affinity that no node group can satisfy
requests are so high that no node type can fit the Pod
node group max size is reached (quota)
PodDisruptionBudgets or constraints block scale down

During an incident, check:

kubectl get pod -n <ns> | rg -n "Pending"
kubectl describe pod -n <ns> <pod> | rg -n "FailedScheduling|Insufficient"

If Pods are Pending due to insufficient resources, autoscaler should react—if it doesn’t, inspect autoscaler logs and node group configuration.

VPA: recommendations are valuable even if you don’t auto-apply

Current VPA modes include:

Off: produce recommendations without changing Pods
Initial: set requests only when a Pod is created
Recreate: apply updates by recreating Pods
InPlaceOrRecreate: try in-place resize, then fall back to recreation when supported
InPlace: attempt only in-place resize and never evict, with the required Feature Gate

Auto is deprecated; select an explicit update mode instead. A low-risk path is Off, then Initial, followed by an in-place mode only after the cluster supports and validates it.

HPA + VPA together: avoid fighting

If you use both:

use HPA for CPU-based scaling
use VPA for memory request recommendations (or non-HPA-controlled resources)

Be careful if VPA changes CPU requests frequently while HPA scales on CPU utilization:

changing CPU requests changes the denominator
HPA may scale differently even if real load is unchanged

Practical pattern:

enable VPA recommendations
adopt them periodically (manual review) or via “Initial” mode

Memory-based scaling: be careful

HPA can scale on memory, but memory is often a lagging indicator:

cache growth raises memory without guaranteeing that more replicas will help
scale-down can force expensive cache rebuilds

If memory must drive scaling:

keep requests.memory close to observed peaks or percentage targets become misleading
use a long scale-down stabilization window
treat memory as a guardrail, not the only demand signal

For many services, a safer combination is HPA on CPU or request rate, VPA for memory requests, and conservative scale-down.

Queue and stream workloads: scale on backlog

CPU utilization often misses the real pressure on queue-driven Pods. Prefer signals such as:

pending messages or queue depth
consumer lag
average processing time per consumer

Event-driven systems such as KEDA can scale from backlog and, where the workload permits, scale to zero. The component is optional; the important choice is a metric that represents business demand.

Roll out in stages

Set reasonable requests for application containers, sidecars, and init containers.
Enable HPA first, with conservative targets and slow scale-down.
Use a load test or real peak to record the path from Pod creation to Ready.
Confirm Cluster Autoscaler can add nodes without quota or placement blockers.
Add VPA recommendations last, then adjust requests over a stable observation window.

Troubleshooting autoscaling (what to check)

HPA status and events

kubectl get hpa -n <ns>
kubectl describe hpa -n <ns> <hpa>

Look for:

“unable to get metrics”
“missing request for cpu”
rapid scale events

Metrics pipeline

If HPA can’t read metrics, it can’t work. Confirm:

metrics-server (for resource metrics)
or Prometheus adapter / custom metrics adapter (for custom/external metrics)

Requests and QoS

If requests are missing or extreme, scaling will be weird. Audit requests in Deployments and sidecars.

Run a scaling drill before release

Autoscaling “looks fine” until your first real burst.

Before relying on it in production, validate:

how long it takes to add Pods (startup + readiness)
how long it takes to add nodes (autoscaler + provisioning)
whether the service can handle partial capacity during the ramp

If your node provisioning takes 3–5 minutes, you may need:

higher minReplicas
pre-warmed node pools
faster scale-up policies

Stable autoscaling is less about one knob and more about aligning boot time, metrics, and capacity.

Pre-rollout self-check

Requests match observed workload behavior
HPA scale-down is conservative
The metrics pipeline is reliable
Cluster Autoscaler can satisfy placement constraints and capacity demand
VPA recommendations feed regular request reviews

Common questions

Q: When should I use HPA vs VPA?

A: Use HPA to scale replica count and VPA to right-size requests. They can target the same workload, but do not let VPA frequently change CPU requests while HPA scales on CPU utilization. Start with VPA Off, or limit VPA to memory recommendations.

Q: Why does HPA oscillate?

A: Metrics lag and aggressive policies can cause thrash. Add stabilization windows, slow scale down, and tune targets.

Q: Do I need metrics-server?

A: For CPU or memory metrics, yes. For custom metrics, use a metrics pipeline (for example Prometheus plus adapter).