Kubernetes Rollout Drops Traffic on Every Deploy? Here's the Fix

Your kubectl rollout status says everything succeeded. The new pods are running, the old ones are gone, and the Deployment reports healthy. But your monitoring lit up with 502s and connection errors for the fifteen seconds the rollout took. Every single deploy does this.

The rollout is not the problem. The problem is what happens at the edges of each pod, in the moments when traffic is being handed off. Here is what is actually going wrong and how to fix it.

The real cause: endpoint timing

Kubernetes routes traffic to a pod through its Service endpoints. A pod gets traffic when its IP is in the endpoint list, and stops getting traffic when it is removed. Errors during rollouts come from two timing gaps in that process:

• A new pod is added to the endpoints before it can actually serve traffic. Kubernetes thinks the pod is ready, sends it requests, and the app is not actually able to answer them yet. Result: errors on the new pods.
• An old pod is removed while it is still serving traffic. Kubernetes starts terminating the pod, but the load balancer or Service is still routing requests to it for a short window. Result: errors on the old pods during shutdown.

Both are timing mismatches between “Kubernetes thinks this pod is ready or gone” and “the pod is actually ready or actually drained.” Fix the timing and the errors disappear.

Fix 1: a readiness probe that tells the truth

The most common cause of errors on new pods is a missing readiness probe, or one that returns healthy before the app can really serve traffic. The readiness probe is what controls whether a pod is added to the Service endpoints, so it must reflect real readiness.

readinessProbe:
  httpGet:
    path: /ready
    port: 8080
  initialDelaySeconds: 5
  periodSeconds: 5
  failureThreshold: 3

Two rules for the endpoint it checks:

• It must return success only when the app can actually handle a request — connections warmed, caches loaded, config read.
• It must check only the app itself, not external dependencies. If the readiness probe also pings your database, one database blip marks every pod unready at once and pulls the whole deployment out of rotation.

Fix 2: a preStop hook to drain traffic

The most common cause of errors on old pods is no preStop hook combined with an app that exits immediately on SIGTERM. When a pod is told to terminate, endpoint removal and the actual shutdown happen almost simultaneously, so the load balancer keeps sending traffic to a process that is already gone.

A preStop hook with a short sleep fixes this. It delays SIGTERM for a few seconds so endpoint removal propagates first:

lifecycle:
  preStop:
    exec:
      command: ["/bin/sh", "-c", "sleep 15"]

During that sleep, Kubernetes removes the pod from the Service endpoints and the load balancer stops routing new traffic to it. Only then does the app receive SIGTERM and shut down, with no new requests arriving mid-shutdown.

Fix 3: a grace period longer than your drain

terminationGracePeriodSeconds is how long Kubernetes waits after SIGTERM before force-killing the pod with SIGKILL. If it is shorter than your preStop delay plus the time to finish in-flight requests, Kubernetes kills the pod mid-request.

Set it longer than preStop delay + longest expected request:

spec:
  terminationGracePeriodSeconds: 60   # preStop(15s) + request drain + buffer

Your application also needs to handle SIGTERM gracefully: stop accepting new connections, finish the in-flight ones, then exit. The grace period gives it the room to do that.

Fix 4 (EKS): target group deregistration delay

On EKS with the AWS Load Balancer Controller in IP target mode, there is an extra layer. The ALB can keep sending traffic to a target that Kubernetes already considers gone, because the target group’s deregistration lags behind endpoint removal.

Two things to check:

• Set the target group deregistration delay to a sensible value (the ALB stops sending to a draining target after this).
• Make your preStop sleep long enough to cover it, so the ALB has finished deregistering the target before the pod process exits.

If your preStop sleep is 15 seconds but the ALB takes 30 seconds to stop routing, you still drop traffic. The sleep has to outlast the load balancer, not just the Service endpoint update.

Putting it together

A Deployment that rolls out without dropping traffic ties all of this into one spec:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 4
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxUnavailable: 0    # never drop below desired capacity
      maxSurge: 1
  template:
    spec:
      terminationGracePeriodSeconds: 60
      containers:
        - name: web
          image: myregistry/web:v2
          ports:
            - containerPort: 8080
          readinessProbe:
            httpGet:
              path: /ready
              port: 8080
            periodSeconds: 5
            failureThreshold: 3
          lifecycle:
            preStop:
              exec:
                command: ["/bin/sh", "-c", "sleep 15"]

With maxUnavailable: 0, the readiness probe gating new traffic, the preStop hook draining old traffic, and a grace period that outlasts both, the handoff at each pod’s edges is clean, and the 502s stop.

Where to go next

To see where the Service and endpoints sit in the request path, read how traffic flows in Kubernetes. For the kubectl commands to inspect a rollout and its endpoints, see the Kubernetes commands you must know reference.