Retries Done Right: Exponential Backoff, Jitter and Retry-After

Retries are the cheapest reliability feature you can add to an API client and one of the most common causes of outages. Exponential backoff with jitter, a hard cap on attempts and a respect for Retry-After is what turns a retry loop from an amplifier of failure into something that helps a struggling service recover.

The usual mistake looks harmless: a for loop that tries three times with a fixed one second sleep. It works fine when one request fails on its own. It goes badly when a dependency has a 20 second blip and every client in the fleet fails at the same moment, sleeps for the same second and comes back in one synchronized wave.

Why naive retries make outages worse

A retry is extra load sent to a system at the exact moment that system is least able to handle it. Three properties of naive retry loops turn that into a feedback loop.

Synchronized clients form a thundering herd

If 10,000 clients lose a connection at t=0 and all retry after a fixed 1 s, the server sees nothing for a second and then 10,000 requests in a few milliseconds. If it was recovering, that spike knocks it over again, the clients all fail together again, and the pattern repeats. Plain exponential backoff (1 s, 2 s, 4 s) spreads the waves out in time but keeps each wave just as sharp, because every client computes the same delays.

Retries multiply across layers

Retries compound when services call services. If the edge retries 3 times, the API tier retries 3 times and the data access layer retries 3 times, a single user request can turn into 3 × 3 × 3 = 27 attempts against the database. The Google SRE book's chapter on addressing cascading failures treats this as a primary cause of cascading outages and recommends retrying at only one layer of the stack.

Retrying the wrong failures wastes the budget

A 400 Bad Request will fail identically on every attempt. A 401 will fail until someone fixes the credentials. Retrying them adds latency for the caller and load for the server with no chance of success.

Exponential backoff, worked through

The basic formula is to wait base × multiplier^attempt milliseconds, clamped to a ceiling:

delay(n) = min(cap, base * 2^n)

With base = 200 ms, multiplier = 2, cap = 10 s and five retries, the delays are 200, 400, 800, 1,600 and 3,200 ms. The worst-case total wait is 6.2 s. A sixth retry would wait 6,400 ms, and from the seventh onward the cap holds every delay at 10 s. The cap matters more than it looks. Without it, attempt 15 would wait about 109 minutes, which for any interactive caller is just a slower way to fail.

The growth is the point: early retries are quick enough to cover a dropped packet or a pod restarting, and later ones back off far enough to give an overloaded service room to drain its queue.

Jitter is what actually breaks up the herd

Jitter randomizes each delay so that clients which failed together don't retry together. Marc Brooker's AWS Architecture Blog post Exponential Backoff And Jitter simulated contending clients and compared several strategies. Its main finding still holds: adding randomness cut both total work and completion time sharply compared with pure exponential backoff. "Full jitter" was the simplest strategy among the best performers.

The common variants, with ceiling = min(cap, base * 2^n):

Full jitter is the sensible default. Its expected delay is half the ceiling, so the five-retry schedule above averages about 3.1 s of total waiting instead of 6.2 s, and the arrivals are spread across the whole window rather than stacked on its edges. If a near-zero first retry worries you (it shouldn't, usually), equal jitter is the conservative alternative.

Decide what is safe to retry

Backoff answers when. The harder question is whether, and it comes down to two things: is the failure likely to be transient, and is the operation safe to repeat?

The timeout row causes most of the real damage. When a POST /payments times out, the client cannot tell whether the charge happened. Retrying blindly risks a double charge, and not retrying risks a lost payment. The fix is to make the operation idempotent so a retry is always safe, which is what idempotency keys are for: the client sends the same key on every attempt and the server returns the stored result instead of executing twice. GET, PUT and DELETE are idempotent by HTTP semantics. POST and PATCH need a key.

Respect Retry-After and rate-limit headers

When the server tells you when to come back, that beats any formula. RFC 9110 section 10.2.3 defines Retry-After as either a number of seconds or an HTTP date:

HTTP/1.1 429 Too Many Requests
Retry-After: 30

HTTP/1.1 503 Service Unavailable
Retry-After: Fri, 13 Nov 2026 18:00:00 GMT

A well-behaved client parses both forms, waits at least that long, and still adds a little jitter on top so that every client told "30" doesn't return in the same millisecond. If the requested wait is longer than the caller can tolerate, fail fast and report it rather than holding a thread for minutes. On the server side, sending Retry-After with every 429 and 503 is one of the cheapest ways to make your clients behave. The IETF RateLimit header fields draft goes further and standardizes how a server advertises its quota and how much remains, so clients can slow down before they hit a 429 at all.

Retry budgets: the circuit breaker's lighter cousin

Per-request limits ("at most 3 retries") don't stop a fleet-wide storm: if every request is failing, every request still makes 3 retries and load quadruples. A retry budget caps retries as a share of total traffic instead. The client keeps a token bucket that each successful request refills a little and each retry drains. When the bucket is empty, failures are returned immediately instead of retried.

This idea is built into mainstream stacks. gRPC's retry support pairs a per-method retryPolicy (maxAttempts, initialBackoff, maxBackoff, backoffMultiplier, retryableStatusCodes) with a retryThrottling token bucket that switches retries off when too many calls are failing. The AWS SDKs' standard retry mode similarly draws retries from a quota that drains on failure. The effect is that retries help with sporadic failures, where they're useful, and switch themselves off during a full outage, where they would only add load.

A retry helper that does it properly

Here is a compact TypeScript version that combines the pieces: full jitter, a cap, Retry-After, a retryable-status allowlist, an overall deadline and a simple retry budget.

const RETRYABLE = new Set([408, 429, 502, 503, 504])

class RetryBudget {
  private tokens = 10
  constructor(private max = 10, private refillPerSuccess = 0.1) {}
  trySpend() { if (this.tokens < 1) return false; this.tokens -= 1; return true }
  onSuccess() { this.tokens = Math.min(this.max, this.tokens + this.refillPerSuccess) }
}

function parseRetryAfter(h: string | null): number | null {
  if (!h) return null
  const secs = Number(h)
  if (Number.isFinite(secs)) return Math.max(0, secs * 1000)
  const date = Date.parse(h)
  return Number.isNaN(date) ? null : Math.max(0, date - Date.now())
}

export async function fetchWithRetry(
  url: string,
  init: RequestInit,
  budget: RetryBudget,
  { maxRetries = 4, baseMs = 200, capMs = 10_000, deadlineMs = 15_000 } = {},
): Promise<Response> {
  const start = Date.now()
  for (let attempt = 0; ; attempt++) {
    let res: Response | undefined
    try {
      res = await fetch(url, init)
      if (!RETRYABLE.has(res.status)) { budget.onSuccess(); return res }
    } catch (err) {
      // Network error. Only safe to repeat if the request is idempotent
      // (GET/PUT/DELETE, or a POST carrying an Idempotency-Key header).
      if (attempt >= maxRetries) throw err
    }
    if (attempt >= maxRetries || !budget.trySpend()) {
      if (res) return res
      throw new Error('retry budget exhausted')
    }
    const ceiling = Math.min(capMs, baseMs * 2 ** attempt)
    const jittered = Math.random() * ceiling
    const serverWait = parseRetryAfter(res?.headers.get('retry-after') ?? null)
    const wait = serverWait != null ? serverWait + Math.random() * 250 : jittered
    if (Date.now() - start + wait > deadlineMs) return res ?? Promise.reject(new Error('deadline exceeded'))
    await new Promise((r) => setTimeout(r, wait))
  }
}

A few choices are deliberate. The deadline bounds the total time, so a caller with a 15 s SLA never waits 40 s across retries. The budget is shared across all requests to the same dependency, not created per call. 500 is left out of the allowlist on purpose. Add it only for endpoints where you know a 500 is usually transient.

Where retries should live

Retry at one layer, ideally the one closest to the user that can still make a sensible decision, and make the layers below fail fast. For background work that needs to survive minutes or hours of downtime, an in-process loop is the wrong tool: push the work into a queue or a durable execution engine that persists the attempt count and schedules the next try even if the process dies. Webhook senders such as Stripe and Svix follow the same idea. They retry deliveries on a widening schedule over hours or days, which is why webhook receivers must deduplicate by event ID rather than assume each event arrives once. The same applies to LLM API calls, where 429s are routine: an LLM gateway that centralizes retries and fallbacks beats every agent tool implementing its own loop.

If you want to sanity-check a schedule before shipping it, a retry backoff calculator shows the per-attempt ceilings, the worst-case total and how the cap kicks in for a given base, multiplier and attempt count.

Takeaway

A retry policy has five parts, and missing any one of them can turn a small blip into an outage:

1. Exponential backoff with a cap, so waits grow but stay bounded.
2. Full jitter, so clients that failed together don't retry together.
3. An allowlist of retryable failures, plus idempotency (keys for POST) for anything that may already have executed.
4. Retry-After honored when present, with a little jitter added.
5. A retry budget and an overall deadline, so retries switch off during a real outage and never exceed the caller's SLA.

Retry at one layer, fail fast below it, and treat every retry as load you are choosing to add.