A dropped connection makes a client retry a POST that already succeeded — without an idempotency key, that retry becomes a duplicate charge, not a safety net. Here's how the pattern actually works and where naive implementations fail.
A payment service receives POST /charges for $49.00. The database write commits, the charge succeeds — and then the response never makes it back to the client. The connection drops a hundred milliseconds after the commit. From the client's point of view, the request timed out, so it does what every retry policy tells it to do: it sends the exact same POST /charges again.
Now the customer has two charges on their card for the same order.
This isn't a hypothetical edge case — it's the default behavior of TCP timeouts, load balancer failovers, and mobile networks dropping mid-response. Any client-side retry logic (and you should have retry logic) turns this into a systematic source of duplicate writes unless the server gives clients a way to say "this is the same request I already sent, not a new one." That mechanism is an idempotency key, and most implementations of it are subtly broken in ways that only surface under concurrent load.
HTTP already has a notion of idempotency baked into the method semantics: GET, PUT, and DELETE are supposed to be safe to retry because applying them twice produces the same end state as applying them once. POST is explicitly not idempotent — two identical POST requests are, by spec, two different operations. That's correct for "create a blog post" and wrong for "charge a card" or "send a payment," where the caller's intent is "make this happen, exactly once" even if the network makes them ask twice.
An idempotency key closes that gap without changing HTTP semantics. The client generates a unique token — a UUID is the standard choice — and sends it on the request, usually as a header:
POST /charges
Idempotency-Key: 7e1a3f2c-9d4b-4a1e-8f7a-2b6c9e0d1f3a
{ "amount": 4900, "currency": "usd", "customer": "cus_88x2" }
The server's job is simple to describe and easy to get wrong: the first request with that key executes normally. Every subsequent request with the same key returns the stored result of the first one, without re-executing the operation. The client can retry as aggressively as it wants — same key, same outcome, exactly one charge.
A first-pass implementation looks like a cache: store key -> response in a table, check it before processing, write to it after. That works for the textbook case of a clean retry after a timeout. It breaks in three specific ways that don't show up until production traffic hits it.
Concurrent retries race past the check. If a client fires two requests with the same key nearly simultaneously — a common pattern when a mobile client retries on a short timeout while the original request is still in flight — both requests can hit the "check if key exists" step before either has written a result. Both see no cached entry, both proceed, and you get the double charge the key was supposed to prevent. The fix is to write a row in an in_progress state before processing starts, using a unique constraint on the key column so the second request's insert fails immediately. That second request should then either block briefly and poll for the first request's result, or return 409 Conflict with a "request already in progress" body — never fall through to re-executing the operation.
Same key, different payload, silent wrong answer. A client bug reuses an idempotency key across two genuinely different requests — say, a retry loop that doesn't regenerate the key when the request body changes. A naive implementation returns the cached response from the first payload, silently discarding the second, different request. The caller thinks they charged $99 and got back a $49 confirmation with no error. Stripe's API, one of the most widely copied implementations of this pattern, handles this by hashing the request parameters alongside the key: if a key is reused with a different parameter hash, the server returns an error instead of a stale cached response.
Unbounded key storage. Every idempotency key needs a retention window, not permanent storage. Stripe uses 24 hours. Pick a TTL long enough to cover realistic retry windows (including a mobile client that retries after being offline for a few hours) and expire keys after that — otherwise the dedup table grows forever and every write pays an ever-larger index lookup.
Assume a table idempotency_keys(key PRIMARY KEY, status, request_hash, response_body, created_at).
POST /charges with key K1, body B1.hash(B1), attempts INSERT (K1, 'in_progress', hash(B1), NULL). Insert succeeds — this is a new key.('completed', hash(B1), <response>).POST /charges with key K1, body B1 (unchanged).K1 — row already exists. Server reads the row: status completed, hash matches B1. Returns the stored response directly. No new charge.If step 5 arrived before step 3 finished (true concurrent retry), the insert in step 6 still fails on the constraint, but the row's status is in_progress. The server returns 409 or polls-and-waits rather than re-running the charge logic — the request that's actually executing is the only one that gets to write the terminal state.
| Approach | Prevents duplicate execution | Handles concurrent retries | Catches payload mismatch | Storage cost |
|---|---|---|---|---|
| No idempotency key (retry as-is) | No | No | N/A | None |
| Cache-style: check-then-write | Yes, for sequential retries | No — race window | No | One row per key |
Key + unique constraint, in_progress state | Yes | Yes | No | One row per key |
| Key + unique constraint + request hash (Stripe-style) | Yes | Yes | Yes | One row per key + hash column |
The last row is the only one that's actually safe to expose on a public API where you don't control client behavior. The middle rows are fine for internal service-to-service calls where you trust the caller not to reuse keys across different payloads.
Idempotency keys are frequently scoped too broadly or too narrowly. Scoping a key globally across all endpoints means a key generated for POST /charges could theoretically dedupe against POST /refunds if a client bug reuses it — scope keys to the combination of endpoint (or operation type) and key value, not the key alone. Scoping per-account rather than globally also matters for multi-tenant APIs: without account scoping, one tenant could accidentally (or maliciously) collide with another tenant's key and read back a cached response that isn't theirs.
If your API has any endpoint where a duplicate execution has a real-world cost — money moved, an email sent, an order placed — retry-safety isn't optional, and "the client shouldn't retry" isn't a policy you can enforce over an unreliable network. Require an idempotency key header on those endpoints, enforce uniqueness at the database level with an explicit in_progress state to close the concurrent-retry race, and hash the request body so a reused key with a different payload fails loudly instead of returning a stale response. Clients need a reliably unique key to send in the first place — a UUID generator like Utilix's UUID tool is the simplest way to produce one per logical request, not per retry attempt.