Idempotency keys: an interactive duplicate-payment simulator

An idempotency key is a unique value a client sends with a request that changes something, such as a payment, so the server can recognise a retry of the same operation and return the original result instead of doing the work again. It makes retries safe in the one situation where they are dangerous: when the client cannot know whether its first attempt succeeded.

The playground charges a card $40 through an API while something goes wrong: a lost response, a lost request, a slow card network, or a retry that changed the amount. Turn the key off and on, and watch the card statement.

Why retries create duplicates

When a request times out, the client cannot tell what happened. The request may have been lost before it arrived, the response may have been lost on the way back after the work was done, or the server may still be working. To the client all three look the same. Retrying is right in the first case and wrong in the other two, so without help from the server a client that retries will sometimes do things twice: in the first preset, the card is charged $80 for a $40 order.

Some requests are naturally safe to repeat. GET, PUT and DELETE are idempotent by definition: setting a value twice, or deleting something twice, leaves the same result. POST, “create a charge”, “send a message”, is not, and those are the requests that need a key.

How idempotency keys work

1. The client picks a key per operation, usually a random UUID, and sends it in a header such as Idempotency-Key. Every retry of that operation reuses the same key; a new operation gets a new one.
2. The server claims the key first: it inserts the key and a fingerprint of the request under a unique constraint, before doing any work. If two attempts arrive together, only one insert succeeds.
3. A repeat is answered from the record: the stored response if the first attempt finished, 409 if it is still running, and 422 if the same key comes with a different request.
4. The result is stored with the work: in the same database transaction as the charge record, so the key can never say done for work that did not happen.
5. Keys expire after long enough to cover any retry; Stripe keeps them for 24 hours.

An idempotency key table in SQL

CREATE TABLE idempotency_keys (
  account_id   bigint      NOT NULL,
  key          text        NOT NULL,
  fingerprint  text        NOT NULL,          -- hash of method, path and body
  response     jsonb,                         -- null while in progress
  locked_until timestamptz,                   -- lets a crashed attempt be retried
  created_at   timestamptz NOT NULL DEFAULT now(),
  PRIMARY KEY (account_id, key)               -- the unique constraint is the lock
);

-- 1. claim the key before doing any work
INSERT INTO idempotency_keys (account_id, key, fingerprint, locked_until)
VALUES ($1, $2, $3, now() + interval '30 seconds')
ON CONFLICT DO NOTHING;
-- no row inserted? read the existing one:
--   different fingerprint      -> 422, the key was used for another request
--   response is null, locked   -> 409, still in progress
--   response is set            -> return the stored response, do nothing else

-- 2. do the work, passing the same key to the payment provider
-- 3. store the result in the same transaction as the work's own writes
UPDATE idempotency_keys SET response = $4, locked_until = NULL
WHERE account_id = $1 AND key = $2;

The edge cases that matter

• Concurrent retries: checking whether a key exists and then inserting it is a race. Two attempts can both see no key and both charge. Let the unique constraint decide, as in the third preset, where the retry gets a 409 instead of a second charge.
• Calls to other services: your database transaction cannot include the card network. Pass a key downstream too, derived from yours, so the provider deduplicates its side; most payment APIs accept one.
• A crash mid-operation: if the server dies after charging but before storing the response, the key stays in progress. Give the claim a lease, and before retrying the work, check with the provider whether it already happened.
• A changed request: the same key with a different body is a bug or a user edit. Reject it with 422, as in the fourth preset, rather than charge again or return a result for the wrong amount.
• Scope: store keys per account, so one customer’s key can never collide with or reveal another’s response.

Idempotency beyond HTTP APIs

Message queues deliver at least once, so consumers face the same problem: a message processed, then redelivered after a crash before the offset was committed. The fix is the same idea: record each message id in a processed table in the same transaction as the work, and skip ids already there. Writes can also be made naturally idempotent: an upsert by a client-chosen id, “set status to paid” instead of “add $40”, or a conditional write that only applies if the version is still 3.

In a system design interview

Bring it up whenever money moves or a retry could repeat a side effect: “The client sends an idempotency key with each payment, generated when the user presses pay and reused on retries. The payments service inserts the key under a unique constraint before calling the provider, passes it on to the provider, and stores the result with the payment record, so a retried request returns the original result.” Pair it with retries and backoff, and say how long keys are kept.

Read the guide to idempotency keys, see exponential backoff and jitter for the retries themselves, or watch a strong candidate design a payment system.

Questions people ask