--- title: 'Idempotency Keys' source: 'https://academia.sh/en/courses/api-design/idempotency-keys' course: 'Web API Design' language: en updated: '2026-08-19T05:19:30+00:00' license: 'CC BY-SA 4.0' --- # Idempotency Keys The idempotency key that prevents a repeated effect on retry: matching the key to its scope and a body digest, a second request with the same key replaying the first response, rejecting a conflict from a different body, and the key store's lifetime. The third lesson measured that POST is not idempotent: sending the same request twice produced two loan records. Back then, that was noted as POST's nature and left alone. Once the eighth lesson closed the read side, this gap remained. The gap comes from a real situation. The client sends a request to create a loan record, and the network drops. The network-error class of the error contract from the Application Architecture course described exactly this: **the request may have reached the server, or it may not have.** If the response was lost on the way back, the record was created but the client does not know it. If the client retries, there is a risk of a second record; if it does not, the member may end up without the book. Neither option is acceptable. This lesson builds a third way: attaching a client-generated identity to the request. ## What the Key Does An **idempotency key** is the unique name a client gives to an operation's intent. It travels in the `Idempotency-Key` header and has three rules. The key is generated **per intent, not per request.** A key is generated when the user presses the "issue loan" button; three attempts made because the network dropped all carry that same key. If the user wants to check out a second book, a new key is generated. The key **is not enough on its own**. The server must also store which operation the key is tied to: which address, with which body. Otherwise, if the client mistakenly reuses the same key on a different request, that request gets silently swallowed. The key **is stored together with the response**. When the second request arrives, the operation is not redone; the response generated the first time is returned as is. The client learns the result even if it never saw the first response. ```js // key-server.mjs — loan creation guarded by an idempotency key import { createServer } from "node:http"; import { DatabaseSync } from "node:sqlite"; import { createHash } from "node:crypto"; const db = new DatabaseSync("library.db"); db.exec(`CREATE TABLE IF NOT EXISTS idempotency ( key TEXT PRIMARY KEY, path TEXT NOT NULL, digest TEXT NOT NULL, status INTEGER, response TEXT, createdAt TEXT NOT NULL )`); const readBody = (request) => new Promise((resolve) => { let data = ""; request.on("data", (p) => (data += p)); request.on("end", () => resolve(data)); }); const send = (response, status, data, extra = {}) => { response.writeHead(status, { "content-type": "application/json; charset=utf-8", ...extra }); response.end(JSON.stringify(data)); }; const server = createServer(async (request, response) => { const path = new URL(request.url, "http://127.0.0.1").pathname; if (request.method !== "POST" || path !== "/loans") return send(response, 404, { error: "path_not_found" }); const raw = await readBody(request); const key = request.headers["idempotency-key"]; const digest = createHash("sha256").update(raw).digest("hex").slice(0, 16); const body = JSON.parse(raw); // No key: every call produces a new record. if (!key) return send(response, ...create(body)); const record = db.prepare("SELECT * FROM idempotency WHERE key = ?").get(key); if (record) { // Same key, different body: the client reused the key. if (record.path !== path || record.digest !== digest) return send(response, 409, { error: "key_conflict", key }); return send(response, record.status, JSON.parse(record.response), { "idempotency-replayed": "true" }); } // Reserving the key and doing the work happen in one block, with no wait in between. db.prepare("INSERT INTO idempotency (key, path, digest, createdAt) VALUES (?,?,?,datetime('now'))") .run(key, path, digest); const [status, data] = create(body); db.prepare("UPDATE idempotency SET status = ?, response = ? WHERE key = ?") .run(status, JSON.stringify(data), key); send(response, status, data); }); function create(body) { if (!db.prepare("SELECT 1 FROM book WHERE isbn = ?").get(body.isbn)) return [422, { error: "validation", field: "isbn" }]; const s = db.prepare("INSERT INTO loan (member, isbn, issuedAt, return) VALUES (?,?,?,NULL)") .run(body.member, body.isbn, body.issuedAt); return [201, { id: Number(s.lastInsertRowid), member: body.member, isbn: body.isbn, issuedAt: body.issuedAt }]; } server.listen(8483, "127.0.0.1", () => console.log("key server 127.0.0.1:8483")); ``` ## Measurement The same body is sent twice: first without a key, then with one. The catalog is set up with the `catalog.sql` file from the seventh lesson. ```bash # The same request is sent twice: first without a key, then with an idempotency key. rm -f library.db && sqlite3 library.db < schema.sql && sqlite3 library.db < catalog.sql sqlite3 library.db "DELETE FROM loan;" node key-server.mjs & server=$! sleep 0.4 BODY='{"member":"U-1001","isbn":"K-02","issuedAt":"2026-03-10"}' say() { printf '%-30s loan rows: %s\n' "$1" "$(sqlite3 library.db 'SELECT COUNT(*) FROM loan;')"; } send() { curl -sS -D - -o /tmp/body.$$ -X POST -H 'content-type: application/json' "$@" \ -d "$BODY" http://127.0.0.1:8483/loans \ | grep -iE '^(HTTP/|idempotency-replayed)' | tr -d '\r'; cat /tmp/body.$$; echo; rm -f /tmp/body.$$; } echo "--- no key, twice ---" send; send; say "no key x2" sqlite3 library.db "DELETE FROM loan; DELETE FROM idempotency;" echo "--- same key, twice ---" send -H 'Idempotency-Key: OD-7f21' send -H 'Idempotency-Key: OD-7f21' say "with key x2" echo "--- same key, different body ---" curl -sS -o /dev/null -w 'status: %{http_code}\n' -X POST -H 'content-type: application/json' \ -H 'Idempotency-Key: OD-7f21' -d '{"member":"U-1002","isbn":"K-03","issuedAt":"2026-03-11"}' \ http://127.0.0.1:8483/loans say "after conflict" kill $server ``` ``` key server 127.0.0.1:8483 --- no key, twice --- HTTP/1.1 201 Created {"id":1,"member":"U-1001","isbn":"K-02","issuedAt":"2026-03-10"} HTTP/1.1 201 Created {"id":2,"member":"U-1001","isbn":"K-02","issuedAt":"2026-03-10"} no key x2 loan rows: 2 --- same key, twice --- HTTP/1.1 201 Created {"id":1,"member":"U-1001","isbn":"K-02","issuedAt":"2026-03-10"} HTTP/1.1 201 Created idempotency-replayed: true {"id":1,"member":"U-1001","isbn":"K-02","issuedAt":"2026-03-10"} with key x2 loan rows: 1 --- same key, different body --- status: 409 after conflict loan rows: 1 ``` The two requests without a key produced two records and returned two different identities. The two requests with a key produced a single record and returned **the same body**: the second response's `id` field is also 1. The second response also carries the `Idempotency-Replayed` header; this header is not mandatory, but it lets the client and the logs distinguish the situation. The last section tests reusing the key. The same `OD-7f21` key arrived with a different body, and the request was rejected with 409; the loan record count stayed at one. Had the body digest not been stored, this request would have been silently swallowed and the client would have assumed a second loan was created. Rejecting it is the right behavior: only the client can fix a key used incorrectly. ## Concurrent Requests Two requests arriving one after another is the easy case. The hard case is both arriving at the same time: while the client times out and retries, the first request may still be being processed. ```bash # Five requests with the same key are sent concurrently; the resulting records are counted. rm -f library.db response-*.json && sqlite3 library.db < schema.sql && sqlite3 library.db < catalog.sql sqlite3 library.db "DELETE FROM loan;" node key-server.mjs & server=$! sleep 0.4 pids="" for i in 1 2 3 4 5; do curl -sS -o "response-$i.json" -X POST -H 'content-type: application/json' \ -H 'Idempotency-Key: OD-9c04' \ -d '{"member":"U-1002","isbn":"K-05","issuedAt":"2026-03-12"}' \ http://127.0.0.1:8483/loans & pids="$pids $!" done wait $pids for f in response-*.json; do cat "$f"; echo; done | sort -u > variants.txt echo "distinct response bodies : $(wc -l < variants.txt | tr -d ' ')" echo "body : $(cat variants.txt)" echo "loan rows : $(sqlite3 library.db 'SELECT COUNT(*) FROM loan;')" echo "--- idempotency store age ---" sqlite3 library.db " INSERT INTO idempotency (key, path, digest, status, response, createdAt) VALUES ('OD-old','/loans','0000',201,'{}',datetime('now','-30 day')); SELECT key || ' age: ' || CAST(julianday('now') - julianday(createdAt) AS INT) || ' days' FROM idempotency ORDER BY createdAt; DELETE FROM idempotency WHERE createdAt < datetime('now','-7 day'); SELECT 'remaining after cleanup: ' || COUNT(*) FROM idempotency;" kill $server ``` ``` key server 127.0.0.1:8483 distinct response bodies : 1 body : {"id":1,"member":"U-1002","isbn":"K-05","issuedAt":"2026-03-12"} loan rows : 1 --- idempotency store age --- OD-old age: 30 days OD-9c04 age: 0 days remaining after cleanup: 1 ``` Five concurrent requests produced a single record, and all five got the same body. What makes this work is the code's structure: there is no wait point between recording the key and doing the work. If a wait were introduced in between, both requests could see "no key" and both could create a record. This arrangement being sufficient on a single-process server does not mean it stays sufficient in a multi-process deployment. The general fix is **writing the key as a primary key**: the second write attempt is rejected by the database, and that request is routed to wait and reread, or to an "operation in progress" response. How races of this kind are isolated at the transaction level is the subject of the Transactions topic in the Advanced SQL course. ## The Key Store's Lifetime The output's last section shows the key store aging. A key written thirty days ago was cleaned up under a seven-day retention window. The retention decision has two bounds. The lower bound is set by the client's **longest retry window**: if a client is still retrying twenty-four hours later, the key must be kept that long. The retry budget defined in the Retry and Backoff lesson of the Application Architecture course gives this window. The upper bound is set by storage cost and the size of the response bodies written into the table. Once the window closes, a request arriving with the same key counts as a new operation and produces a new record. This is the design's accepted limit; the duration is chosen with this in mind. One last boundary is scope. A key must be unique together with the authenticated client, not on its own: two different clients producing the same key string must not lead one's request to receive the other's response. In the table above, the key alone is the primary key; in a real deployment, the client's identity also becomes part of the key. ## Summary - Under a network error, whether the request reached the server is unknown; retrying risks a repeated effect, not retrying risks a missing one. - An idempotency key is generated per intent, not per request; every attempt of the same operation carries the same key. - The server stores the key together with the path and a body digest; in the measurement, two requests without a key produced two records, two requests with one produced one record, and the second response matched the first exactly. - The same key arriving with a different body is rejected with 409; silently swallowing it would show the client an operation that never happened as if it had. - No wait must fall between recording the key and doing the work; five concurrent requests produced a single record under this structure. - The key store's retention period cannot be shorter than the client's longest retry window; once the window closes, the same key counts as a new operation. ## Next Step Across nine lessons, the client was assumed to build every address itself: the collection address, the page cursor, the field list, even the address of a newly created record. To do this, the client embeds the contract's shape into itself — and when the contract changes, every client gets updated separately. The next lesson loosens this dependency: the response carries within itself the addresses the client will use in its next request. Which operations a loan record is open to is also read from the same place; a returned record no longer offers a return link.