>_
EngineeringNotes
← Back to Important Concepts

Idempotency

Guaranteeing exact-once execution in unreliable networks to prevent disastrous double-charges.

01

What is Idempotency?

"An operation is idempotent if it produces the same result regardless of whether it is executed once or multiple times."

execute same operation multiple times, result is same as if operation was applied justONCE

02

The Like Button Example

User
/post/123/like
API
Database

No matter how many times user likes one post, the like count should just increase by 1

729
730
03

The Payment Problem

The situation becomes even more serious when $ is involved

User
/pay/B/20000
API
Database

Say A wants to transfer $ 20,000 to B, and if due to any reason the API call is retried. We would not want A to transfer twice the amount to B.

we need idempotence
04

Other Places Where This Is Relevant

  • -user to not tweet the same tweet again
  • -not placing the same order twice on Amazon
  • -not sending the same message again
05

Why Would a Transaction Repeat?

Why would the transaction repeat?user or service

You retry when something goes wrong

payment servicepayment gateway
REQRESSAME REQRES
X
Service Crashedor transient issue
X
Payment service retries
after recovery
A B $20000

Deducted
From A
Credited
to B

But your service doesn't know
hence it retries !!

Retry only when
you are sure

06

Failure Scenarios

user A
X
API

Client knows that the request didn't even reach the server, hence, safe to retry

user A
XAPI

Failure while server is executing. Client cannot be sure of retrying.

user A
X
API

Server completed execution but before the response reach client, network fails. Client cannot sure of retrying.

07

Implementing Idempotence

Approach 1: You won't need idempotence if weDo not retry

Depending on your product/usecase this might be the best thing

If operation failed, propagate the error and show it to the end user

let the user retry
X
API

End user retries if he/she wants to


Approach 2 :Check and update

Idea:Get the status of payment & process only when not already processed.

Implementation

create a unique payment_id

and weave your API calls with it

Your API server can use this to get, check & update if needed

08

Idempotent Flow

1.

Payments service talks to Payment Gateway & generates a

Payment ID

2.

Payment service initiates the payment through this ID

A B $20,000
3.

If payment service retries, it first

checks the status of payment on ID
if the status is COMPLETED
then does not retry
else retry through the same ID

Payments
Service
Payments
Gateway
Generate ID
P1729
A -> B, $200P1729
Transfer done
X
P1729
AlreadyCompleted
09

Implementation Example (Express.js)

Here is a conceptual implementation of an idempotent API using Express.js. It demonstrates the "Check and Update" logic by validating a unique Idempotency-Key header against an internal data store.

server.js
import express from "express";
const app = express();
app.use(express.json());

// Mock Database table for idempotent requests
// Schema: { payment_id: string, status: "PROCESSING" | "COMPLETED", result: any }
const idempotencyStore = new Map();

app.post("/api/pay", async (req, res) => {
  const { amount, from_user, to_user } = req.body;
  const payment_id = req.headers["idempotency-key"];

  if (!payment_id) {
    return res.status(400).json({ error: "Idempotency-Key header is required" });
  }

  // 1. Check if the payment_id has been seen before (Generate ID / Check Status)
  if (idempotencyStore.has(payment_id)) {
    const record = idempotencyStore.get(payment_id);

    if (record.status === "COMPLETED") {
      // Return exactly what was returned the first time (Already Completed)
      return res.status(200).json({
        message: "Already completed.",
        data: record.result
      });
    }

    if (record.status === "PROCESSING") {
      // Concurrent request issue (e.g. user double clicked quickly)
      return res.status(409).json({ error: "Request is already being processed." });
    }
  }

  // 2. Mark as processing to handle concurrent identical requests
  idempotencyStore.set(payment_id, { status: "PROCESSING", result: null });

  try {
    // 3. Process the actual payment logic (A -> B, $20,000)
    const result = await processPayment(from_user, to_user, amount);

    // 4. Update the record status to COMPLETED
    idempotencyStore.set(payment_id, { status: "COMPLETED", result: result });

    return res.status(200).json({ message: "Transfer done", data: result });
  } catch (error) {
    // If it fails, remove the key so it can be safely retried
    idempotencyStore.delete(payment_id);
    return res.status(500).json({ error: "Internal processing error." });
  }
});