Designing Idempotent APIs: From Theory to Implementation

What Is Idempotency?

An operation is idempotent if performing it multiple times produces the same result as performing it once. The term comes from mathematics: applying a function twice gives the same result as applying it once.

In HTTP, idempotency is defined per method:

Safe means no server-side state changes. Idempotent means repeating the operation does not change the final state beyond the first application.

Why POST Needs Idempotency Keys

Network failures are inevitable. When a client sends a POST and the connection drops before receiving a response, it faces a dilemma: was the request received?

Client → POST /api/charges { amount: 100 } → [Network drops] → ???

Option A: Request never reached server → retry is safe
Option B: Request reached server, response lost → retry creates duplicate charge

Without idempotency keys, clients must choose between:

• Under-delivering (never retrying, leaving users without confirmation)
• Double-charging (retrying without protection)

Idempotency keys solve this by letting clients safely retry POST requests.

Idempotency Key Implementation

Client-Side: Generating and Sending Keys

The client generates a unique key per logical operation and includes it in a header:

import uuid
import httpx

def create_charge(amount: int, currency: str) -> dict:
    idempotency_key = str(uuid.uuid4())

    # Store key locally before making request
    pending_operations[idempotency_key] = {'amount': amount, 'status': 'pending'}

    response = httpx.post(
        'https://api.example.com/charges',
        json={'amount': amount, 'currency': currency},
        headers={'Idempotency-Key': idempotency_key},
    )
    return response.json()

Key rules for clients:

• Generate a new key per logical operation (not per request attempt)
• Reuse the same key when retrying a failed request
• Use UUID v4 (random) or UUID v7 (time-ordered) for uniqueness
• Do not reuse keys across different operations

Server-Side: Key Storage and Response Replay

import json
from django.core.cache import cache
from django.http import JsonResponse

IDEMPOTENCY_KEY_TTL = 86400 * 7  # 7 days

def charge_endpoint(request):
    idempotency_key = request.headers.get('Idempotency-Key')

    if idempotency_key:
        cache_key = f'idempotency:{request.user.id}:{idempotency_key}'

        # Check for existing result
        cached = cache.get(cache_key)
        if cached:
            stored = json.loads(cached)
            return JsonResponse(
                stored['body'],
                status=stored['status'],
                headers={'Idempotency-Key': idempotency_key,
                         'X-Idempotent-Replayed': 'true'},
            )

    # Process the charge
    result = process_charge(request.data)
    response_data = {'charge_id': result.id, 'status': 'success'}

    # Store result for future replay
    if idempotency_key:
        cache.set(
            cache_key,
            json.dumps({'status': 200, 'body': response_data}),
            IDEMPOTENCY_KEY_TTL,
        )

    return JsonResponse(response_data, status=200)

Key Scoping

Always scope idempotency keys by user/account to prevent cross-account key conflicts:

# Wrong — global key namespace
cache_key = f'idempotency:{idempotency_key}'

# Correct — scoped by authenticated user
cache_key = f'idempotency:{request.user.id}:{idempotency_key}'

TTL and Eviction

Choose TTL based on your retry window:

• Stripe: 24 hours
• Adyen: 72 hours
• General recommendation: 7-30 days (based on client retry patterns)

After the TTL expires, the server treats the key as new — old completed operations are no longer protected.

Database-Level Idempotency

For even stronger guarantees, persist idempotency keys in the database:

# models.py
class IdempotencyRecord(models.Model):
    key = models.CharField(max_length=255)
    user = models.ForeignKey(User, on_delete=models.CASCADE)
    response_status = models.IntegerField()
    response_body = models.JSONField()
    created_at = models.DateTimeField(auto_now_add=True)

    class Meta:
        unique_together = [('key', 'user')]  # DB-level uniqueness
        indexes = [models.Index(fields=['key', 'user'])]

Using unique_together means duplicate concurrent requests hit a database constraint — only one succeeds, the other gets an integrity error which the view handles by returning the existing result.

Upsert Patterns

For create operations, database upserts naturally provide idempotency:

# PostgreSQL ON CONFLICT DO NOTHING via Django
User.objects.get_or_create(
    email=email,
    defaults={'name': name, 'role': role},
)

# PostgreSQL ON CONFLICT DO UPDATE
User.objects.update_or_create(
    email=email,
    defaults={'name': name},
)

Real-World Examples

Stripe's Idempotency

Stripe pioneered idempotency key adoption for payment APIs:

curl https://api.stripe.com/v1/charges \
  -u sk_test_key: \
  -H 'Idempotency-Key: IkBJTGlzdE9mVXNlcnM=' \
  -d amount=2000 \
  -d currency=usd \
  -d source=tok_visa

Stripe stores the key for 24 hours and returns an X-Stripe-Should-Retry: false header when a request fails in a non-idempotent way (e.g., card declined).

FastAPI Dependency Injection

from fastapi import Header, Depends
from functools import lru_cache

async def idempotency_check(
    idempotency_key: str | None = Header(None, alias='Idempotency-Key'),
):
    if idempotency_key is None:
        return None

    cached = await redis.get(f'idem:{idempotency_key}')
    if cached:
        raise IdempotentReplayException(json.loads(cached))
    return idempotency_key

@app.post('/charges')
async def create_charge(
    data: ChargeRequest,
    idempotency_key: str | None = Depends(idempotency_check),
):
    result = await process_payment(data)

    if idempotency_key:
        await redis.setex(
            f'idem:{idempotency_key}',
            86400,
            json.dumps(result.model_dump())
        )

    return result

Common Pitfalls

Generating a new key per retry: The client must reuse the same key when retrying. A new key on each retry defeats the purpose entirely.

Using a non-unique key: Timestamps, sequential integers, or request parameters as keys risk collision. Always use UUID v4 or a CSPRNG-generated token.

Ignoring partial failures: If a charge succeeds but recording the result fails, the retry will replay correctly only if the storage write is atomic with the charge. Use database transactions to ensure both complete or neither does.