Idempotency Keys for Safe API Retries

The Double-Submit Problem

Consider a payment API. The client sends a charge request, but the network drops before the response arrives. The payment processed — the client just never got the confirmation. What does the client do?

• Don't retry: The user thinks the payment failed. They try again manually. Now they're charged twice.
• Retry blindly: The server processes two payments. The user is charged twice.

Idempotency keys solve this by allowing safe retries of mutating requests.

What Are Idempotency Keys?

An idempotency key is a unique client-generated identifier sent with every mutating request. The server uses this key to detect duplicate requests and return the same response as the first successful attempt, without re-executing the operation.

POST /charges HTTP/1.1
Idempotency-Key: a4e1b2c3-d4e5-6789-abcd-ef0123456789
Content-Type: application/json

{"amount": 5000, "currency": "usd", "customer": "cus_abc123"}

If the client retries with the same key, the server returns the original response — no duplicate charge.

Server-Side Implementation

Step 1: Storage

Store idempotency records with: key, request fingerprint, response status, response body, and creation timestamp.

from django.db import models

class IdempotencyRecord(models.Model):
    key = models.CharField(max_length=255, unique=True)
    request_fingerprint = models.CharField(max_length=64)
    response_status = models.IntegerField()
    response_body = models.JSONField()
    created_at = models.DateTimeField(auto_now_add=True)

Step 2: Request Handling

def charge_view(request):
    idempotency_key = request.headers.get('Idempotency-Key')
    if not idempotency_key:
        return HttpResponse(status=400, content='Idempotency-Key required')
    # Check for existing record
    existing = IdempotencyRecord.objects.filter(
        key=idempotency_key
    ).first()
    if existing:
        return JsonResponse(existing.response_body,
                           status=existing.response_status)
    # Process the charge
    result = process_charge(request.data)
    # Store the result
    IdempotencyRecord.objects.create(
        key=idempotency_key,
        response_status=200,
        response_body=result,
    )
    return JsonResponse(result)

Step 3: TTL

Idempotency records do not need to live forever. Stripe keeps them for 24 hours. Choose a TTL that covers your retry window:

# Django management command to clean up expired records
from django.utils import timezone
from datetime import timedelta

IdempotencyRecord.objects.filter(
    created_at__lt=timezone.now() - timedelta(hours=24)
).delete()

Step 4: Concurrency

Two concurrent requests with the same key (a network retry arriving while the original is still processing) must be handled carefully:

• Use a database unique constraint on the key
• Use SELECT FOR UPDATE or Redis lock during processing
• Return 409 Conflict if a request with the same key is in-flight

Client-Side Best Practices

• Generate keys client-side: Use UUID v4 — never rely on the server
• Use one key per logical operation: Don't reuse keys across different operations
• Retry on network errors and 5xx: Do NOT retry on 4xx (the request itself is wrong)
• Include the key in logs: Essential for debugging 'did it process?' questions

Stripe Case Study

Stripe popularized the pattern. Their implementation:

• Header: Idempotency-Key: <your-key> (required for POST requests)
• TTL: 24 hours
• Key mismatch: If the same key is sent with different request bodies, Stripe returns 422 Unprocessable Entity
• Response: The original response is replayed exactly, including headers

Status Codes

Summary

Idempotency keys are essential for any API that accepts payments, sends emails, or creates resources. Require the Idempotency-Key header on all POST endpoints that have side effects, store results with a TTL, handle concurrent requests with database locks, and validate that the key is not reused with different request bodies.