Measure twice, cut once. Once is enough.

Measure twice, cut once. Once is enough.
Photo by ERIK SETH / Unsplash

Picture this: a customer submits a warranty claim for their damaged goods at exactly the wrong moment. The request hits your server, the refund voucher is issued, but the response times out before it reaches the browser. The browser shows an error. The customer submits again. The voucher is issued _again_. Two credits land in their account. One very confused customer. One support escalation that shouldn't exist.


The fix is idempotency - and it's not just about slapping a unique key on a request and calling it done. Idempotency is end-to-end. It has to survive the HTTP layer, the job queue, the downstream API calls, and the database. Every hop in the chain is a place it can break.


This note to myself from the tranches of building idempotent endpoints in a production system. Couple concrete techniques, drawn from building a helpdesk service for customer support. The service processes warranty claims, coordinates refunds, and integrates with data store and AI models across multiple async job queues - every hop a place where a duplicate operation means a duplicate voucher, a duplicate refund, or a date store record in an inconsistent state.

What Idempotency Actually Means

An operation is idempotent if calling it N times produces the same result as calling it once. The second, third, and tenth call should be no-ops - not errors, not duplicates, just silent successes.
For HTTP endpoints, this matters in three situations:
- Network retries - the client sends again because it never got a response
- Client timeouts - the browser retries after a spinner-of-death
- Duplicate submissions - the user clicks the button twice
The standard mechanism is the Idempotency-Key header. The client generates a unique key (typically a UUID) and sends it with every request. If the server has already processed a request with that key, it returns the same result without re-executing the operation. The key is the contract between client and server.

Technique 1 - Require the Header at the Entry Point
The first thing your endpoint should do is demand the Idempotency-Key header. Not optionally accept it - _demand_ it. If it's missing, reject the request immediately.

idempotencyKey := c.Request().Header.Get("Idempotency-Key")

if idempotencyKey == "" {
    return echo.NewHTTPError(http.StatusBadRequest, "Idempotency-Key header is required")
}

Making the key mandatory is a deliberate choice. If you make it optional, clients will forget to send it, and you lose all the safety guarantees downstream. A 400 at the door is better than a duplicate charge in production.

Technique 2 - Derive Per-Entity Keys for Batch Operations
A single HTTP request often contains multiple entities. In the warranty claim system, one POST /warranty-claim might carry a dozen claims. You have one request-level key, but you need a unique key per claim - because each claim will be processed independently downstream.
The solution is deterministic key derivation: concatenate the request key with the entity's own ID.

for _, claim := range request.Claims {
    switch claim.Status {
    case models.ClaimStatusApproved:
        claimIdempotencyKey := idempotencyKey + claim.ClaimID
        event := events.NewClaimApproved_v1WithIdempotencyKey(
            claim.ClaimID,
            claim.CustomerEmail,
            claim.Price,
            claimIdempotencyKey,
            claim.BookingID,
        )
        h.eventBus.Publish(c.Request().Context(), event)
    case models.ClaimStatusRejected:
        claimIdempotencyKey := idempotencyKey + claim.ClaimID
        // ...
    }
}

The key property here is determinism: given the same request key and the same claim ID, you always get the same derived key. So if the client retries the request, the handler will derive the exact same per-claim keys, and every downstream operation will recognize them as duplicates.

The anti-pattern to avoid: deriving the key using any non-deterministic input (timestamps, random values). If the derived key changes on retry, you've broken idempotency.

Technique 3 - Embed the Key in Every Message Header
The request handler is not the end of the story. In an event-driven system, processing continues in handlers that consume events from a message bus. Those handlers call external services. Those services call others. The idempotency key needs to survive every hop.
The way to carry it is to embed it in the message header itself. Every event and command in the project has a MessageHeader with an IdempotencyKey field:

type MessageHeader struct {
    ID             string    `json:"id"`
    PublishedAt    time.Time `json:"published_at"`
    IdempotencyKey string    `json:"idempotency_key"`
}

func NewMessageHeader() MessageHeader {
    return MessageHeader{
        ID:             uuid.NewString(),
        PublishedAt:    time.Now().UTC(),
        IdempotencyKey: uuid.NewString(),
    }
}

func NewMessageHeaderWithIdempotencyKey(idempotencyKey string) MessageHeader {
    return MessageHeader{
        ID:             uuid.NewString(),
        PublishedAt:    time.Now().UTC(),
        IdempotencyKey: idempotencyKey,
    }
}

The two-constructor pattern is intentional:
- NewMessageHeader() - for events generated internally, where no external key exists. Auto-generates a UUID.- NewMessageHeaderWithIdempotencyKey(key) - for events that trace back to an HTTP request. Carries the derived key through.
The event struct uses the explicit constructor when it originates from an HTTP request:


func NewWarrantyClaimConfirmed_v1WithIdempotencyKey(
    claimID, customerEmail string,
    price models.Money,
    idempotencyKey string,
    bookingID string,
) WarrantyClaimConfirmed_v1 {
    return WarrantyClaimConfirmed_v1{
        Header:        NewMessageHeaderWithIdempotencyKey(idempotencyKey),
        ClaimID:      claimID,
        CustomerEmail: customerEmail,
        Price:         price,
        BookingID:     bookingID,
    }
}

Without this, the key dies at the controller and handlers are left to their own devices.

Technique 4 - Forward the Key to External Services
When a handler needs to call an external API (receipts service, transportation API), it should extract the idempotency key from the event header and pass it along. This makes the external call idempotent too, not just the local processing.
Here's the receipts adapter doing exactly that:


func (c ReceiptsServiceClient) IssueReceipt(
    ctx context.Context,
    request events.WarrantyClaimConfirmed_v1,
) (entities.IssueReceiptResponse, error) {
    idempotencyKey := request.Header.IdempotencyKey
    resp, err := c.clients.Receipts.PutReceiptsWithResponse(ctx, receipts.CreateReceipt{
        IdempotencyKey: &idempotencyKey,
        ClaimId:       request.ClaimID,
        Price: receipts.Money{
            MoneyAmount:   request.Price.Amount,
            MoneyCurrency: request.Price.Currency,
        },
    })
    // ...
}

And the refund client does the same for vouchers and money refunds:

resp, err := c.clients.Refund.PutRefundWithResponse(ctx, refund.CreateRefundRequest{
    CustomerEmail:  customerEmail,
    RefundId:       refundUUID,
    ReferenceId:    referenceID,
    IdempotencyKey: idempotencyKey,
})

The pattern is always the same: pull event.Header.IdempotencyKey, pass it into the external request body or header. If the handler is retried (at-least-once delivery means it will be), the external service sees the same key and treats it as a duplicate. No double claim, no double charge.


Technique 5 - In-Memory Deduplication
Sometimes you don't need a database. For simple, single-instance services or test environments, an in-memory map does the job:

type PaymentsRepository struct {
    processedIDs map[string]struct{}
    payments     []PaymentTaken
}

func (p *PaymentsRepository) SavePaymentTaken(ctx context.Context, event *PaymentTaken) error {
    if _, exists := p.processedIDs[event.PaymentID]; exists {
        return nil // already processed — skip silently
    }

    p.processedIDs[event.PaymentID] = struct{}{}
    p.payments = append(p.payments, *event)

    return nil
}

The check-before-insert pattern is simple and effective: if we've seen this ID, return `nil` (not an error). The caller has no idea a duplicate arrived - from their perspective, the operation succeeded.
When to use it:
- Single-instance services where state doesn't need to survive restarts
- Test helpers and stubs
- Prototyping
When NOT to use it:
- Multi-instance deployments - each pod has its own map, so a request handled by pod A will be re-processed by pod B
- Services that restart frequently
- the map is lost on restart

Technique 6 - Database-Level Idempotency with ON CONFLICT DO NOTHING
For production, multi-instance services, the database is the right place to enforce uniqueness. PostgreSQL's ON CONFLICT DO NOTHING makes an insert idempotent at the SQL level:

_, err := t.db.ExecContext(
    ctx,
    `INSERT INTO warrantyClaim (claim_id, price_amount, price_currency, customer_email, confirmed_at, refunded_at, deleted_at)
     VALUES ($1, $2, $3, $4, $5::TIMESTAMPTZ, $6::TIMESTAMPTZ, $7::TIMESTAMPTZ)
     ON CONFLICT DO NOTHING`,
    claim.ClaimID,
    claim.Price.Amount,
    claim.Price.Currency,
    claim.CustomerEmail,
    confirmedAt,
    refundedAt,
    deletedAt,
)

If the claim already exists (same claim_id), the insert does nothing - no error, no duplicate row. The handler can be retried as many times as needed.
This is the most reliable deduplication technique because:
- It survives service restarts
- It works across any number of instances
- The database guarantees atomicity
The catch: ON CONFLICT DO NOTHING only protects the insert. It doesn't deduplicate the side effects that happen _around_ the insert (sending emails, calling external APIs). That's why techniques 4 and 5 still matter - you need idempotency at every layer, not just the database.

The Full Flow
Put it all together and the idempotency chain looks like this:

Common Mistakes
Not propagating the key through the message chain. If you read the key in the controller but don't embed it in the event header, handlers have no key to work with. Every retry creates a fresh operation.
Generating a new UUID on every request.The idempotency key must come from the _client_, not the server. If the server generates a new UUID on each call, retries look like brand-new requests.
Assuming `ON CONFLICT DO NOTHING` covers side effects.It makes the insert a no-op. It does nothing about the email you sent, the external API you called, or the webhook you fired before the insert. Those still need their own idempotency mechanisms.
Making the key optional. Clients under pressure (deadlines, copy-paste code) will skip optional headers. Make it required and return 400 when it's missing. The friction is worth it.
Using non-deterministic key derivation. If you derive per-entity keys using a timestamp or `rand.String()`, retries will produce different keys and bypass all your deduplication logic.

Sources
- Stripe API documentation - idempotent requests
- Oskar Dudycz - idempotency in event-driven systems
- Three Dots Labs - go-event-driven training