id: 0007-at-least-once-with-idempotency-keys
type: decision
owner: justin
status: accepted
reviewed: 2026-10-07

0007 At-least-once delivery, made safe with the order id

Context

Between a payment and a shipment there are three hops that can each be repeated: Stripe redelivers webhooks, the queue redelivers messages, and fulfilment retries the warehouse call. Exactly-once delivery is not on offer from any of them.

Decision

Accept repeats everywhere and make every step idempotent by order id:

• order state changes only move forward from a given status ([[payment-webhook]]), so a repeat changes nothing;
• order.paid may be sent more than once ([[order-events]]);
• fulfilment checks the order's status before acting and sends Idempotency-Key: order-<id> to the warehouse, which returns the existing shipment for a key it has seen ([[fulfilment]]).

Consequences

• No dedupe table and no distributed locks.
• Two copies of the same event handled at the same moment can both reach the warehouse. The idempotency key makes the second call return the first shipment, and recordShipment only applies once.
• This depends on the warehouse honoring idempotency keys for at least as long as our retries last (about two hours). Theirs is 24 hours.