Idempotency

Conduit’s external-task API is at-least-once. A worker that crashes between starting the side effect and calling complete will see the task re-delivered to itself or another worker after the lock TTL. This is by design — exactly-once delivery on top of a network is a fairy tale, and no SDK can paper over it.

That means: every handler must be idempotent under retry. Two layers of guarantee compose to make the whole system “feel exactly-once”:

Orchestration durability (the engine, already shipped): every BPMN state change persists in PostgreSQL inside one transaction; lock TTLs cause crashed workers’ tasks to be reclaimed.
Side-effect idempotency (per handler, your responsibility): each handler is written so that running it twice produces the same outcome as running it once.

If you only ship layer 1, retries replay side effects and you charge customers twice. If you only ship layer 2, retries replay correctly but the orchestrator still loses tasks on crash. Both are necessary.

For the broader picture — including what Conduit deliberately does not ship (step-level workflow replay, à la Temporal) and how to compose with a Layer 3 runtime when you need it — see Durability Layers.

Per-handler strategies

Handler	Strategy
`http.call` (writes)	`Idempotency-Key` header derived from `task.id` (template configurable). For non-idempotent verbs the upstream service is expected to honour the header per the Idempotency-Key RFC draft / Stripe convention.
`http.call` (reads)	Naturally idempotent — `GET`/`HEAD` pass through unchanged.
`csv.read`	Naturally idempotent.
`csv.write`	Write to `{path}.tmp.{task_id}` and atomic-rename. Retry overwrites the temp file safely; the final rename is idempotent.
`gcs.read`	Pin to a generation when consistency matters; otherwise naturally idempotent.
`gcs.write`	`ifGenerationMatch=0` for create-only writes; generation-match for updates. Object versioning on the bucket gives a safety net for accidental overwrites.
`kafka.produce`	`enable.idempotence=true` on the client (in-session dedupe). For cross-restart dedupe, deterministic message key derived from `task.id`.

The `Idempotency-Key` pattern

The most common case: your handler calls a third-party API with a non-idempotent verb (POST, PATCH, DELETE). The upstream service supports the Idempotency-Key header — if you send the same key twice, the second call is a no-op and returns the cached response.

POST /v1/charges HTTP/1.1
Authorization: Bearer ...
Idempotency-Key: conduit-task-0a3f...     ← derived from task.id

The key MUST be deterministic from the task — task.id is a UUID generated by the engine and is stable across retries. Don’t derive the key from time.Now(), the worker’s hostname, or anything that changes between attempts.

When the upstream doesn’t support `Idempotency-Key`

Some APIs only offer “create only if not exists” semantics, or no idempotency at all. Two options:

Front it with a dedupe table. The reference HTTP worker maintains a small (task_id, attempt) → response table in Postgres. On retry, the handler checks the table before issuing the upstream call: if the attempt is already recorded, replay the stored response.
Wrap the call in a “create record locally first, then call upstream, then mark complete” sandwich. Equivalent to a dedupe table but inlined per handler. Use only if you can’t share a dedupe table across the fleet.

The dedupe-table schema is documented in workers/docs/idempotency-store.md. One store per worker fleet, not per process — the table’s durability across worker restarts is what makes the pattern safe.

Boundaries the SDK won’t cross

The SDKs deliberately do not ship a built-in dedupe store. Your handlers may not need one (most reads don’t, and most upstreams support Idempotency-Key), and bundling it would force a Postgres dependency on every worker process.

If you need a dedupe store, you wire it in yourself — typically as a struct field on a Handler impl, or a dependency-injected service in your container.

Crash testing

A handler claiming idempotency should be tested under crash. The pattern that catches the most bugs:

Start the engine + worker.
Trigger a BPMN that hits your task.
After the side effect lands but before complete returns, kill the worker (SIGKILL).
Restart the worker.
Confirm the side effect did not duplicate.

This is part of the verification checklist for every reference handler — see PHASE-21.