Add idempotency key support for task deduplication

[AntoineToussaint]

Summary

Two new fields on SpawnOptions for controlling task deduplication:

• only_once: bool — auto-derives idempotency key from hash(task_name, params). First spawn creates the task, duplicates are no-ops.
• idempotency_key: Option<String> — explicit key for when params differ but the operation is the same. Takes precedence over only_once.

Default behavior (only_once: false, no key) is unchanged — every spawn creates a new task.

Motivation

In multi-instance deployments, multiple clients may observe the same event and independently try to spawn the same logical task. Without dedup, N instances create N tasks that all execute. This is wasteful for expensive operations and incorrect for non-idempotent ones.

DB Changes

• New nullable idempotency_key TEXT column on task tables
• Partial unique index WHERE idempotency_key IS NOT NULL
• spawn_task checks for existing non-terminal task with same key before inserting
• Migration adds column/index to all existing queues
• ensure_queue_tables updated for new queues

Design doc

See docs/design/durable-idempotency-key.md in the autopilot repo.

Test plan

• Spawn with only_once: true — second spawn returns existing task
• Spawn with explicit idempotency_key — same behavior
• Spawn without key — creates new task (backward compatible)
• Completed/failed/cancelled task with same key — new spawn succeeds
• Existing tests pass

🤖 Generated with Claude Code