Blog · · 11 min read
AI Agent Concurrency Control: Prevent Conflicting Product Writes

AI agent concurrency control breaks when two valid runs read one product record and act on separate stale copies. One agent closes a ticket while another escalates it. A parallel tool batch updates the same account twice. The last write wins even though neither operation was a duplicate. The product boundary, rather than the prompt, must settle the collision. Classify actions by whether they may overlap, attach the observed record version to proposed writes, serialize only work that can collide, and return typed conflicts. This keeps useful parallelism without one global lock slowing every tenant and workflow.
Use five concurrency invariants
A production implementation can enforce five rules:
- Every mutable tool declares what resource it can change and whether its effect is commutative.
- Every state-dependent write carries an
expected_versionsupplied by trusted product code. - The product service applies the write only when the current version still matches.
- Work is serialized by the smallest conflict key, never by the entire agent process.
- A conflict returns to the runtime as data, not as hidden success or a blind retry.
These rules cover separate agent runs, branches inside one run, delayed approvals, callbacks, and ordinary human edits. Idempotency still protects duplicate delivery and retry recovery. It cannot choose between two different actions based on the same old state.
Why agent race conditions survive normal safeguards
Parallel tool calls create real overlap
A model can request several functions in one turn. OpenAI's function-calling documentation states that the model may call multiple functions. Setting parallel_tool_calls: false limits a turn to zero or one call. That switch is useful during an incident, but it does not amount to a concurrency policy. Two HTTP requests, background runs, or approval resumes can still overlap when each model turn invokes only one tool.
Some calls belong in parallel. Reading a ticket while loading an account policy is safe if both sources can handle the load. Independent audit facts may be mergeable. Changing the ticket owner while closing the same ticket is different because either decision may depend on the previous owner or status.
A valid retry key does not resolve a conflicting decision
An idempotency key can answer, "Have I already applied this operation?" It cannot answer whether the state used to choose that operation is still current. Consider two distinct actions:
- Run A reads ticket version 17 and proposes
assign_to=payments. - Run B reads ticket version 17 and proposes
status=resolved.
The operations have different action IDs, so deduplication accepts both. If Run B resolves the ticket first, Run A's routing decision may no longer be valid. A version precondition detects that change; a conflict rule decides what happens next.
Agent state can race before product state does
Runtime storage has its own races. An OpenAI Agents SDK issue about concurrent session initialization reports that two first operations on one session created separate conversations, split the history, and left one write inaccessible. A current LangGraph checkpoint issue reports that a non-atomic read, filter, assign, and extend sequence can silently lose pending writes during parallel execution. Both are author reports rather than guarantees about every deployment. The examples still expose three separate collision domains: session history, checkpoints, and product records.
Build the AI agent concurrency control contract
1. Classify every tool by conflict behavior
Store concurrency metadata beside each tool's authorization and effect metadata. The host can enforce categories such as these:
read_only: no mutation, safe to run in parallel when the source can handle the load.append_commutative: independent facts can merge under a defined reducer or unique key.aggregate_write: changes one product aggregate and requires an expected version.exclusive_workflow: a multi-step transition must hold one aggregate-scoped execution lane.external_irreversible: the product must combine conflict checks with idempotency and an outcome-recovery procedure.
Tool owners assign the category from the product invariant; the model does not. A tool named update_ticket is too broad when its fields have different conflict rules. Split the capability, or make the product service evaluate each proposed transition against current state.
2. Derive the conflict key from trusted identity
The conflict key identifies the smallest aggregate whose invariant could break. One useful shape is:
tenant_id + resource_type + resource_id
A subscription change should lock one subscription rather than every subscription in the tenant. Moving inventory across two warehouses may require two resource keys. Acquire multiple keys in a stable order to avoid deadlocks.
Never accept a complete lock key from model output. Resolve the tenant from authenticated authority and the canonical resource identity inside the tool adapter. Without that normalization, a malformed or hostile call could lock another tenant's work or bypass serialization by spelling the same resource differently.
3. Carry a version from observation to mutation
When context assembly loads a mutable record, retain its database version, ETag, or equivalent change token. The host injects that value into the proposed action. The model can describe the desired change, but it cannot invent the precondition.
HTTP already has this pattern. RFC 9110's If-Match precondition prevents a lost update by applying a state-changing method only when the current entity tag matches. An internal tool can enforce the same rule even if the product does not expose HTTP ETags publicly.
A versioned write can use one atomic database statement:
run proposes assign_ticket(ticket_id, queue_id)
host injects tenant_id, actor_id, expected_version, action_id
product service:
authorize actor for ticket and queue
begin transaction
update tickets
set queue_id = requested_queue,
version = version + 1
where tenant_id = trusted_tenant
and id = canonical_ticket_id
and version = expected_version
if affected_rows == 0:
rollback
return conflict(current_version, changed_fields)
insert outbox event keyed by action_id
commit
return ok(new_version)
Keep the update and outbox insert in the same transaction. Keep the model call outside it. A transaction held open while the model reasons creates long lock times and makes recovery harder.
4. Serialize only work that cannot use a conditional write
Use optimistic version checks when conflicts are uncommon and a rejected write can restart from fresh state. Some operations still need a short exclusive lane. Allocating the last unit of inventory is one example. So is a state transition across several records or an external operation that cannot accept a version precondition.
Use a queue per aggregate, a database advisory lock, or a leased lock with a fencing token. Limit the critical section to current-state validation and the deterministic product transition. Never hold the lease through unrestricted model reasoning. If reasoning must happen first, create a proposal without the lease, then acquire the lane and revalidate immediately before commit.
A LangGraph issue about an instance-wide asynchronous checkpoint lock reports that the lock prevented local races but serialized unrelated traffic and raised latency under load. The measurements are practitioner reported. Regardless of the exact figures, the lock followed the process boundary instead of the resources that could collide. Product locks should do the opposite.
5. Define merge rules for parallel state
Parallel branches can collide in shared run state before either branch reaches a product tool. A list of independent findings can append. A scalar such as selected_account, approved_amount, or next_state cannot accept two arbitrary values safely.
LangGraph's concurrent graph update guidance rejects ambiguous updates to one state property and uses an explicit reducer for values that can combine. The same constraint works outside LangGraph. Give every shared field one declared policy: single writer, deterministic reducer, compare-and-swap, or conflict. "Whichever finishes last" leaves the result to timing.
Session and product serialization need separate controls. A session may require ordered message appends while its read tools run in parallel. Two unrelated sessions can still collide on the same ticket. One lock cannot represent both boundaries.
Return typed conflict outcomes
The tool layer should expose enough information for policy and recovery without leaking unauthorized state. Use outcomes such as these:
ok: the write committed at a new version.conflict: the expected version was stale and the action was not applied.busy: an aggregate-scoped lane could not be acquired within its deadline.superseded: current state makes the requested work unnecessary.policy_denied: current authority no longer allows the transition.unknown: an external call may have succeeded but its outcome cannot yet be proven.
Do not turn conflict into an automatic retry with the same arguments. Reload the record, identify the relevant changes, and apply a declared rule:
- Re-run planning when the change can alter the decision.
- Merge deterministically only for fields the product defines as commutative.
- Request approval again when the approved proposal depended on the old version.
- Escalate to a human when two valid intents cannot be reconciled safely.
PostgreSQL's transaction-isolation documentation explains serialization anomalies and says applications using Serializable transactions must be prepared to retry failed transactions. Such a retry reruns deterministic transaction logic against current data. An agent decision may need fresh planning too, rather than another call with the old tool arguments.
Work through two overlapping ticket runs
A support product starts Run A when a ticket breaches its SLA. Seconds later, a customer reply starts Run B. Both runs observe ticket version 42.
Run A proposes moving the ticket to the escalation queue. Run B proposes changing the status to waiting_for_agent and assigning the responding specialist. Run B commits first and creates version 43. Run A then calls the assignment tool with expected_version=42.
The product service updates zero rows and returns conflict. Its response includes only the current version and the fact that owner and status changed. The runtime does not repeat Run A's old assignment. It reloads the ticket, sees the specialist response, and marks the SLA escalation superseded. Both proposals and the conflict remain in the causal history, but the product commits only one transition.
Independent internal notes need a different policy. The tool registry can classify those writes as commutative, give each note a stable ID, and commit both without serializing the ticket. Safe overlap remains available.
Handle failure and recovery
A lease may expire while its worker is paused. Include a monotonically increasing fencing token in every protected write so that a stale worker cannot commit after a newer lease holder. If renewal fails, the worker has lost ownership and must stop issuing writes. Verify any external outcome separately.
A process can crash after an external side effect and before it records success. Version checks cannot prove whether the email, payment, or vendor mutation happened. Use the action's idempotency key, query the provider when possible, and return unknown until reconciliation proves the outcome. A missing worker is not enough reason to release a conflicting action.
Set a ceiling on conflict loops. If a hot aggregate changes faster than the agent can plan again, stop after a bounded number of attempts and show the competing activity to an operator. Random backoff reduces load; it does not reconcile incompatible intent.
History and checkpoints also need protection from partial writes. Initialize one session identity atomically. Append ordered units under that session's sequence rule, then persist checkpoint changes through transactions or compare-and-swap. A correct product write cannot repair a corrupted decision history.
Verify concurrency before production
Write deterministic tests that force the dangerous interleavings. A load test that happens not to reproduce the race proves little.
- Pause two runs after both read version 10, commit one write, then prove the other returns
conflictwithout changing state. - Run two commutative appends together and prove both facts remain with distinct IDs.
- Start work for two tenants that share the same resource ID and prove their conflict keys never collide.
- Hold one aggregate lane and prove unrelated aggregates still meet the normal latency target.
- Expire a lease, issue a newer fencing token, and prove the stale worker cannot commit.
- Change a record while approval waits and prove the old approval cannot authorize the new version.
- Crash after dispatching an external action and verify reconciliation uses its action ID before any retry.
- Run parallel graph branches against one scalar field and prove the runtime invokes a reducer or returns a conflict.
- Initialize one session from two concurrent requests and prove both operations use one retained session identity.
- Generate repeated conflicts on a hot record and prove the retry ceiling routes the case to a visible recovery path.
Track conflict rate by tool and aggregate type, lock-wait duration, stale-approval rejection, retries per successful action, unknown external outcomes, and throughput for unrelated keys. A rising rate can point to an aggregate that is too broad. It can also expose an event source creating needless overlap or a tool contract missing a commutative operation.
Avoid common concurrency mistakes
A global mutex around the agent runtime hides local races by removing useful parallelism. Unless shared infrastructure backs it, that mutex does not coordinate other processes either.
Avoid timestamps as record versions when the storage layer provides a monotonic revision. Limited clock precision, equal timestamps, and skew between systems produce weak preconditions.
Do not ask the model whether a stale write is safe. The product detects the conflict first. The model may create a new proposal after the runtime loads current state and policy.
Do not retry every database conflict with the same tool arguments. A transaction retry repeats deterministic logic against a fresh snapshot. A decision retry may require another model turn and renewed approval.
Do not make every write exclusive. Defined merge rules, unique append IDs, and conditional updates preserve concurrency where the product's invariants permit it.
Start with one aggregate
Start AI agent concurrency control with the product record most likely to receive overlapping work. Add a monotonic version and classify its tools. Inject expected_version into writes, return a typed conflict, and write the forced-interleaving test before enabling another parallel path. Measure whether the lock or queue blocks unrelated records.
AI agent concurrency control should make each collision end in a valid merge, one accepted transition, or a visible conflict that causes no unintended write. It should not eliminate concurrency that the product can handle safely.
References
- OpenAI function calling supports the behavior and control of parallel function calls.
- RFC 9110: If-Match defines conditional state-changing requests used to prevent lost updates.
- PostgreSQL transaction isolation supports the explanation of concurrent transaction anomalies, Serializable behavior, and retry requirements.
- LangGraph INVALID_CONCURRENT_GRAPH_UPDATE supports explicit reducers for state keys updated by parallel graph nodes.
- OpenAI Agents SDK issue #3820 provides an author-reported reproduction of concurrent session initialization splitting conversation history.
- LangGraph issue #8115 provides an author-reported reproduction of a checkpoint write race causing silent data loss.
- LangGraph issue #7259 provides practitioner-reported evidence that an instance-wide lock can serialize unrelated checkpoint traffic and increase latency.