Blog · · 10 min read

Mid-conversation tool changes for product agents without cache misses or stale privileges

Mid-conversation tool changes for product agents without cache misses or stale privileges

Your support agent verified the customer, then needed issue_refund. Mutating the top-level tools array re-bills the entire cached prefix. Leaving refund available from turn one lets the model call it before verification. A long product session needs neither. This guide implements mid-conversation tool changes as a product-owned capability lifecycle: bind authenticated product events to a ledger, keep the cached tools prefix byte-stable, refuse withdrawn tools at dispatch, and audit which tools were live when a side effect ran.

Direct answer: the capability lifecycle

Use this contract instead of editing prompts ad hoc:

  1. Map product events to capabilities. Identity verified, current page or CRM record, approval granted, plan entitlement, and operator mode each produce a typed delta: offer, withdraw, or leave unchanged.
  2. Declare the candidate catalog once. Put every tool the session might need in the first request. Mark tools that start withheld with defer_loading where the provider supports it (Claude mid-conversation tool changes).
  3. Offer and withdraw with prefix-safe messages. Append tool_addition or tool_removal after a user or tool-result turn so the hashed tools and system prefix stays identical (Dreaming Press on tools-prefix cache invalidation).
  4. Enforce at dispatch. If the model names a withdrawn tool, your product runtime must refuse before any side effect. A removal block is a model hint, not a security boundary.
  5. Record a live-tool receipt. Persist the capability ledger generation and offered tool names beside every consequential action.
  6. Keep an explicit fallback. When the active model lacks mid-conversation tool changes, either accept one deliberate cache miss, start a new run, or keep the dangerous tool withheld and finish through a human path.

Why product sessions break fixed tool lists

Three pressures collide in an existing SaaS agent.

Product state changes during the run. Refund tools should appear after identity verification. Write tools should appear when the user opens a mutable record. Destructive tools should disappear after the approved step finishes. Leaving every schema offered for the whole conversation creates stale privileges; mutating the array recreates the cache problem below (Get Claude Skills on deferred unlock and post-use withdrawal).

Prompt caching hashes tools first. On Claude, the request renders tools, then system, then messages. A one-byte change in tools invalidates tools, system, and every cached turn after them. For agents with large catalogs, that miss is often the most expensive transition in the session (Dreaming Press; Claude Platform docs).

Frameworks already feel the pain. Goose reports that enabling or disabling an extension mid-session rewrites the extensions section of system and grows or shrinks tools in place, busting cache and dropping preserved thinking on Anthropic models that bind signatures to the exact prefix (aaif-goose#11905). Pydantic AI measured that deferred tool definitions sit outside Anthropic's cache key and proposed lazy advertisement so capability-gated tools are not pre-searchable by native tool search (pydantic-ai#7087).

Distinguish control from discovery

Treat OpenAI tool search as discovery and Claude tool_addition as control; product events need the control path.

PrimitiveWho decidesTypical useCache behavior
Tool search / deferred corpusThe model finds tools from a libraryHundreds of optional schemasSchemas append without rewriting the stable tools prefix when designed correctly (OpenAI tool search)
Mid-conversation tool changesYour application states the set changedIdentity unlock, approval grant, mode switchtools[] stays byte-identical; change rides in messages (Dreaming Press)
Mid-turn steeringYour application updates instructionsPolicy nudge while tools continueInstructions change without canceling in-flight tools (OpenAI mid-turn steering)

The host already knows the customer is verified. Tool search would ask the model to rediscover issue_refund, and pre-advertising the whole deferred pool into a searchable corpus can surface capability-gated tools early (pydantic-ai#7087).

Build the CapabilityLedger

Keep one durable ledger per agent run.

CapabilityLedger {
  run_id
  generation            # increments on every applied delta
  offered[]             # tool names currently callable
  deferred[]            # declared but withheld
  withdrawn[]           # previously offered, now refused
  events[] {
    at
    product_fact        # identity_verified | record_opened | approval_granted | ...
    delta               # offer | withdraw
    tools[]
    actor               # system | user | approver
    evidence_ref        # verification receipt, approval id, entitlement check
  }
}

Event matrix for an existing product

Product factExample deltaNotes
Identity verifiedOffer issue_refund, update_billing_addressKeep verification evidence on the ledger event
Record opened (ticket 8821)Offer update_ticket_8821, withdraw unrelated write toolsScope tools to the active aggregate when possible
Approval granted for destroyOffer delete_workspace onceWithdraw immediately after success or expiry
Plan entitlement missingWithdraw premium toolsPrefer entitlement checks in your API too
Operator ends elevated modeWithdraw deploy / admin toolsDo not rely on the model to forget them

Authorization still belongs to your per-call policy. The ledger answers a different question: which schemas the model is allowed to see and name right now.

Implement prefix-stable mid-conversation tool changes

1. Declare once, defer what starts hidden

tools = [
    {
        "name": "search_orders",
        "description": "Look up a customer's orders.",
        "input_schema": {
            "type": "object",
            "properties": {"email": {"type": "string"}},
            "required": ["email"],
        },
    },
    {
        "name": "issue_refund",
        "description": "Issue a refund after identity verification.",
        "input_schema": {
            "type": "object",
            "properties": {"order_id": {"type": "string"}},
            "required": ["order_id"],
        },
        "defer_loading": True,
    },
]

Declare tools you know at request start. Keep at least one non-deferred tool in tools when you may later define tools by value, so the first definition does not rewrite the start of the rendered prompt (Claude Platform docs).

2. Place additions and removals after tool results

In an agentic loop, append the system message after the user turn that delivers tool_result blocks:

[
  {"role": "user", "content": "Verify this customer, then refund order 991."},
  {"role": "assistant", "content": [{"type": "tool_use", "id": "toolu_1", "name": "verify_identity", "input": {}}]},
  {"role": "user", "content": [{"type": "tool_result", "tool_use_id": "toolu_1", "content": "verified"}]},
  {
    "role": "system",
    "content": [
      {
        "type": "tool_addition",
        "tool": {"type": "tool_reference", "name": "issue_refund"}
      }
    ]
  }
]

Placement rules matter. A content-bearing system message must follow a user turn (including tool results) or an assistant turn ending in a server tool result, and must not sit between tool_use and its tool_result (Claude Platform docs). Goose's proposed fix mirrors the same idea: leave system and existing tools entries byte-identical after the first assistant turn, and describe the change with tool_addition / tool_removal (aaif-goose#11905).

3. Prefer lazy advertisement for capability-gated tools

If your provider allows appending a deferred tool in the same request as its first tool_addition, keep hidden capabilities off the wire until the product event fires. That prevents native tool search from surfacing a refund tool before verification (pydantic-ai#7087). When the provider requires the name to already exist in tools, declare it deferred from turn one and never graduate it into a loaded entry that would mutate the hashed prefix.

4. Toggle MCP toolsets as phases change

For connected MCP servers, reference a whole toolset by server_name when a workflow phase needs a different external system. Keep URLs and tokens in mcp_servers, never inside the addition block (Claude Platform docs; Get Claude Skills).

Enforce withdrawn tools at the product boundary

Refuse withdrawn tools at your product dispatch boundary; a tool_removal block is not an enforcement guarantee.

A tool_removal block tells the model the tool is no longer offered. It does not guarantee the model will never emit that name, and it does not stop a buggy client from replaying an old tool call. Your product dispatch layer must:

  1. Resolve the current CapabilityLedger.generation.
  2. Reject any tool name in withdrawn or absent from offered.
  3. Re-check ordinary authorization (user, tenant, record, approval).
  4. Execute only after both checks pass.
  5. Attach the live-tool receipt to the action audit row.
on tool_call(name, args, run_id):
  ledger = load_ledger(run_id)
  if name not in ledger.offered:
    return ToolRefusal(code="capability_not_offered", generation=ledger.generation)
  if not authorize(user, tenant, name, args):
    return ToolRefusal(code="unauthorized")
  result = execute(name, args)
  audit.write(run_id, name, args, result, live_tools=ledger.offered, generation=ledger.generation)
  return result

Treat the model-visible catalog as UX and token economics. Treat dispatch refuse as the security boundary. A hidden button in a UI is not a lock; neither is a withdrawn schema.

Persist live-tool receipts

Persist a live-tool receipt on every consequential action so audits know which capabilities were offered when the side effect ran.

When an incident asks "was refund available when the agent charged the card?", the transcript alone is a weak answer. Persist:

  • capability_generation
  • offered_tools[] (or a content hash of that set)
  • product_facts[] that justified the last deltas
  • provider channel used (tool_addition, tool search, fallback mutate)

Without that receipt, you cannot prove whether a side effect happened under an elevated mode that should already have ended. Multi-step product flows need that proof more than single-turn chat demos do.

Handle failure and recovery

FailureProduct response
Provider rejects tool_addition (unsupported model)Route to fallback: withhold the tool and finish via human approval, or start a new run with the expanded catalog declared up front (Get Claude Skills model limits)
Model calls a withdrawn toolRefuse at dispatch, tell the model the capability is unavailable, do not execute
Product event is revoked (identity check expired)Append tool_removal, increment ledger generation, refuse in-flight calls that race the revocation
Compaction summarizes a range that contained tool changesOn providers that return net tool_changes with a compaction block, re-apply them unmodified so post-compact availability matches pre-compact state (see provider compaction docs for your stack)
Extension or plugin toggle rewrites system/toolsFreeze the prefix after the first assistant turn; send capability deltas only through prefix-safe channels (aaif-goose#11905)

Avoid silently mutating tools[] "just this once" on a cached conversation. Take the deliberate cache miss on a new run boundary, or keep the privilege withheld.

Provider matrix for an existing product

Provider surfaceControl primitiveUse when
Claude API / Bedrock / GCP with mid-conversation tool changesdefer_loading + tool_addition / tool_removalApp-observed product events must change the offered set (Claude Platform docs)
OpenAI tool searchdefer_loading + model discoveryLarge optional corpora where the model should find tools (OpenAI tool search)
OpenAI mid-turn steeringInstruction updates mid-runPolicy or goal nudges that must not cancel in-flight tools (OpenAI steering)
No native control channelNew run or human pathPrefer a clean catalog boundary over a hidden privilege left offered

Verify the solution

Run these checks before you call the lifecycle done:

  1. Cache stability: On a supporting Claude model, offer a deferred tool after ten cached turns and confirm the tools/system prefix still cache-hits (Dreaming Press).
  2. Dispatch refuse: After tool_removal, force a synthetic model call for that name and assert your runtime returns capability_not_offered with no side effect.
  3. Event binding: Fire identity verification without appending tool_addition and assert refund remains unoffered and unexecutable.
  4. Lazy advertisement: If you mix tool search with capability gates, confirm hidden tools are absent from the searchable pool until offered (pydantic-ai#7087).
  5. Audit: Complete a refund and confirm the action row stores capability_generation and the offered set that included issue_refund.
  6. Fallback: On an unsupported model tier, assert the product takes the documented fallback instead of mutating tools[] mid-cache.

Common mistakes

  • Mutating tools[] to unlock one action. You pay a full prefix miss and the stack learns that privileges can reshuffle cheaply (Dreaming Press).
  • Leaving dangerous tools offered "for simplicity". The model can call them before your product fact is true.
  • Trusting removal as enforcement. Without dispatch refuse, a stale tool_use still executes.
  • Using tool search for product events. Discovery fits large libraries; control fits facts your code already knows (Dreaming Press; OpenAI tool search).
  • Rewriting earlier addition/removal messages. Editing history busts cache and can invalidate thinking blocks; append a new delta instead (Claude Platform docs).
  • Skipping live-tool receipts. Incidents then cannot reconstruct which privileges were live.

What to do next

Pick one high-risk tool in your product agent, usually a refund, delete, or deploy action. Declare it deferred, unlock it only after the real product event, refuse it at dispatch when withdrawn, and store a live-tool receipt on the first successful call. Expand the ledger to the rest of your catalog only after those checks pass on your primary provider and your fallback path.

If you package product knowledge for agents with Make Your Agent, keep procedural guidance in your knowledge base. Keep privileged mutations behind this capability lifecycle instead of exposing every write tool for the whole chat.

Be first in line.

Join the waitlist and we'll email you the moment it's ready. No sales call.

Try the demo →
or talk to us