Blog · · 9 min read

Handle streaming AI agent tool call arguments without executing truncated or premature calls

Handle streaming AI agent tool call arguments without executing truncated or premature calls

Your product chat shows a tool card filling in: update_ticket, then {"ticket_id":"T-19, then more JSON fragments. A host that treats a coincidentally parsable prefix, an empty first SSE chunk, or an aborted mid-stream buffer as ready can write truncated CRM fields, invoke tools with {} before required fields arrive, or crash the turn with an unhandled JSONDecodeError. To handle streaming AI agent tool call arguments safely, accumulate deltas by call identity, keep draft UI non-executable, wait for a provider terminal event, schema-validate, and fail closed when the stream dies early.

Direct answer: the accumulate-then-dispatch lifecycle

Ship one inbound argument contract before any product write:

  1. Ingest deltas by call identity. Key pending calls by OpenAI output_index/item_id, Chat Completions index, or Anthropic tool_use index, and append argument text without deciding readiness (OpenAI Function calling).
  2. Render draft UI only. Show "Preparing update..." or a progressive field preview. Never imply the tool will run or has succeeded (MaximoFN accumulate guide).
  3. Gate on a terminal event. Refuse executeTool until response.function_call_arguments.done / response.completed, Anthropic message_stop, or an equivalent provider stop (OpenAI Function calling; MaximoFN).
  4. Schema-validate complete JSON. Parse once after the terminal event, then validate against the tool schema. Invalid complete JSON becomes a model-visible tool error, not a silent {} (Vercel AI SDK tools).
  5. Fail closed on abort or truncation. If the stream ends without a terminal completion while arguments are incomplete, patch an incomplete/rejected tool_result and do not dispatch (google/adk-python#6716).
  6. Keep the next turn valid. Every announced tool_use gets a matching result marker before the next model call, including incomplete and rejected cases.
ToolArgLifecycle {
  pending: Map<call_key, { name, argument_text, state }>
  states: drafting | ready_to_validate | dispatched | rejected_incomplete
  on_delta(call_key, text): append; emit draft UI; never execute
  on_terminal(call_key): parse + schema_validate -> dispatch or reject
  on_stream_abort(): mark incomplete; patch tool_result; never execute
}

Why streaming arguments break product hosts

Without streaming, a tool call arrives as one JSON object. With streaming, the same object can arrive as many fragments, and a network chunk boundary is not a protocol boundary (MaximoFN). OpenAI's Responses API emits response.output_item.added with an empty arguments string, then a series of response.function_call_arguments.delta events, then response.function_call_arguments.done with the full arguments (OpenAI Function calling). Chat Completions streaming follows the same idea: accumulate delta.tool_calls[].function.arguments by index before json.loads.

Three production failure modes show up repeatedly in practitioner reports:

FailureWhat the host does wrongObserved impact
Parsable-prefix finalizeEmit tool-call when JSON.parse or isParsableJson succeeds mid-streamTruncated large args (for example code strings) dispatched early (vercel/ai#12052)
Empty-first-chunk as {}Treat args="" like missing args and materialize a complete callRequired-field tools run with empty objects before real args arrive (langchain#38682)
Abort without length finishParse partial args when the stream dies with no finish_reasonUnhandled JSONDecodeError kills the whole invocation (adk-python#6716)

The Vercel AI issue describes large tool arguments for deploy_edge_function emitting truncated tool-call events while small tools appeared fine (vercel/ai#12052). LangChain reports agents crashing with ValidationError: Field required when OpenRouter-style providers send an initial empty args fragment (langchain#38682). Google ADK's LiteLLM path handled truncated args only when finish_reason == "length", so provider aborts with partial {"agent_name": raised through the stack (adk-python#6716).

Inside a product agent, the same bugs hit customer data. A truncated update_crm_note can persist half a message. An empty-args fire can create a ticket with no title. An unhandled parse exception can leave the chat UI stuck while a background write already happened on a prior partial dispatch.

Implement streaming AI agent tool call arguments as a product contract

1. Accumulate by call identity, not by arrival order

OpenAI documents aggregating response.function_call_arguments.delta into the matching output item, and Chat Completions deltas by index (OpenAI Function calling). MaximoFN's cross-provider map is the same idea with different keys: OpenAI item_id/output_index, Anthropic tool_use block index, Google step index (MaximoFN).

Practical rules for an existing SaaS host:

  • Create the pending record on the first identity-bearing event (name + id/index).
  • Append argument text exactly as received. Do not parse each fragment.
  • Support parallel tool calls with separate pending records.
  • Keep argument text in diagnostics or draft state; do not promote it to executable input yet.

If you use the Vercel AI SDK, prefer the documented lifecycle hooks: onInputStart, onInputDelta, and onInputAvailable for complete validated input (AI SDK tools). Treat onInputAvailable as the first moment dispatch is even eligible, not as automatic execution for write tools.

2. Keep draft UI non-executable

Users want to see that a write is being prepared. They lose trust when the UI implies success or when the backend has already acted on a draft.

Recommended UX:

  • While drafting: "Preparing ticket update..." plus optional progressive field preview marked as draft.
  • On terminal + valid schema: transition the card to "Running..." only after the host has accepted the call for dispatch.
  • On incomplete/abort: "The model response ended before the tool arguments were complete. Retry or edit." Never invent empty defaults.

MaximoFN puts the rule plainly: showing a draft is fine; executing it is not (MaximoFN).

3. Never treat JSON.parse success as readiness

Never treat a successful JSON.parse on a growing argument buffer as readiness; only a provider terminal event proves the field is done changing.

A prefix can look like valid JSON while more fields are still streaming. Commenters on the Vercel AI issue note that JSON.parse success only means "the buffer parses so far," which does not answer whether a field is settled (vercel/ai#12052). MaximoFN makes the same separation: first ask whether the turn completed; only then ask whether arguments match the schema (MaximoFN).

What to do differently:

  • Delete any if (isParsableJson(buffer)) finalize() path in product code.
  • Finalize only on provider terminal events or SDK hooks that already encode that gate (function_call_arguments.done, onInputAvailable after stream completion semantics).
  • If you need progressive field UX, track settled JSON fields with a streaming parser, but still withhold executeTool until the terminal event.

4. Empty first chunks are not zero-arg tools

Treat a first SSE chunk with empty args as still-streaming, not as a finished zero-argument tool call that materializes as {}.

LangChain's report is precise: providers can send args="" before the real JSON arrives, and treating that empty string like args=None creates an immediate complete tool call with {} (langchain#38682). Agents then hit schema validation failures or, worse, execute tools that accept empty objects.

What to do differently:

  • Distinguish three cases in your accumulator: missing name/id, empty args while the stream is open, and final empty args for a true zero-argument tool.
  • Only resolve empty args to {} after a terminal/last-chunk signal.
  • For required-field tools, refuse dispatch when the final object is missing required keys, even if parse succeeded.

5. Schema-validate after the terminal event

Once the provider says the call is complete, parse and validate:

on_terminal(call):
  if truncated_or_aborted: reject_incomplete(call); return
  try:
    args = json.loads(call.argument_text)
  except JSONDecodeError:
    reject_incomplete(call); return
  errors = validate(args, tool.input_schema)
  if errors:
    return_tool_error(call, errors)  # model-visible, no side effect
  else:
    execute_tool(call, args)

The AI SDK documents schema validation on tool input and surfaces InvalidToolInputError when inputs do not match (AI SDK tools). Product hosts should keep the same fail-closed behavior even when not using that SDK: never coerce invalid JSON to {} to keep an old compatibility layer quiet (MaximoFN).

6. Fail closed when the stream aborts mid-arguments

Give aborted mid-stream tool calls the same fail-closed path as max-token truncation so missing finish_reason cannot raise an unhandled parse error or dispatch partial JSON.

ADK's LiteLLM adapter handled incomplete tool JSON only when finish_reason == "length". Streams that died mid-call with no finish_reason bypassed the guard and raised JSONDecodeError out of generate_content_async, killing the invocation (adk-python#6716). MaximoFN's harness table matches the product policy: connection closed before the terminal event means reject the turn; do not call executeTool with partial arguments (MaximoFN).

What to do differently:

  • Treat "stream ended while pending argument text does not parse" as incomplete, regardless of finish reason.
  • Convert that into a structured incomplete tool result the model and UI can see.
  • Log provider abort separately from user Stop. Cancel of an already-running tool is a different lifecycle; this article owns the inbound argument path before dispatch.

Handle failure and recovery

SituationHost actionUser-visible resultNext model turn
Mid-stream parsable prefixKeep accumulatingDraft card onlyN/A
Terminal + schema validDispatch onceRunning / then tool resultNormal tool_result
Terminal + schema invalidDo not dispatchError on tool cardError tool_result with validation details
Abort / truncate mid-argsDo not dispatchInterrupted / incompleteExplicit incomplete tool_result
Empty first chunkKeep pendingDraft or "preparing"Wait for later deltas or terminal empty {}

Recovery checklist for operators:

  1. Confirm no product write occurred for rejected or incomplete calls (idempotency keys help here, but the first line of defense is not dispatching).
  2. Confirm conversation history has a result marker for every tool announcement.
  3. Offer retry without replaying a partial write.
  4. Preserve diagnostics: raw argument buffer length, whether a terminal event arrived, and the reject reason.

Verify the solution

Run these checks before enabling write tools behind streaming UI:

  1. Parsable-prefix trap. Replay a recorded SSE fixture where an intermediate buffer is valid JSON ({"query":"test"}) and later deltas add more fields. Assert executeTool is not called until the terminal event, and the final args include every field (vercel/ai#12052; MaximoFN).
  2. Empty-first-chunk trap. Replay a first chunk with args="" for a required-field tool. Assert no tool invocation occurs and tool_calls stay empty until later deltas or a true terminal empty object (langchain#38682).
  3. Abort trap. Cut the stream after {"ticket_id": " with no finish reason. Assert the host returns a structured incomplete result rather than an unhandled exception, and that no write API is called (adk-python#6716).
  4. Parallel calls. Interleave deltas for two call indexes and assert each accumulator stays isolated (OpenAI Function calling).
  5. Draft UI contract. In an integration test, assert the UI never shows "Updated ticket" before a successful tool result event.
  6. Schema gate. Send terminal complete JSON missing a required field and assert a model-visible tool error with zero side effects (AI SDK tools).

MaximoFN recommends replaying recorded SSE with arbitrary chunk boundaries so network framing cannot change the reconstructed result (MaximoFN). Adopt that as a CI fixture suite for every provider your product supports.

Common mistakes

  1. Finalizing on isParsableJson. Fails on large strings and coincidental prefixes (vercel/ai#12052).
  2. Coercing bad JSON to {}. Hides incomplete turns and can fire zero-arg behavior for tools that require fields (MaximoFN; langchain#38682).
  3. Handling only finish_reason == "length". Provider aborts and proxy cuts need the same incomplete path (adk-python#6716).
  4. Letting the UI promise execution. Draft cards that say "Updating..." before dispatch train users to trust incomplete work.
  5. Confusing this with Stop/cancel. Cancel owns tools already running. This contract owns whether a tool is allowed to start from a streamed argument buffer.
  6. One accumulator for all providers without identity keys. Parallel tool calls will merge arguments across calls when indexes collide.

What to do next

Pick one write tool behind your streaming product chat (for example ticket update). Add a pending-call accumulator keyed by provider identity, wire draft-only UI to deltas, and move dispatch behind the terminal event plus schema validation. Add the three replay fixtures above to CI before enabling the next write tool. If you are building knowledge-base or persona agents on top of an existing product surface, platforms such as Make Your Agent still need this host-side gate whenever the chat UI streams tool arguments into real product actions.

Be first in line.

Join the waitlist and we'll email you the moment it's ready. No sales call.

Try the demo →
or talk to us