Blog · · 10 min read
AI agent structured output: enforce contracts on every tool handoff

Your agent triages a ticket, picks a route, and writes priority into the CRM. The model returned valid JSON. Three weeks later forecasting breaks because half the records store "high" and the rest store 1. Reasoning held up. The missing piece was an AI agent structured output contract at a handoff boundary. OpenNash describes the same pattern when silent field drops and type drift reach systems of record.
Product and platform engineers need a boundary map, an enforcement ladder, a typed-repair policy that forces a corrected tool call, and rejection telemetry that feeds evals. The checklist below stops malformed tool arguments from updating CRM, billing, or support state.
Validate four boundaries before any side effect
Treat every place model-shaped data crosses into product code as a contract:
- Tool input: arguments the model emits for a function or MCP tool.
- Tool output: the JSON or text your tool returns before the next model turn.
- Step handoff: structured state passed between agent steps or workers.
- Final action: the write, send, charge, or ticket update that mutates a system of record.
At each boundary, keep a single schema source of truth (often a Pydantic or Zod model exported as JSON Schema), validate before proceeding, fail closed on violation, and allow at most a small typed-repair budget. OpenAI function-calling docs and Anthropic tool-use overview already force a schema at the tool-input edge. Your product still has to own the other three edges and the loop policy after a validation failure.
BoundaryMap -> SchemaRegistry -> EnforcementLadder
-> TypedRepair(budget=1..2) -> FailClosedGate -> RejectionReceipt
Why malformed handoffs outrank clever prompts
Agent pipelines compound small shape errors. A step that usually looks fine becomes a durable bad write when dict.get() defaults missing fields to None and the CRM API accepts the incomplete object. OpenNash documents three recurring patterns: silent field drops, type drift across days, and schema drift from vendor tool responses that the model then confabulates around.
Practitioner reports match that mechanism. LangChain issue 36700 describes LLMs emitting missing required fields, wrong types, hallucinated empty values, and extra keys, especially against MCP dict schemas the model was not trained on. LangChain issue 37778 tracks a sharper loop failure: ToolNode correctly turns a Pydantic ValidationError into an error ToolMessage, yet the next model turn may answer with corrected JSON in plain text instead of issuing a repaired tool call. Users then see internals, and the intended side effect never runs.
Prompt-only instructions do not fix this, because models sometimes ignore formatting guidance. Contracts are checkable in microseconds next to a multi-second model call, which is why Husnain production-agent guidance treats Pydantic validation before side effects as required scaffolding, not polish.
Build one SchemaRegistry and reuse it
Define each write-capable tool as a typed model, export JSON Schema for the provider, and re-validate in your process before touching storage. OpenAI recommends strict: true with additionalProperties: false and every property listed in required (optional fields use a nullable type). Anthropic takes an input_schema per tool and documents schema-conformance controls including strict tool use. Husnain shows the practical split: provider schema for shape, local model for domain ranges, then business checks for meaning.
from typing import Literal
from pydantic import BaseModel, Field
class UpsertDeal(BaseModel):
deal_id: str = Field(pattern=r"^DEAL-[0-9]{6}$")
amount_cents: int = Field(gt=0, le=5_000_000)
stage: Literal["qualify", "propose", "negotiate", "won", "lost"]
owner_user_id: str = Field(pattern=r"^USR-[0-9]{6}$")
confidence: float = Field(ge=0.0, le=1.0)
TOOLS = [{
"type": "function",
"function": {
"name": "upsert_deal",
"description": "Update one CRM deal. Never invent deal_id values.",
"parameters": UpsertDeal.model_json_schema(),
"strict": True,
},
}]
def upsert_deal(raw: dict) -> dict:
args = UpsertDeal.model_validate(raw) # raises before the CRM write
return crm.upsert(args, idempotency_key=f"deal:{args.deal_id}:{args.stage}")
Keep enums closed. Cap blast radius with numeric bounds. Put misuse warnings in the tool description. Return structured tool errors the model can act on, not stack traces that kill the run. Prefer three to seven tools per agent so selection stays accurate; split agents when the surface grows past that.
Register schemas in a versioned registry keyed by tool name and semver. Ship schema changes with the adapter that produces or consumes them. That keeps staging and production from silently disagreeing about which fields are required, a deployment concern raised in discussion on LangChain issue 36700.
Enforcement ladder: parse, schema, then constrained decoding
Not every boundary needs the strongest technique. Use the ladder OpenNash and Husnain both describe:
- Parse check: is it JSON at all, or did the model wrap the payload in markdown fences or an apology?
- Schema validation: types, required fields, enums, ranges, formats. Default this at all four boundaries.
- Constrained decoding / strict mode: provider or serving-side token constraints so invalid shape cannot be produced. Use it where retry latency hurts and the schema is stable.
Use both provider strictness and local validation. Provider guarantees cover shape. They do not cover "this customer ID exists" or "refund amount must not exceed the original charge." Those stay in your validators.
Strict mode can hide abstention
Strict or constrained decoding can turn an honest abstention into an undetectable wrong enum, so keep semantic validators even after shape is guaranteed. Constrained decoding guarantees a schema-valid token sequence. It does not guarantee an honest value. If the correct answer is that none of the CRM stages apply, a strict enum still emits one of qualify|propose|negotiate|won|lost. OpenNash notes that constraints can convert a detectable failure into an undetectable wrong enum, and that cross-field rules still need schema validators with custom checks. Do this differently: keep a free-text reasoning or abstain_reason field, allow an explicit needs_human outcome in the schema, and run semantic checks after strict decoding so a fluent wrong enum still fails closed.
Typed repair that forces a corrected tool call
Fail closed means invalid output does not proceed. A warning-plus-continue policy is just a bad CRM write with an alibi. Then repair with the exact error, not a blind resample.
Husnain recommends feeding the validation error back and allowing exactly one repair attempt before escalating. OpenNash reports that one typed retry resolves most schema violations because the model was capable of conforming on that sample.
Force tool_choice after ValidationError
After a schema ValidationError, force the next model turn onto the same tool with tool_choice; otherwise models often leak corrected JSON as plain text. LangChain issue 37778 documents the product-visible failure: after a schema error ToolMessage, the model may emit corrected JSON as assistant text. The loop treats that as a final answer, leaks implementation detail, and skips the tool. Do this differently in your agent runtime:
- Tag schema and invocation failures as retryable, separate from business tool errors.
- While the repair budget remains, set
tool_choice(or equivalent) to the same tool name on the next model call. - Cap the budget at one or two attempts. Exhaustion routes to a human queue or safe fallback with the full trace.
- Persist a receipt: original violation, whether policy forced the retry, whether the second attempt changed the outcome, and the final execution evidence.
Without forced tool choice, "Please fix the error and try again" is only a suggestion. Forced tool choice keeps repair inside the tool protocol your product already trusts.
on ValidationError(tool=upsert_deal):
append ToolMessage(error=exact_pydantic_errors)
if repair_count < budget:
next_model_call(tool_choice=upsert_deal)
else:
escalate(human_queue, receipt=RejectionReceipt(...))
Validate tool outputs and step handoffs too
Most teams that validate at all only gate the final write or the provider-facing tool input. That catches the last error and none of the upstream ones. OpenNash argues tool outputs are skipped most often because APIs look deterministic until a vendor renames a field and the model invents a plausible value. Step handoffs matter for the same reason: intermediate state can drop fields no single step reads until a later write depends on them.
Do this differently for an existing product agent:
- Validate tool outputs against the fields the next prompt or step declares as required.
- Version schemas beside tool adapters so a vendor response change fails the adapter test before production agents absorb it.
- Carry the full intermediate schema across workers, including fields used only by audit or compensation.
- Put the strictest gate on final actions into CRM, billing, and support systems, including business rules that JSON Schema alone cannot express.
Provider docs and generic "use Pydantic" posts still leave a gap: a product-owned map of every seam, not only the first function call. Designing good tool schemas for legacy APIs helps, but runtime enforcement across all four boundaries is what stops silent corruption.
Wire rejection receipts into ops and evals
Log every rejection as structured telemetry: boundary, schema version, tool name, field, violation code, retry count, and resolution (repaired, escalated, budget_exceeded). OpenNash treats that log as a free eval set of the cases your model finds hard. Husnain likewise grows the eval harness from production failures before patching prompts.
Operational alerts that matter:
- Rising validation failures on a single tool after a prompt or model change.
- Repair success rate dropping below your baseline.
- Any final-action rejection that somehow bypassed the gate (should be zero).
- Plain-text final answers immediately after a schema error (signals missing forced
tool_choice).
Store the prompt version hash and schema version on every span. The first question after a regression is which prompt and which contract produced the bad run, and hashes answer it in one query.
Handle failure without pretending the write succeeded
When validation fails after the repair budget:
- Do not partially apply the write.
- Return a countable status such as
schema_rejectedorescalated, matching the status style Husnain uses for budget and step ceilings. - Show the user a product-safe message, not the raw schema dump, while operators still get the receipt.
- Keep idempotency keys on the eventual successful write so a later approved retry cannot double-apply.
Separately retry only transient transport errors (429, 5xx, timeouts) with backoff and jitter. Retrying a validation error without changing the prompt context is paying again for the same wrong request. Keep those two retry classes in different code paths so schema failures never inherit transport backoff loops.
A realistic product walkthrough
Imagine a support agent with tools lookup_subscription, draft_macro, and apply_credit. The dangerous path is apply_credit.
- Inventory boundaries: model args into
apply_credit, billing API response shape, handoff object into the approval worker, and the final ledger write. - Put amount caps, currency enums, and subscription ID patterns in the schema. Export that schema to the provider with strict mode enabled.
- Validate the billing API response before the model sees it. If
balance_centsdisappears after a vendor release, fail the adapter, do not let the model invent a balance. - On invalid args, append the exact Pydantic errors and force
tool_choice=apply_creditonce. - If repair fails, escalate to a human with the rejection receipt. Never paste the corrected JSON into the customer thread.
- On success, write with an idempotency key derived from
subscription_idplus credit reason code.
That sequence covers the gap left by pages that stop at "enable structured outputs" without a product loop policy.
Verify the solution before you call it done
Run these checks in CI and in a staging tenant:
- Boundary inventory: Every write tool has documented input, output, handoff, and final-action schemas.
- Malformed-arg suite: Missing field, wrong type, extra key, empty object hallucination, and out-of-range amount each fail closed.
- Repair path: One seeded invalid call is repaired into a real tool invocation, not plain text.
- Forced tool_choice: After
ValidationError, the next model request includes the forced tool; assert that in the test double. - Abstention path: Strict enums still allow
needs_humanor equivalent; a forced wrong enum fails semantic validation. - Vendor drift: Fixture a renamed tool-output field and assert the tool-output contract fails before the model sees it.
- Telemetry: Rejection events include boundary and field; dashboards show top violators.
If any check fails, keep the agent away from customer records.
Common mistakes
- Enabling provider
strictmode and skipping local validators. - Validating only the final CRM write while tool outputs remain untyped.
- Returning stack traces to the model instead of structured validation errors.
- Allowing unbounded repair loops that burn budget without escalating.
- Letting the agent dump corrected JSON as the user-visible answer after a schema error (LangChain issue 37778).
What to do next
Pick the single agent write that would hurt most if it went bad silently, usually a CRM stage change, refund, or support-macro apply. Define its schema with real enums and bounds, fail closed with one typed repair and forced tool_choice, and ship rejection logging for a week. Read the log. The first uncomfortable number is how often you were previously consuming malformed output as if it were fine.
If you embed product or support agents with tools that reach Pipedrive, Intercom, or similar systems, Make Your Agent can host the agent runtime while your boundary map and schema registry stay in your product control plane. Start with the write path, not the chat UI.