Blog · · 11 min read

AI agent error handling: make every failure visible

AI agent error handling: make every failure visible

AI agent error handling fails when a tool returns an error but the host workflow still records success. The customer sees incomplete work, the ordinary alert never fires, and the agent may keep calling tools until a runtime limit stops it. An n8n community report describes this exact mismatch: the tool failed, but the agent node and overall execution did not. Handle this at the execution boundary. Give every run a typed outcome, route it into the product's existing workflow status and alerts, and enforce hard retry, turn, time, tool-call, and spend budgets.

AI agent error handling starts with a typed outcome

Do not let a model-written sentence decide whether a run succeeded. The execution layer should return one of a small set of outcomes with a machine-readable reason and enough context for recovery.

OutcomeMeaningHost application action
okThe requested work completed and its result passed validationMark the workflow complete and record the resulting object IDs
retryableThe attempt failed for a temporary, explicitly retryable reasonSchedule a bounded retry with the same logical action identity
fatalThe request is invalid, forbidden, or cannot succeed without a code or data changeStop, alert at the appropriate severity, and show a specific failure
needs_humanProgress is blocked by missing input, approval, conflict, or an uncertain external outcomeCreate a durable recovery task with an owner
cancelledA user, operator, deadline, or policy stopped the runStop new tool calls and record what already completed

Every outcome should carry the same core fields:

{
  "run_id": "run_1042",
  "workflow": "resolve_billing_case",
  "outcome": "retryable",
  "reason_code": "provider_rate_limited",
  "failed_step": "load_invoice",
  "attempt": 2,
  "retry_after_seconds": 30,
  "completed_effects": [],
  "customer_message": "Invoice lookup is temporarily unavailable.",
  "operator_detail": "Upstream returned a retryable rate-limit response."
}

Keep reason_code on a controlled list. Free text can explain the error, but alerts, retries, dashboards, and tests need stable values. Do not expose stack traces or credentials in customer_message. Put diagnostic data in protected telemetry linked by run_id.

Why failures disappear inside agent loops

An agent runtime usually treats a tool error as information the model can reason about. That can be useful when the model can correct an argument or choose another read-only tool. It becomes dangerous when the host application assumes that a completed loop means the customer's task completed.

Some SDKs expose this choice directly. OpenAI's function-tool documentation says the default handler turns a tool exception into error output for the model. Passing None as the error handler re-raises the exception so application code can handle it. Neither choice defines the business outcome on its own. If an error stays inside the loop, the outer workflow must inspect the final typed result. If the error is re-raised, the boundary must translate the exception into a stable application status.

Loops create a second failure path. A LangGraph issue includes a reproducible agent that kept calling tools until its recursion limit. The issue is a reporter's account for a specific version, not a claim about every agent framework. It shows why a prompt such as "stop after an error" is not an execution control. The host must enforce the stop condition.

A separate Hacker News author report lists cascading tool failures and partial-workflow recovery among the cases teams miss before production. Treat that thread as practitioner evidence, not measured prevalence. The reported failure is specific: one bad step can contaminate later steps if the runtime does not change state immediately.

Implement one tool execution boundary

Wrap every tool call before its result returns to the model. The wrapper should validate input, authorize the action, execute the connector, classify the result, record the attempt, and only then produce model-visible output.

call_tool(run, request):
  validate_schema(request)
  authorize(run.actor, run.tenant, request)
  enforce_run_budget(run)

  try:
      raw = connector.execute(request)
      result = validate_tool_result(raw)
      record_attempt(run, request, outcome="ok")
      return ToolOutcome.ok(result)

  catch RateLimit or TemporaryUnavailable as error:
      outcome = ToolOutcome.retryable(
          reason_code=classify_retry(error),
          retry_after=bounded_delay(error)
      )
      record_attempt(run, request, outcome)
      return outcome

  catch InvalidInput or PermissionDenied as error:
      outcome = ToolOutcome.fatal(reason_code=classify_fatal(error))
      record_attempt(run, request, outcome)
      return outcome

  catch AmbiguousWriteTimeout as error:
      outcome = ToolOutcome.needs_human(reason_code="external_outcome_unknown")
      record_attempt(run, request, outcome)
      stop_run(run)
      return outcome

The model may receive a safe summary such as invoice service temporarily unavailable. The application receives the full typed outcome. This separation prevents the model from turning an authorization denial into a generic retry or declaring success after a partial result.

Classify failures by cause, not by HTTP status alone. A timeout on a read may be retryable. A timeout after dispatching a payment or message can leave the external outcome unknown. A 409 might mean a temporary version conflict, a duplicate action that already succeeded, or a permanent business-rule violation. The connector owns that distinction because it understands the remote operation.

Map outcomes into the existing product

Reuse the product's current workflow, alerting, and support concepts. Translate each agent outcome into the states that the rest of the application already understands.

Agent outcomeWorkflow stateAlert or taskUser experience
okcompletedNo incident; retain traceShow the completed result
retryablewaiting_retryAlert only after the retry ceiling or according to service policyShow delayed progress and the next attempt time
fatalfailedPage or ticket according to consequence and scopeState what cannot continue and what input must change
needs_humanblockedAssign a recovery task with evidence and deadlineShow that review is required, not a spinner
cancelledcancelledNotify the owner if partial effects remainConfirm cancellation and list completed changes

Emit the transition once, next to the durable run record. Include run_id, tenant, user, workflow, failed step, reason code, attempt count, elapsed time, tool calls, and completed effects. Redact prompts and tool arguments according to the product's data policy.

Do not page on every temporary failure. A single rate limit can be ordinary. Page when a consequential action has an unknown outcome, a retry ceiling is exhausted, many runs share the same fatal reason, a policy boundary fails, or customer work is blocked beyond its deadline. These are implementation rules to tune against the product's own service expectations.

Set execution budgets outside the prompt

A retry ceiling controls repeated attempts at one step. A turn ceiling controls model and tool iterations across the run. They solve different problems, so enforce both.

OpenAI's runner documentation states that exceeding max_turns raises MaxTurnsExceeded. Catch that exception at the application boundary and return a typed result such as needs_human with reason_code: turn_budget_exhausted. Do not turn it into a generic 500 response or let the model summarize it as success.

Not every runtime exposes all needed limits. A Strands Agents SDK issue requested first-class turn and token budgets and described external cancellation as insufficient for that use case. The issue is a repository report and design request, not a guarantee about the current SDK. When a framework lacks a limit, implement it in the host runner.

The host runner can enforce a budget record like this:

{
  "max_turns": 12,
  "max_tool_calls": 8,
  "max_attempts_per_step": 3,
  "max_elapsed_seconds": 45,
  "max_input_tokens": 50000,
  "max_output_tokens": 8000,
  "max_estimated_cost_minor_units": 25
}

Check the applicable limit before starting the next unit of work. A deadline should also cancel connector requests where cancellation is supported. Hitting any limit must produce a named outcome, a final trace event, and a visible workflow state. "Stopped" is not enough; operators need to know which budget stopped the run.

Retry only the failures that can change

Temporal's Python guidance distinguishes transient, intermittent, and permanent failures. It recommends retrying failures according to type, marking permanent errors as non-retryable, and making retried activities idempotent. The same classification works at an application's agent boundary:

  • Retry a brief transport interruption with bounded backoff.
  • Retry a rate limit after the provider's delay, then increase the delay within a ceiling.
  • Do not retry invalid input, missing authorization, policy denial, or an unsupported operation.
  • Reconcile an uncertain write before retrying it.
  • Require new user input when the request itself must change.

Keep the same logical action identity across attempts. If the model changes the tool arguments, treat that as a new proposed action and run validation and authorization again. This prevents a retry loop from gradually changing a rejected request until something passes.

After the retry ceiling, move the run to needs_human or fatal according to the reason. Put a record on the dead-letter queue only if it carries enough context to recover: validated request, action identity, completed effects, failure class, attempt history, next allowed action, and owner. Replaying a bare serialized exception will reproduce the failure without telling an operator what to do.

Work through a realistic partial failure

Consider an agent that handles a billing case by loading an invoice, applying an approved credit, and drafting a customer reply.

If invoice lookup is rate-limited, return retryable, schedule a bounded retry, and do not run later steps. If the credit request times out after dispatch, return needs_human with external_outcome_unknown, reconcile by action identity, and block the reply. If credit application succeeds but reply generation fails, record the credit as a completed effect. The product may retry the draft step without repeating the credit.

Base the final customer-facing outcome on the workflow, not the last model message. Neither a friendly explanation nor an ugly error string proves what happened to the credit. The durable tool records provide that answer.

Handle failure and recovery without losing context

Recovery begins from the last proven state. Store completed effects separately from planned and failed steps. A human recovery task should show the original intent, authorized scope, tool outcomes, remaining work, and the action that is safe to take next.

Record cancellation as a state transition too. Stop undispatched work, request cancellation for in-flight reads where supported, and reconcile in-flight writes. Keep the run record so a later callback has a place to store the external result.

When the process crashes, reconstruct the run from durable records. Any step left in executing needs a lease expiry and connector-specific reconciliation rule. A restarted worker should not ask the model whether the step probably finished.

Verify AI agent error handling with failure injection

Test the boundary with deterministic failures before relying on production alerts:

  1. Return a tool validation error. The run becomes fatal, no retry occurs, and the workflow is not marked complete.
  2. Return a rate limit twice and success once. The same action identity is used, the attempt count is three, and only one logical result exists.
  3. Make a tool return an error object without throwing. The wrapper still detects it from the declared schema.
  4. Keep requesting tools until the turn ceiling. The run emits turn_budget_exhausted, stops new calls, and activates the expected workflow route.
  5. Exhaust the retry ceiling. One dead-letter record appears with the complete attempt history and owner.
  6. Commit a write, then drop its response. The next step stays blocked until reconciliation establishes the result.
  7. Cancel during an in-flight tool call. No new calls start, and any completed effect remains visible.
  8. Crash after recording executing. A later worker recovers the lease without blindly repeating a consequential write.
  9. Send two failures with the same reason code across tenants. Alerts retain tenant scope and do not leak arguments or prompts.
  10. Return ok without the required result fields. Output validation rejects the false success.

Add dashboard checks for runs with no terminal outcome, ok outcomes that lack required result identity, exhausted budgets without an alert or task, and dead-letter records without an owner. Those invariants catch silent gaps between agent telemetry and product operations.

Common mistakes

Parsing the model's final prose for words such as "failed" or "success" makes workflow state depend on phrasing. Use the execution record.

Catching every exception and returning a string keeps the loop alive but can hide the failure from the host. Return a typed tool outcome and make the outer runner consume it.

Retrying every error wastes time and can repeat side effects. Classify permanent, temporary, and uncertain outcomes before scheduling another attempt.

Setting only max_turns leaves one slow tool able to exceed the user deadline. Setting only a timeout leaves a fast loop able to make many calls. Use several budgets and record which one fired.

Sending every temporary failure to on-call creates noise. Suppressing every tool failure hides customer impact. Alert based on consequence, exhausted recovery, shared cause, and service deadline.

What to do next

Choose one existing agent workflow and list every path that can currently end without a typed outcome. Add the five-outcome contract at its tool boundary, then write the test where a tool fails but the runtime tries to report success. Do not expand the agent's write access until that test activates the product's ordinary failure route.

References

Be first in line.

Join the waitlist and we'll email you the moment it's ready. No sales call.

Try the demo →
or talk to us