When ordinary application code fails, one request log and a stack trace pinpoint the problem. When an AI agent fails, a single request fans out into a dozen model calls, tool executions, retries, handoffs, and approvals, and flat logs cannot tell you whether a timeout came from a slow downstream API or a model looping on bad arguments. Build a hierarchical end-to-end trace model that anchors every span to a tenant, user, and product action, then track tokens, cost, and latency as first-class operational metrics.
The agent platform build vs buy decision is rarely binary. Building the whole orchestration layer in-house means owning retries, connectors, and observability; buying a monolithic platform risks vendor lock-in and blocks the logic that differentiates your product. Score each layer independently, runtime, model gateway, state, connectors, identity, evaluation, and approvals, then buy the commodity layers, extend open-source runtimes, and build only the core differentiators.
Teams adding AI to an existing product hit the same crossroads: what stays a direct API call, what becomes a reusable Model Context Protocol (MCP) tool, and what crosses an Agent-to-Agent (A2A) boundary. Wrapping every endpoint in a direct function call creates a brittle, tightly coupled monolith. This guide gives a worked architecture: direct APIs for synchronous high-risk transactions, MCP for reusable tools and data, and A2A for autonomous long-running delegation.
Agent UX patterns break down when chat is the only surface users have for an action-taking system: a run can sit queued, wait for approval, finish only some steps, or keep running after cancellation, and a spinner plus a final message hides all of it. Build the interface on a durable product state model that covers previews, progress, approvals, partial failure, undo, cancellation, and human handoff, so the page shows what actually happened even after a reload.
A multi-tenant AI agent architecture breaks when tenant identity is treated as prompt text: a stale cache, an unscoped retrieval query, a reused credential, or a resumed job can expose another customer data or act in the wrong account. Resolve tenant identity once from an authenticated request, carry an immutable tenant envelope through retrieval, memory, tools, credentials, queues, and audit logs, then prove isolation with 12 cross-tenant negative tests.
AI agent cost and latency turn unpredictable when a team adds model calls, tools, retries, and handoffs without defining where a run must stop. Set product limits first, record every unit of work in a per-run ledger, and require measured evidence before increasing complexity: a budgeted single-agent baseline, hard orchestrator limits, and a promotion gate keyed to cost per successful outcome.
AI agent prompt injection security breaks down when a team treats untrusted content as a text-cleaning problem. Build the system so manipulated model output cannot cross authorization, data, or side-effect boundaries unchecked: six explicit boundaries from source to effect, each enforced by deterministic controls the model cannot override.
An agent may pass every demo and fail on the first customer request outside the happy path. Checking the final answer will not reveal a bad tool choice, unauthorized side effect, or missing recovery step. Turn each confirmed production failure into a versioned, replayable case with tool-call checks, release gates, and rollback rules.
Stateful AI agents on Kubernetes can look healthy in single-pod tests, then lose their place when a Service routes the next request to another replica. Store durable run state outside the replicas, make resume idempotent with fencing tokens and action records, and test each workflow while forcing it to change pods.
Agent state management breaks when a product treats every useful fact as agent memory. A checkpoint, chat transcript, customer record, and remembered preference do not share an owner or lifecycle. Classify each value first: keep product facts under product services, use checkpoints only to resume, and treat long-term memory as derived data with provenance and expiry.
A valid tool call is only a well-formed request, not proof that the caller has permission. AI agent authorization belongs between the runtime and every product tool: decide with authenticated identity, tenant, tool, normalized arguments, and current state, then allow, narrow, defer, or reject.
Human in the loop AI agents break when the approval screen and the execution state drift apart. Approval needs its own durable workflow: store the exact proposed action, bind each decision to an authenticated approver and an immutable action version, and resume through an idempotent execution path.
Teams often choose a model and framework before they define the product workflow, and get demos that bypass permissions, hide partial failures, or cannot be reversed. Start with one workflow, give the agent only the authority it requires, and keep product rules in charge of identity and state changes.
Giving an agent direct access to a legacy API often produces the wrong action with valid JSON. Put a narrow contract between the model and the product: name one business action, bound its inputs, carry authorization context outside model control, and return a typed outcome.
AI agent error handling fails when a tool returns an error but the host workflow still records success. Give every run a typed outcome (ok, retryable, fatal, needs_human, cancelled), route it into the alerts and workflow status your product already has, and enforce hard retry, turn, time, and spend budgets.
An agent rollout strategy goes wrong when a team jumps from a convincing demo to live write access. Move one workflow through five stages (offline replay, shadow mode, suggestions, approval-gated actions, and bounded autonomy) with entry criteria, exit evidence, and rollback behavior for each.
Connecting a model to a legacy application is the easy part. Leave the existing system in charge of its records and business rules, add a narrow tool gateway instead of direct database access, and introduce agent access in stages: replay, shadow, read-only, approval-gated writes, then bounded autonomy.
An agent does not have to choose an action twice for your product to perform it twice. A practical design for idempotency keys, durable action records, retry rules, unknown outcomes, and compensation when agents call real product APIs.
