Blog · July 16, 2026 · 11 min read

How to integrate AI agents into existing applications

Teams often choose a model and framework before they define the product workflow. When they integrate AI agents into existing applications in that order, they get demos that look convincing but bypass permissions, hide partial failures, or leave no safe way to reverse an action. Those problems come from the integration boundaries rather than the model's first response.

Start with one workflow and give the agent only the authority that workflow requires. Keep product rules in charge of identity and state changes. Add narrow tools, visible supervision, task-level evaluation, and a rollout switch that can stop new runs without disabling the product. This approach adds controlled judgment to a working application instead of rebuilding the application around a chat loop.

Why agent integration fails when the agent comes first

Write down one user outcome before choosing a model or framework. A suitable first workflow meets five conditions:

A user can state the goal clearly.
The product already has reliable data and actions for completing it.
Application code can check success and failure without asking the model to grade itself.
A person can recover the workflow when automation stops.
The expected benefit justifies the extra latency and operating cost.

An agent is not always the right abstraction. Anthropic distinguishes workflows from agents: workflows follow predefined code paths, while agents direct their own process and tool use. Anthropic recommends the simplest design that meets the requirement, with more complexity added only when it produces measurable value. Microsoft reaches a similar conclusion, noting that extra orchestration brings coordination overhead, latency, and cost.

Use the workflow itself to choose the design:

Situation	Start with
One response with no external action	A single model call
Known steps with limited judgment inside each step	A deterministic workflow with model-assisted steps
The next action depends on changing context or tool results	One tool-using agent with hard limits
Independent specialists must collaborate on a task one agent cannot handle reliably	Multi-agent orchestration

The bottom row is not automatically more capable. Google Cloud's design guide compares single-agent, multi-agent, human-in-the-loop, and custom-logic patterns against task complexity, latency, cost, and the need for human involvement. Choose the pattern from those requirements.

Define the contract before the architecture

An existing product already has user identities, tenant boundaries, business rules, database constraints, audit records, notifications, and support procedures. The agent has to work through those contracts. A prompt should not create a second, weaker set of product rules.

Document these parts of the selected workflow:

Name the user action or system event that starts a run.
Define the product outcome that counts as complete.
List the reads and writes the agent may request.
Keep deterministic and human-only operations on a separate forbidden list.
Attach a user, tenant, role, and scope to every action.
Set ceilings for steps, tool calls, tokens, elapsed time, and retries.
State which actions need approval and who may approve them.
Define machine-readable outcomes for success, failure, cancellation, and uncertain side effects.
Assign recovery to the user, support team, or an automated workflow for each failure class.
Describe which product state can be restored, compensated, or left unchanged.

"Resolve the customer's issue" is too vague to test. A workable contract says: "Draft a refund recommendation from order data and policy, request approval when the amount exceeds the user's limit, execute it through the existing refund service, and return the refund record or a typed failure." That version names the participating systems and gives the application a result it can check.

How to integrate AI agents into existing applications safely

Keep the product as the system of record. The agent can propose or request work, but domain services must execute valid state changes.

Product UI or API
        |
        v
Run controller  ---->  Run state and trace store
        |
        v
Policy boundary ---->  user, tenant, role, approval, budget
        |
        v
Agent runtime  ------>  model and context retrieval
        |
        v
Tool gateway  ------->  narrow product capabilities
        |
        v
Existing domain services, databases, queues, and audit logs

The run controller creates a durable run ID, records the requested outcome, enforces budgets, and maps the result to a product status. The policy boundary checks each sensitive action. The tool gateway turns model requests into typed calls to existing services. The runtime does not write directly to a database simply because the model can generate SQL.

This layout supports gradual adoption and works as a retrofit around a legacy system you are not rewriting. AWS shows how to wrap a documented REST API as agent tools and warns against allowing the agent to bypass existing authentication and authorization. Apply the same boundary with other tool protocols: inspect the existing capability surface, expose only approved operations, and keep enforcement behind the interface.

Build the integration in eight steps

1. Choose one bounded workflow

Pick work that matters but remains recoverable. Do not begin with your riskiest financial, security, or account-administration operation. Map the current user journey, including handoffs and exceptions. The current process is your baseline, even when it is messy.

Record its completion rules. If the existing workflow has no reliable definition of success, an agent will make the ambiguity harder to diagnose.

2. Set the minimum autonomy

Use the smallest grant of authority that produces a useful result:

Read-only access can gather and summarize product data.
Draft-only access can prepare a response, plan, or change set.
Approval-gated access can request a product action but must wait for an authorized person.
Bounded autonomy can execute a narrow class of reversible, policy-approved actions.

Require evidence before each promotion. Move from draft-only to approval-gated after the evaluation set shows acceptable tool selection and arguments. Allow bounded autonomy only for action classes with reliable verification and recovery.

3. Carry identity through every layer

A generic service identity plus a prompt instruction does not preserve user permissions. Put the authenticated user, tenant, role, scopes, and session context on the run. Application code must resolve permissions before a tool executes.

Return one of four policy results: allow, deny, require_approval, or modify. The modify result can reduce scope, strip unsafe fields, or clamp a value to an allowed limit. Log that decision beside the requested action, not only beside the final run.

4. Turn product actions into narrow tools

Do not expose a broad CRUD client and expect the model to assemble business logic. Tools should represent user intent and preserve domain invariants. Prefer prepare_refund(order_id, reason) and execute_approved_refund(approval_id) to unrestricted record updates.

Every tool needs a specific name and description, typed and bounded inputs, application-supplied user and tenant context, and an explicit read or write classification. Document side effects. Set timeout and retry behavior. Return machine-readable outcomes. Sensitive changes also need a dry-run or preview mode.

A current integration guide from AppVerticals puts authorization and observability in the initial integration design, rather than adding them after the model is connected. Build the boundary before expanding the tool catalog.

5. Separate product state from agent state

Product records remain authoritative for orders, accounts, tickets, permissions, and other domain facts. Agent run state holds the plan, current step, tool results, approval waits, budgets, and final outcome. Conversation history can help interpret the request, but it cannot override current product data.

Persist enough run state to answer four questions after a restart:

What outcome did the user request?
Which action was the agent about to perform?
Which external actions definitely completed?
What can safely happen next?

Do not resume from a transcript alone. Resume from typed checkpoints tied to product action IDs.

6. Design supervision into the product UI

Show more than a spinner and a final paragraph. Use product language for each run state: preparing, waiting for approval, executing, partially complete, failed, cancelled, or recovered. Before a sensitive action, display the target, changed fields, expected effect, and approving identity.

Users should be able to reject or modify a proposal without restarting the whole workflow. Define cancellation precisely. It may stop work before the next action, request cancellation of an in-flight operation, or start compensation for completed work. Do not display "cancelled" while the outcome of a side effect remains unknown.

7. Evaluate the complete task

Component tests cannot show whether the whole run behaved correctly. Google Cloud's production guide describes agent evaluation in terms of trajectories, including tool selection, intermediate steps, recovery, and the final outcome. Build a versioned evaluation set from real workflow variants and known failures.

Score these dimensions separately:

workflow choice;
tool selection and arguments;
permission and tenant compliance;
final product outcome;
unnecessary actions;
recovery after a tool failure;
latency and resource budget;
user-facing explanation quality.

One aggregate score can hide a policy failure. An agent might reach the correct result after attempting an unauthorized action. The outcome may look right, but the run must still fail the policy test.

8. Roll out in stages with a real rollback path

Begin with recorded or synthetic cases. Run against live inputs in shadow mode without changing product state. Then expose suggestions to internal users, move to approval-gated actions, and finally allow a narrow autonomous action class. Compare each stage with the existing workflow.

Google Cloud recommends moving from sandbox to canary and then production. An enterprise integration roadmap from BotsCrew also separates assessment, architecture, testing, secure deployment, monitoring, and maintenance. Give each stage explicit entry and exit criteria. Require measured task, policy, and recovery results before promotion.

The rollback control should block new agent runs without disabling the rest of the product. Keep the previous deterministic workflow available while the agent path proves itself. Before launch, decide whether active runs will drain, stop, move to a person, or trigger compensation.

Handle failures as product states

Do not reduce every failed run to "something went wrong." Return a typed result the product can route:

run_outcome:
  status: ok | retryable | fatal | needs_human | cancelled | unknown
  run_id: string
  completed_actions: [action_id]
  pending_actions: [action_id]
  user_message: string
  operator_detail: string
  retry_after: timestamp | null
  recovery_owner: user | support | system

Use unknown when the application cannot confirm whether an external side effect completed. Do not retry an uncertain write until you can query the external system with a stable action ID. Restrict automatic retries to transient failures and operations designed for replay. Route denied requests, invalid input, and exhausted budgets without asking the model to improvise around them.

Users and operators need different details. The user needs the current state and next action. Operators need tool arguments, policy decisions, trace relationships, budget consumption, and the last confirmed side effect. Redact sensitive context before sending it to the model or a general log.

Verify the integration before increasing autonomy

Test the persisted system, not only a local agent console:

Check that every run maps to one declared product goal and final status.
Test a matrix of users, roles, tenants, and tools against the expected allow, deny, modify, or approval result.
Insert cross-tenant record IDs into retrieved text and tool arguments, then confirm that access fails.
Send invalid fields, oversized values, missing records, timeouts, and duplicate requests through each tool contract.
Restart the runtime at every checkpoint and confirm that completed writes do not repeat.
Reject, modify, expire, and approve an action. Only the approved version may execute.
Hit the step, time, tool-call, and resource ceilings and confirm that the run stops in a visible state.
Run the versioned scenario set and reject releases that regress policy or recovery checks.
Disable the feature flag, confirm that new runs stop, and verify the documented drain policy and old workflow.
Follow one product action ID through the user request, model calls, policy decisions, tools, side effects, and final outcome.

Verify the connections between these controls before release. A trace must reach the product result. An approval must bind to the exact action that executes. A retry must identify any side effects from earlier attempts.

Common integration mistakes

Building a parallel permission system

A prompt rule such as "only act for the current customer" is not authorization. Reuse the product's identity and policy layer at every tool boundary.

Exposing technical APIs instead of user capabilities

A large endpoint catalog creates ambiguity and gives the model more ways to construct an invalid sequence. Begin with a small capability surface that matches the selected workflow.

Treating chat history as the system of record

Conversation context may be stale, incomplete, or manipulated. Read current product facts through authorized tools and keep durable run state separate.

Measuring only response quality

Polished prose can hide the wrong tool choice, an exceeded budget, or a partial write. Grade the trajectory and product outcome independently.

Launching autonomy without a recovery owner

Every non-success state needs a destination. If nobody owns an approval timeout, uncertain side effect, or exhausted retry budget, the workflow is not ready for autonomous execution.

What to do next

Choose one existing workflow and write its contract today. Include the trigger, goal, allowed actions, identity, budgets, approval points, typed outcomes, recovery owner, and rollback procedure. Implement the first version with read-only or draft-only access. Add another tool or increase autonomy only after the persisted workflow passes the authorization, recovery, evaluation, and rollback checks above. Use this sequence to integrate AI agents into existing applications while keeping the product controls that already protect customer work.

References

Anthropic: Building effective agents supports the workflow-versus-agent distinction, minimum-complexity approach, and composable agent patterns.
Microsoft: AI agent orchestration patterns supports complexity selection and the operational tradeoffs of orchestration.
Google Cloud: Choose a design pattern for your agentic AI system supports requirement-driven selection among single-agent, multi-agent, human-in-the-loop, and custom patterns.
AWS: Enabling AI agents to interact with existing applications supports wrapping existing application capabilities as tools while preserving authentication and authorization.
AppVerticals: AI agent integration for an existing app provides directly competing coverage of integration boundaries, authorization, observability, and implementation sequencing.
Google Cloud: A developer's guide to production-ready AI agents supports trajectory evaluation, permissions, memory, logging, and staged deployment.
BotsCrew: Integrating agents with existing enterprise systems provides an independent phased roadmap covering assessment, architecture, testing, secure deployment, monitoring, and maintenance.