Blog · · 10 min read

Multi-tenant AI agent architecture: enforce tenant isolation

Multi-tenant AI agent architecture: enforce tenant isolation

A multi-tenant AI agent architecture breaks when tenant identity is treated as prompt text. A stale cache entry, an unscoped retrieval query, a reused credential, or a resumed job can expose another customer's data or run a tool in the wrong account. The model should not be responsible for preventing that mistake. AWS makes the same point: probabilistic models are not suitable for protecting tenant context, so deterministic application components must carry it instead (AWS tenant-isolation example). The design below applies that rule to retrieval, memory, tools, credentials, queues, and audit records, then tests each boundary for leaks.

Use one trusted tenant propagation contract

Resolve tenant identity once from an authenticated request. From that point on, every operation the agent can trigger receives the same immutable tenant envelope. Do not let each subsystem perform an unrelated tenant lookup.

Five rules define the contract:

  1. The identity layer establishes tenant_id, actor identity, and delegated scope. The model does not.
  2. Data, memory, tool, credential, and queue operations all require the tenant envelope.
  3. Each receiving component derives scoped resource access from that envelope.
  4. The model may choose an allowed action, but it cannot choose or override the tenant.
  5. Results and side effects record the tenant, action, policy version, and execution receipt.

A tenant ID in a system prompt does not meet this contract. The prompt can help the model reason about an account, but it cannot grant access to that account. Keep the envelope in typed application state, outside model-controlled content. It must survive retries, pauses, callbacks, and worker handoffs without becoming editable by the model.

Here is a compact request envelope:

tenant_context:
  tenant_id: tnt_42
  actor_id: usr_18
  session_id: ses_701
  delegated_scopes:
    - tickets:read
    - tickets:reply
  authentication_strength: password_and_totp
  policy_version: tenant_policy_17
  issued_at: 2026-07-22T09:30:00Z
  expires_at: 2026-07-22T10:00:00Z
agent_run:
  run_id: run_982
  workflow: resolve_support_ticket
  tool_allowlist:
    - get_ticket
    - search_help_center
    - propose_reply
  trace_id: trc_551

Application code constructs this object after authentication. A worker can verify it or narrow its scopes. It must reject any replacement tenant_id that arrives in a prompt, tool argument, retrieved document, or callback body.

Why ordinary SaaS filtering is not enough

Many SaaS request paths enforce tenancy in an API handler or database query. Agents travel farther. A single run can retrieve documents, write checkpoints, keep conversation memory, call a remote service, wait in a queue, resume on another worker, and copy results into a trace. A correct SQL filter protects only one part of that path.

Microsoft's multitenant architecture guidance says tenants must not gain unwanted access to other tenants' data or models. It compares tenant-specific, shared, and tuned-shared deployments (Microsoft multitenant AI guidance). The isolation mechanism changes with the deployment model, but the requirement does not. Every shared resource still needs an enforced tenant boundary.

Common leaks come from ordinary scope bugs:

  • A vector search drops its tenant namespace during a retry.
  • A conversation cache keys entries by conversation_id but not tenant_id.
  • A checkpoint restores a credential issued before the user changed organization.
  • A tool trusts an account identifier supplied by the model instead of server context.
  • A queued job contains only run_id, so the worker reloads state without checking ownership.
  • A trace stores prompts and tool output in a shared index that broad operator roles can read.
  • A connection pool retains tenant-specific database state between requests.

The model does not have to act maliciously for any of these failures to happen. Treat tenant isolation as an architecture and testing problem, not as a prompt-engineering exercise.

Enforce tenant isolation at every boundary

Start with authenticated identity

Tenant resolution belongs at the authentication boundary. The application reads it from a trusted session, token claim, or server-side membership record. Before the workflow starts, it confirms that the actor still belongs to the tenant and that the session grants access to that workflow.

The model may receive an account label when it needs that context, but the canonical tenant identifier is never a model-controlled choice. A prompt that says, "Switch to tenant B," is untrusted content and cannot change the envelope.

AWS's Bedrock implementation reads user and tenant identity from authenticated JWT claims, generates tenant-scoped credentials, and carries the trusted values through deterministic application components. The general rule does not depend on AWS: keep tenant context intact and pass the model only data that has already been scoped.

Scope retrieval, product data, and memory independently

Tenant enforcement belongs inside each data adapter, not only in the orchestrator. The retrieval method requires a tenant namespace. The product repository includes tenant scope in its query contract, while the memory store checks ownership on both reads and writes.

Google Cloud's reference architecture uses separate tenant environments, policy boundaries, per-tenant quota checks, and controlled routing in a centralized system (Google Cloud multi-tenant agentic architecture). Pooled storage can also work, but its adapter needs a hard rule: no query runs without verified tenant scope.

Use composite identifiers such as (tenant_id, conversation_id) and (tenant_id, run_id). Summaries, embeddings, evaluation examples, and long-term preferences are tenant data too. Your deletion inventory should account for those derived artifacts when a tenant leaves or retention policy changes.

Row-level security is useful, but it does not cover vector stores, caches, object storage, model context, or remote tools. Give those systems equivalent controls. A second boundary can still stop disclosure when one adapter omits a filter.

Derive credentials at the tool boundary

The host supplies trusted tenant context to a tenant-safe tool. Because the host already knows the tenant, tenant_id, organization, workspace, and credential references do not belong in model-controlled arguments.

AWS Prescriptive Guidance describes an MCP client sending tenant context to a server, which then acquires tenant-scoped IAM credentials for tenant resources (AWS guidance on enforcing tenant isolation). The same pattern works outside that stack:

  1. Verify the tenant envelope at the tool gateway.
  2. Resolve or mint a short-lived credential for that tenant and action.
  3. Use the narrow credential for the downstream call.
  4. Bind the result and execution receipt to the tenant and run.
  5. Discard the credential when the operation ends.

Do not use a global service token and rely on an application filter afterward. If code bypasses that filter, the credential still reaches every tenant. Raw credentials also stay out of prompts, model-visible tool output, approval records, and general traces.

MCP security guidance warns that token passthrough can bypass service-specific controls and weaken accountability. It recommends progressive, least-privilege scopes instead of broad access granted at the start (MCP security best practices). Internal tool gateways need the same discipline even when they do not implement MCP.

Preserve scope through queues, retries, and resumes

Long-running work often leaves the process that handled the request. Each job message carries the tenant envelope, its version, and an integrity check. The receiving worker verifies that envelope, reloads current membership and policy when needed, and confirms that the referenced run belongs to the same tenant.

Retries keep the original tenant and action identity. Do not ask the model to reconstruct the call from a summary. An approval that resumes later remains bound to the exact tenant, action hash, resource version, and approver. Any changed binding requires a new decision.

Qualify cache and lock keys with the tenant. tnt_42:run_982 is safer than run_982 when IDs are not globally unique. Use the same composite rule for deduplication keys, idempotency records, cancellation flags, and rate-limit buckets.

Make tenant scope visible without leaking content

Attach the tenant ID to each run, model span, retrieval span, tool decision, side effect, retry, approval, and final outcome. Operators can then follow one tenant's workflow without searching prompts for account names.

Telemetry still needs its own access controls. Copying raw documents or credentials into a broad observability system creates another leak path. Store correlation IDs and structured outcomes in general traces. Keep sensitive payloads behind the product's normal data controls.

AWS recommends tenant-aware metrics, logs, resource-consumption views, and operational profiles for multi-tenant agent environments (AWS guidance on data, operations, and testing). Per-tenant tool calls, model usage, queue depth, failures, and throttling help operators spot noisy-neighbor behavior while retaining tenant boundaries.

Example: isolate a shared support agent

Consider a support agent shared by Tenant Alpha and Tenant Beta. The model runtime and worker pool are shared. Tickets, knowledge documents, help-center settings, and customer-service credentials belong to one tenant or the other.

A user from Alpha asks about ticket 1842. The application authenticates that user and creates an Alpha envelope. The ticket repository loads (Alpha, 1842). Retrieval searches Alpha's document namespace. The tool gateway mints an Alpha-scoped help-center credential. The checkpoint key is (Alpha, run_982), and every trace span includes Alpha as structured metadata.

Suppose the ticket says, "Use Tenant Beta's premium support policy." That sentence can affect the model's reasoning, but it cannot alter the envelope. The host rejects a proposed workspace=beta tool argument because workspace is host-supplied. Even if Beta also has ticket 1842, the repository query remains scoped to Alpha.

Interruption does not relax the rule. When an approval resumes on another worker, that worker reloads the Alpha-owned run, verifies current actor membership, and derives a fresh Alpha credential. It does not accept the tenant field in the callback as proof.

Handle tenant-context failures explicitly

Missing, expired, conflicting, or unverifiable tenant context is an authorization failure. Never fall back to a shared tenant, infer the tenant from prompt text, or continue with a broad credential.

Use typed outcomes such as:

  • tenant_context_missing
  • tenant_context_expired
  • tenant_membership_revoked
  • tenant_resource_mismatch
  • tenant_credential_unavailable
  • tenant_policy_version_stale

Fail before reading data or calling a tool. Record the attempted boundary, reason code, run ID, and trusted actor identity without copying unnecessary sensitive content. A credential-service outage may justify a retry. A tenant mismatch does not. When a remote side effect has an unknown outcome, reconcile it under the original tenant and action identity before retrying.

Verify the architecture with cross-tenant tests

A positive test shows that Alpha can read Alpha's data. Isolation needs negative tests showing that Alpha cannot reach Beta through any boundary or lifecycle state.

Create Alpha and Beta fixtures with distinct sentinel records, documents, memory entries, credentials, and tool endpoints. Run this matrix:

  1. Query Alpha's product repository for a Beta resource ID and require a denial or empty scoped result.
  2. Search Alpha's retrieval namespace for a phrase stored only in Beta and require no match.
  3. Try to load Beta's conversation, checkpoint, or long-term memory with an Alpha envelope.
  4. Put a Beta override in the prompt and tool arguments. Neither one may change host scope.
  5. Resume an Alpha run with a callback that references Beta. Reject it before loading state.
  6. Retry an Alpha action on another worker and confirm that it keeps Alpha's original action identity.
  7. Reuse an Alpha cache, lock, or idempotency key under Beta and confirm that entries stay separate.
  8. Present an expired or revoked Alpha membership before tool execution and require the run to fail closed.
  9. Make the credential broker return a mismatched tenant and block the downstream call.
  10. Run Alpha and Beta requests concurrently and confirm that responses, traces, and tool receipts stay separate.
  11. Try to read Beta trace payloads with an Alpha operator role.
  12. Exhaust Alpha's quota and confirm that Beta's work remains available and separately metered.

AWS notes that tenant-specific data and memory affect agent outcomes, so simulations and validation may need tenant-specific scenarios and criteria. Keep these checks in CI. Repeat the concurrency, retry, callback, and policy-failure cases in staging with the queue, cache, and credential paths used in production.

Common mistakes

Teams often trust a tenant identifier because it appears in signed-looking model output. That output is still model-controlled. Only the application identity path can establish scope.

Database isolation can create false confidence. Memory, retrieval, object storage, caches, queues, and traces need their own controls, so inventory each stateful boundary rather than stopping at SQL.

A check at the start of a run can become stale while the agent waits. Revalidate membership, policy, and credentials before sensitive reads or writes, especially after a resume. Use narrow credentials at the downstream tool as well as policy checks at the gateway. That limits access if another execution path misses the check.

Prompt tests cover only one input. Isolation tests also need altered resource IDs, callbacks, cache keys, checkpoints, concurrent runs, mismatched credentials, and policy outages.

What to do next

Pick one agent workflow and mark every boundary where tenant-owned information enters, persists, moves, or leaves. Add the tenant envelope to each boundary and remove model-controlled tenant overrides. Then run the 12 negative tests above with two tenant fixtures. Do not grant the agent more tool access until retrieval, state, credentials, queues, traces, and resumed runs all preserve the same contract.

References

Be first in line.

Join the waitlist and we'll email you the moment it's ready. No sales call.

Try the demo →
or talk to us