Blog · · 9 min read

How to stop AI agent goal drift with a runtime scope contract

How to stop AI agent goal drift with a runtime scope contract

AI agent goal drift happens when a run moves away from the job the user accepted, even though each individual step looks reasonable. A product can authenticate the user, authorize every tool call, and still let the agent turn “clean up these five records” into a wider data-repair project. The fix is a runtime scope contract: persist the accepted outcome and boundaries, compare every plan revision and cumulative effect against them, and require a new decision before the job expands. This guide shows how to build that contract without sending harmless replanning through an approval queue.

Why permission checks do not stop goal drift

Authorization checks whether a principal may perform an action on a resource. A separate scope check asks whether this action still serves the job that principal asked the agent to do.

A sequence of individually permitted actions can still exceed the user's accepted job. A support manager may be allowed to edit tickets, send customer email, and issue credits. If the manager asks an agent to classify five disputed tickets, those permissions do not imply permission to email every affected customer or issue credits. Each tool call may pass access control while the sequence violates the request.

OWASP's excessive-agency guidance recommends minimizing extensions, permissions, and autonomy, checking authorization in downstream systems, and requiring approval for high-impact actions. A scope contract adds another question before execution: is this permitted effect part of this accepted job?

Goal drift can enter through several routes:

  • A subtask creates another subtask whose target set is broader than the original one.
  • A tool result introduces an attractive but irrelevant objective.
  • A long context weakens the original constraints.
  • A delegated agent receives the objective without its exclusions.
  • A completed instrumental goal continues to drive actions after its purpose has ended.
  • A vague reply such as “looks good” is interpreted as permission for new effect classes.

In a Codex issue about committing unrelated files, the reporter says the agent stated a narrow repository policy, then treated a generic confirmation as permission to stage unrelated changes. The report is one user's account, not a universal product guarantee. Knowing or repeating the rule did not enforce it at the action boundary.

Turn the accepted request into durable state

A prompt is not a control record. It can be summarized, truncated, copied without exclusions, or reinterpreted after many steps. Store a normalized scope envelope when the product accepts the request.

AWS Prescriptive Guidance on agent intent and scope recommends defining the outcome, acting principal, autonomy boundary, failure policy, and measurable success criteria. Those fields are a useful starting point, but a runtime needs machine-checkable effect and resource boundaries too.

A minimal envelope could look like this:

{
  "scope_id": "scope_017",
  "principal": "user:1842",
  "outcome": "Classify the five disputed support tickets",
  "targets": ["ticket:41", "ticket:42", "ticket:43", "ticket:44", "ticket:45"],
  "allowed_effects": ["ticket.read", "ticket.classification.write"],
  "excluded_effects": ["customer.email.send", "credit.issue", "ticket.delete"],
  "resource_limits": {"max_tool_calls": 30, "max_runtime_seconds": 300},
  "completion": {"classified_targets": 5, "unresolved_allowed": true},
  "failure_policy": "return_partial_results",
  "revision": 1
}

Do not ask the model to invent this record and then trust it. Resolve targets, effect classes, limits, and principal from product state. Use the model to propose a semantic outcome or flag ambiguity, then let typed application code create the durable envelope. If the user request does not identify a safe target set or effect class, stop for clarification before the first external effect.

The envelope should travel with every queued step, callback, retry, resumed run, and delegated task. A run without a valid scope revision is not eligible to execute a side effect.

Compare plan revisions, not just isolated actions

Plans change during useful work. Blocking every change would turn a capable agent into a brittle script. The runtime should instead require a structured delta whenever the plan changes.

PlanDelta
  added_targets[]
  removed_targets[]
  added_effect_classes[]
  removed_effect_classes[]
  budget_change
  new_recipients[]
  new_external_systems[]
  completion_change
  rationale
  supporting_evidence[]

Classify a delta into three outcomes:

  1. Equivalent: The new step is another way to produce the same permitted effect on the same targets. Continue and record the revision.
  2. Narrower: The plan removes targets, effects, or cost while preserving a useful partial result. Continue if the failure policy permits partial completion.
  3. Expanding: The plan adds a target, recipient, system, effect class, budget, or materially different outcome. Stop before execution and create a new authorization boundary.

Suppose the ticket classifier wants to fetch the customer's billing history to improve its decision. The user has access to billing, and billing.read is a valid tool. The call is still outside the envelope because billing was not an accepted resource for this job. The runtime should return a structured scope violation, not a generic tool error.

Evaluate both the next effect and the cumulative effect set. Twenty individually harmless reads can broaden a target set far beyond five tickets. A plan that stays within the allowed tool list can still add recipients, external systems, or new business outcomes.

Use behavioral evidence instead of asking the agent if it drifted

A self-check prompt such as “Are you still following the user's goal?” is useful as a diagnostic hint. It is not enforcement. A model can repeat the original goal while choosing actions that serve a competing objective.

The primary research report Evaluating Goal Drift in Language Model Agents studies inference-time drift under competing objectives and long contexts. Its methodology treats actions and omissions as the important evidence and compares them with self-reported reasoning. Committed actions and omissions reveal drift more reliably than the agent's self-reported goal.

Build the scope decision from observable records:

  • the accepted envelope and its revision;
  • the proposed plan delta;
  • typed tool arguments and declared effect class;
  • targets and recipients resolved by the product;
  • effects already committed in this run;
  • evidence used to justify the change;
  • completion conditions already satisfied or abandoned.

Use deterministic checks for exact boundaries. Target IDs, recipients, tool classes, destinations, and budgets should not depend on an embedding score. A semantic evaluator can help compare the proposed rationale or outcome with the accepted job, but it should not override a deterministic exclusion. AI Governance Institute's task-boundary control similarly distinguishes structural limits from semantic out-of-scope behavior and recommends runtime monitoring plus adversarial tests.

A safe decision function keeps those responsibilities separate:

def decide_scope(envelope, plan_delta, next_effect, ledger):
    if next_effect.kind in envelope.excluded_effects:
        return "deny"
    if not targets_within(next_effect.targets, envelope.targets):
        return "reauthorize"
    if expands_effect_set(plan_delta, envelope.allowed_effects):
        return "reauthorize"
    if exceeds_budget(plan_delta, envelope.resource_limits, ledger):
        return "reauthorize"
    if semantic_goal_score(envelope.outcome, plan_delta) < REVIEW_THRESHOLD:
        return "review"
    return "allow"

Use the semantic score as evidence for review, not as a claim of mathematical certainty about intent. Calibrate it on your own workflow fixtures and keep an unknown outcome that fails closed for side effects.

Make scope expansion a new authorization boundary

Material plan expansion should create a new authorization boundary, not reuse a generic confirmation. A confirmation is meaningful only when the product shows the proposed change and binds the response to that exact revision.

Present the user with:

  • what new target, recipient, system, effect, or cost is being added;
  • why the current job cannot finish without it;
  • what remains possible if the user declines;
  • which previous scope revision the request replaces;
  • a stable hash or version for the proposed envelope.

An approval should create revision 2 of the envelope. It should not mutate revision 1 in place. Queued work created under revision 1 must not inherit new authority unless the runtime deliberately rebinds and rechecks it. A delayed callback carrying revision 1 should be rejected after the run moves to revision 2.

This avoids approval spam because equivalent and narrower plan changes continue automatically. It also avoids approval laundering because “continue,” “looks good,” or an approval for one effect cannot authorize a different expansion later. AWS's intent-and-scope guidance makes the autonomy boundary explicit at design time. The runtime rule here is a first-principles implementation recommendation: treat a material boundary change as a new decision with its own durable evidence.

Keep a cumulative effect ledger and close with a receipt

Log proposed actions, denied actions, committed effects, skipped targets, and the scope revision used for each decision. Store typed product events instead of chain-of-thought.

At completion, reconcile the ledger against the envelope:

  • Did every committed effect use an allowed class?
  • Did every target belong to the accepted set?
  • Did any recipient or external system appear without a scope revision?
  • Were completion conditions satisfied, or did the failure policy permit the omissions?
  • Did any operation finish after its scope revision expired?

Return a goal receipt with the accepted outcome, final revision, completed targets, committed effect IDs, omitted work, and verification result. A successful model response without this reconciliation is not proof that the run stayed in scope.

Published coverage discusses detection and recovery. For example, Syrin's guide to agent drift covers long contexts, tool feedback, handoffs, semantic-distance monitoring, and recovery. A product still needs cumulative proof. A dashboard can flag suspicious behavior, but the execution boundary must stop an unapproved effect before it commits.

Handle the failure modes explicitly

A scope contract needs defined behavior when its own checks fail.

  • Semantic evaluator unavailable: Allow read-only steps that pass deterministic checks. Block side effects that need semantic review.
  • Plan delta missing: Reject execution. Do not infer that an unreported change is harmless.
  • Tool result proposes a new objective: Treat it as evidence, not instruction. Require a plan delta before adopting it.
  • Subagent delegation: Pass the same or a narrower envelope. Never let a child expand targets or effects.
  • Long context or resumed run: Reload the current envelope and ledger from durable product state. Do not reconstruct authority from conversation text.
  • Completion already reached: Deny further instrumental actions unless a new goal is accepted.
  • Unknown target or effect class: Route to clarification or reauthorization. Do not map unknown to allowed.

The tradeoff is extra runtime work. Typed deltas add orchestration overhead, semantic checks add latency, and users may see more interruptions during immature deployments. Start with effectful workflows where scope errors have real cost. Measure how often equivalent changes are incorrectly escalated, then improve the classifier without weakening deterministic boundaries.

Test scope control with effect-level fixtures

Unit tests should cover more than prompt adherence. Build fixtures around the scope envelope, revisions, and committed effects.

Required cases include:

  • an unrelated resource reachable through an allowed tool;
  • a new effect class on an allowed target;
  • a broadened target set hidden inside a batch call;
  • a new customer or external recipient;
  • a higher cost or iteration budget;
  • continued action after an instrumental goal is complete;
  • a tool result that recommends unrelated work;
  • a generic confirmation after a material plan expansion;
  • harmless replanning with the same targets and effects;
  • a delayed callback carrying an obsolete scope revision.

For every expanding case, assert that no external effect occurs before a new scope decision. For equivalent cases, assert that the run continues without an approval. Finally, reconcile the effect ledger and verify that the goal receipt matches the actual writes, not merely the agent's final summary.

Put the scope gate before your next side effect

Choose one effectful agent workflow. Capture its accepted outcome, targets, effect classes, exclusions, limits, and completion conditions in a versioned scope envelope. Add plan deltas and cumulative-effect checks before the tool executor, then run the ten fixtures above. Do not ship wider autonomy until material expansion blocks before execution and the final goal receipt reconciles with the product's real state.

Be first in line.

Join the waitlist and we'll email you the moment it's ready. No sales call.

Try the demo →
or talk to us