Blog · August 20, 2026 · 10 min read

AI Agent Tool Discovery for Large Product API Catalogs

AI agent tool discovery starts to fail when a mature product puts every action into every model request. The model spends context on tools the user cannot call, sorts through similar schemas, and may depend on every connected backend just to start a session. Picking a shorter list by hand only postpones the same problem. A scalable catalog starts with the current user's authority, finds a small set for the task, loads exact schema versions on demand, and leaves execution behind the product's normal policy checks. This guide explains that boundary and the tests needed at each stage.

Use a two-stage tool boundary

Tool access consists of two related sets:

1. The eligible catalog is the maximum set allowed by the authenticated user, tenant, product plan, and current workflow.
2. The loaded candidate set is the small subset the model needs for this request.

The candidate set must always be a subset of the eligible catalog. Semantic retrieval may reduce an allowed set, but it cannot create authority.

Process each request in this order:

1. Authenticate the user and resolve tenant, role, plan, environment, and workflow constraints.
2. Freeze an eligible catalog snapshot for the run.
3. Search that snapshot with the user's task, product domain, and current state.
4. Load complete schemas for the best candidates.
5. Let the model choose a tool and supply arguments.
6. Authorize the exact call again against current product state.
7. Execute through the service that owns the action, then record the outcome.

These stages distinguish catalog eligibility from discovery relevance, argument generation, and product execution. When a request fails, the trace can identify which boundary rejected it.

Why large tool catalogs fail

Every tool definition occupies prompt space. Names, descriptions, parameters, examples, and annotations compete with the user's request and the model's working context. Anthropic's advanced tool use guidance describes deferred definitions and on-demand search for catalogs with hundreds or thousands of tools. Its code execution with MCP guidance explains the related cost of passing many definitions and intermediate results through model context.

A smaller prompt does not guarantee a better choice, but a large set of similar actions creates obvious ambiguity. Tools named find_customer, search_accounts, lookup_contact, and resolve_identity can all look plausible without enough product context. The Claude Agent SDK's tool search documentation treats context use and selection accuracy as different reasons to load tools on demand.

Large catalogs can also make startup depend on unrelated systems. One repository issue reports slow or failed handshakes when full catalog exposure waits for an unavailable backend. It also reports memory pressure when multiple proxy processes initialize every backend and cache every schema. The proposed configuration exposes a few discovery tools and connects to backends only when needed (1MCP issue #396). That is one author's deployment report, not a general benchmark, but the failure modes are straightforward to reproduce in a test environment.

The catalog keeps changing after a session starts. Teams add tools, retire old versions, disable unsafe actions, and replace backends. MCP provides paginated tools/list requests and optional list-change notifications in the MCP tools specification. Those messages carry discovery data. The product still needs its own version and policy rules.

Design the catalog as product infrastructure

A vector index is not a source of truth. Keep one canonical record for each capability version, then derive search indexes from those records.

Searchable descriptions and executable schemas have different jobs. A search record can include user language, entity names, examples, and negative hints such as "not for bulk updates." The executable schema should stay immutable within a version.

A minimal resolver can look like this:

type ToolRequest = {
  task: string;
  userId: string;
  tenantId: string;
  scopes: string[];
  workflow: string;
};

async function resolveTools(request: ToolRequest) {
  const eligible = await catalog.snapshot({
    tenantId: request.tenantId,
    scopes: request.scopes,
    workflow: request.workflow,
    status: "active",
  });

  const candidates = await search.rank({
    query: request.task,
    toolIds: eligible.toolIds,
    limit: 8,
  });

  const definitions = await catalog.hydrateExact(candidates);
  return definitions.filter((tool) =>
    eligible.versionDigests.has(`${tool.id}:${tool.version}:${tool.digest}`),
  );
}

The limit of eight is only an example. Set the number from recall and selection tests for your own catalog. Two checks matter regardless of the limit: search receives only eligible tool IDs, and hydration verifies exact version digests from the frozen snapshot.

Implement permission-aware AI agent tool discovery

Build the authority filter first

Resolve permissions from authenticated product identity. Do not put tenant, role, or scope decisions in a prompt and ask the model to obey them. Trusted identity context should come from the application and constrain the catalog query.

Filter by tenant, environment, product plan, feature flags, data residency, workflow stage, and required scopes before ranking for relevance. If a user cannot call refund_payment, its name and description should not appear in their search results. This avoids disclosing the capability and prevents it from influencing the model's choice.

Eligibility does not replace invocation authorization. A tool may be eligible at the beginning of a run but unauthorized by the time the model calls it. Account state, resource ownership, and approval requirements can change. Check the exact arguments immediately before execution.

Index tasks, not only tool names

A tool name is a weak retrieval document. Index a compact capability card with:

• the user job completed by the tool;
• the entities it reads or changes;
• required preconditions and side-effect class;
• common phrases and product synonyms;
• guidance for choosing a nearby tool instead;
• the stable tool ID and schema version.

Use lexical matching for exact product terms and semantic matching for natural requests. Apply hard product filters first, then combine retrieval signals inside the eligible set. Spring AI's dynamic tool discovery guide shows the basic pattern: retrieve relevant tools and expand only those definitions into model context.

A few tools may be present in every request, but make that an explicit decision. Help and capability search might qualify. A broad write tool does not.

Hydrate exact schemas only after retrieval

A search result identifies a capability. It is not an executable definition. After ranking, load the complete schema from the canonical catalog and verify its ID, version, digest, availability, and backend binding against the run snapshot.

That check prevents a stale search record from invoking a changed schema. It also lets a team improve search wording without changing the executable contract. When a version is deprecated, the resolver should receive explicit replacement metadata rather than forcing the model to guess.

For MCP servers, feed tools/list into a catalog synchronizer rather than treating it as the product's policy database. Paginate deliberately. Record the server identity and observed version. Process list-change notifications through the same validation path used for initial registration.

Keep discovery separate from execution

The discovery service returns candidate definitions. It should not hold product superuser credentials or execute arbitrary calls. The runtime passes the selected tool ID, version, arguments, run identity, and catalog snapshot ID to the owning service or gateway.

That execution layer checks current authorization, validates arguments, applies idempotency and approval rules, and records a typed outcome. Relevance alone never proves that a call is authorized or safe.

This split produces useful telemetry. Record which tools were eligible, which candidates were returned, which definition the model selected, why execution was allowed or denied, and which version ran. Discovery debugging rarely requires sensitive argument values, so keep them out of these records.

Handle catalog and backend failures

Discovery returns no candidates

Do not respond by exposing the full catalog. Return a typed no_relevant_tool result with the search terms, applied filters, and catalog snapshot ID. The product can request missing context or route the task to a deterministic workflow.

A backend is unavailable

Mark the affected tools unavailable or degraded and exclude them from new candidate sets. An existing run that references the backend should receive a typed availability failure. A write operation should fail closed instead of switching to an unverified substitute.

Connecting lazily can reduce startup coupling, but backend health still needs an owner. Monitor the discovery service, catalog synchronizer, and execution backends as different components.

A schema changes during a run

Bind the run to an immutable catalog snapshot. Execute the exact version while it remains supported. If the version has been revoked for safety, return tool_version_revoked and require the runtime to plan again against a fresh snapshot. Never translate old arguments into a new schema silently.

Search is unavailable

Some products may need a minimal, deterministic group of read-only tools for basic operation. Otherwise, return discovery_unavailable. Loading every tool as an emergency measure changes authority exposure and model behavior during an incident.

Catalog notifications are missed

Reconcile the catalog against canonical server and product registries on a schedule. Notifications improve freshness but should not be the only correctness mechanism. Alert when the search index, catalog database, and backend inventory disagree on active versions.

Verify discovery before rollout

Test each boundary on its own. One end-to-end success rate cannot show whether the catalog, model, or product API caused a failure.

Catalog integrity tests

• Every active search record maps to one immutable schema version.
• Every version digest matches the hydrated definition.
• A deprecated tool has a valid replacement or an explicit terminal state.
• Disabled and cross-tenant tools never enter an eligible snapshot.
• Reconciliation detects tools that were added, removed, or changed.

Discovery tests

Build a dataset of real product requests, acceptable tools, forbidden tools, user scopes, tenant context, and workflow state. Measure whether the candidate set contains an acceptable tool. Track unauthorized candidates, irrelevant candidates, set size, and search latency as separate results.

Include hard negatives. "Show the last refund" should find a read tool, not issue_refund. "Draft a cancellation message" should not select cancel_subscription. Repeat the cases with similar tool names and missing context.

Selection and argument tests

Give the model a fixed candidate set, then check whether it selects an acceptable tool and supplies valid arguments. This isolates model behavior from retrieval. Include deprecated versions, ambiguous entities, missing required values, and misleading descriptions.

Execution and failure tests

Give the execution layer a fixed tool call. Test current authorization, tenant isolation, validation, idempotency, approval, typed errors, and audit records. Then inject a dead backend, stale search record, revoked schema, delayed list notification, and unavailable discovery service.

Report the release gate as a chain of measured stages:

catalog integrity -> discovery recall and precision -> tool selection
-> argument validity -> authorization -> product outcome

A high final success rate can hide an unauthorized candidate leak. A low rate may come from a broken product endpoint even when discovery worked. The stage results remove that ambiguity.

Common mistakes

Retrieving before filtering permissions

Searching the global catalog and removing unauthorized tools afterward can disclose capability names. Forbidden tools can also influence ranking before they are removed. Build the eligible set first.

Letting semantic search define the executable contract

Embeddings and search records help with retrieval. They are not versioned schemas. Load and verify the canonical definition after retrieval.

Testing only the final answer

A plausible answer may come through the wrong tool, an unauthorized action, or a stale schema. Assert the candidate set, selected version, policy decision, arguments, and product outcome.

Treating lazy loading as only a token optimization

Deferred loading also changes startup dependencies and failure isolation. Following the concrete 1MCP report, test session startup with slow, failed, and partially registered backends.

Adding a broad catch-all tool

A generic call_api tool defeats discovery, schema safety, authorization clarity, and useful evaluation. Business actions should remain explicit even when the runtime loads their definitions on demand.

What to do next

Pick one workflow that exposes at least 20 tools. Capture 30 representative requests with allowed and forbidden tools. Build the eligible snapshot, retrieve a bounded candidate set, load exact versions, and score discovery separately from selection and execution. Do not connect another backend until the test report shows all four stages.

References

• Anthropic: Introducing advanced tool use supports deferred tool definitions, tool search, and on-demand access to large catalogs.
• Anthropic: Code execution with MCP supports the context and intermediate-result costs of large MCP integrations.
• Claude Agent SDK: Scale to many tools with tool search supports dynamic discovery and the separation between context efficiency and tool-selection accuracy.
• 1MCP issue #396 is a practitioner report about startup coupling, failed backends, process memory pressure, and lazy loading.
• MCP specification: Tools defines paginated tool listing and list-change notifications.
• Spring AI: Dynamic Tool Discovery with Tool Search Tool provides a provider-neutral retrieval and selective-schema expansion implementation.