Discovery: Finding the Right Tool

At three tools you can hand the model every definition on every turn and never feel it. At three hundred you cannot — and the cost is not just size. Every definition in the prompt is text the model reads again on every turn, so a bloated tool list is a recurring tax on tokens and latency. Worse, a sea of options makes the model pick worse, not better: choice overload is real, and a model facing fifty similar tools reaches for the wrong one more often than one facing the five that actually fit.

Discovery is how a toolset scales. Instead of dumping every definition into the prompt, you keep a registry of all of them and expose only the subset relevant to the current task. This lesson builds that registry, selects from it by tag and keyword, and measures the token cost that selection saves.

flowchart LR
  R["registry: many tools"] --> SEL["select by tag or keyword"]
  SEL --> SUB["small relevant subset"]
  SUB --> M["model"]
  R -.-> ALL["expose ALL tools"]
  ALL -.-> BLOATED["model: bloated, confused"]
  style SUB fill:#166534,color:#fff
  style BLOATED fill:#b91c1c,color:#fff

Select a small relevant subset for the model, instead of exposing the whole registry.

The registry and its catalogue

A registry is the master list of every tool definition your system knows about — its name, description, and schema — plus whatever metadata helps you find it. The most useful metadata is a set of tags or a category: search, email, read-only, destructive. Tags are how you group tools so selection is a filter, not a search through prose.

Two operations do the work. Listing returns the whole catalogue — useful for an admin console, never for the model. Selecting returns the subset that matches a tag or keyword, and that subset is what you send to the model this turn. Keeping 'everything we have' and 'what the model sees right now' as two different operations is the whole shape of discovery: one is inventory, the other is curation. The catalogue changes rarely; the selection changes with every task.

A registry and selectByTag: filtering by tag returns just the matching tools. Press Run.

const registry = [
  { name: 'search_web', description: 'Search the public web', tags: ['search', 'web'] },
  { name: 'search_docs', description: 'Search internal documentation', tags: ['search', 'docs'] },
  { name: 'send_email', description: 'Send an email to a user', tags: ['email', 'send'] },
  { name: 'create_issue', description: 'Create a tracking issue', tags: ['issue', 'create'] }
];

function selectByTag(registry, tag) {
  return registry.filter(t => t.tags.includes(tag));
}

console.log(selectByTag(registry, 'search').length);
console.log(selectByTag(registry, 'email').length);

Selection and the token bill

Selecting is a filter: walk the registry and keep the entries whose tags contain the one you asked for, or whose description mentions a keyword. The result is a smaller array of definitions, and that array — not the whole registry — is what the model sees. Because each definition contributes its name and description to the prompt on every turn, the cost of a tool set is roughly the sum of those lengths. Cutting the exposed set from a hundred tools to five cuts that recurring bill by roughly twenty times, every turn, for the whole conversation.

That is why discovery is not an optimisation you bolt on later. It is the difference between an agent that is cheap and focused and one that is expensive and distracted from its very first turn.

The token bill, made concrete: the full registry costs more than twice what the search subset does. Press Run.

const registry = [
  { name: 'search_web', description: 'Search the public web', tags: ['search', 'web'] },
  { name: 'search_docs', description: 'Search internal documentation', tags: ['search', 'docs'] },
  { name: 'send_email', description: 'Send an email to a user', tags: ['email', 'send'] },
  { name: 'create_issue', description: 'Create a tracking issue', tags: ['issue', 'create'] }
];

function selectByTag(registry, tag) {
  return registry.filter(t => t.tags.includes(tag));
}
function tokenCost(tools) {
  return tools.reduce((sum, t) => sum + t.name.length + t.description.length, 0);
}

console.log(tokenCost(registry));
console.log(tokenCost(selectByTag(registry, 'search')));

flowchart TD
  ALL["expose every tool"] --> H1["cost: every definition repeats each turn"]
  ALL --> H2["misuse: a sensitive tool is callable"]
  ALL --> H3["confusion: too many options, worse picks"]
  style H1 fill:#b45309,color:#fff
  style H2 fill:#b91c1c,color:#fff
  style H3 fill:#b45309,color:#fff

Exposing every tool harms three ways at once: cost, misuse, and confusion.

When nothing matches

A selector has to decide what happens when the query fits nothing. The safe answer is fail closed: return an empty list, so the model sees no tools and the caller handles the gap explicitly. The tempting answer is fail open — return everything 'just in case' — but that surrenders the point of discovery and dumps the full registry back into the prompt. Fail-closed matters most for destructive tools: a selector that fails open on a vague query can hand the model a delete_record tool it should never have seen.

Matching should be case-insensitive. Descriptions are written by humans and queried by models, and the two will not agree on capitalisation, so a search for email should still find Send an Email. Normalising both sides to lowercase before comparing prevents a whole class of 'the tool is there but never found' bugs.

discover matches descriptions case-insensitively and caps the result. Press Run.

const registry = [
  { name: 'search_web', description: 'Search the public web', tags: ['search', 'web'] },
  { name: 'search_docs', description: 'Search internal documentation', tags: ['search', 'docs'] },
  { name: 'send_email', description: 'Send an email to a user', tags: ['email', 'send'] }
];

function discover(registry, query, limit) {
  const k = query.toLowerCase();
  const matches = registry.filter(t => t.description.toLowerCase().includes(k));
  return matches.slice(0, limit);
}

console.log(discover(registry, 'search', 10).length);
console.log(discover(registry, 'search', 1).length);

flowchart TD
  GROW["registry grows: 5 then 50 then 500 tools"] --> NOPE["expose all: cost grows with it"]
  SEL["select a fixed subset each turn"] --> FLAT["cost stays small and flat"]
  style NOPE fill:#b91c1c,color:#fff
  style FLAT fill:#166534,color:#fff

Without selection, cost climbs as the registry grows; selection keeps it small and flat.

Exercise

What does this print? Each tool's size is its name length plus its description length; total sums them.

const tools = [
  { name: 'add', description: 'Add two numbers' },
  { name: 'get', description: 'Get a record' }
];
function size(t) { return t.name.length + t.description.length; }
function total(tools) { return tools.reduce((s, t) => s + size(t), 0); }
console.log(total(tools));
console.log(tools.length);

Your prediction — what does this print?

Exercise

Write selectByTag(registry, tag) that returns the entries whose tags array includes tag. The registry is already defined for you.

const registry = [
  { name: 'search_web', description: 'Search the public web', tags: ['search', 'web'] },
  { name: 'search_docs', description: 'Search internal documentation', tags: ['search', 'docs'] },
  { name: 'send_email', description: 'Send an email to a user', tags: ['email', 'send'] },
  { name: 'create_issue', description: 'Create a tracking issue', tags: ['issue', 'create'] }
];
function selectByTag(registry, tag) {
  // return entries whose tags include tag
}

const registry = [
  { name: 'search_web', description: 'Search the public web', tags: ['search', 'web'] },
  { name: 'search_docs', description: 'Search internal documentation', tags: ['search', 'docs'] },
  { name: 'send_email', description: 'Send an email to a user', tags: ['email', 'send'] },
  { name: 'create_issue', description: 'Create a tracking issue', tags: ['issue', 'create'] }
];
function selectByTag(registry, tag) {
  return registry.filter(t => t.tags.includes(tag));
}

Exercise

Write tokenCost(tools) that returns the total length of every tool's name plus description, summed across the array. (A rough proxy for how many tokens a tool set adds to the prompt.)

function tokenCost(tools) {
  // sum of name.length + description.length across all tools
}

function tokenCost(tools) {
  return tools.reduce((sum, t) => sum + t.name.length + t.description.length, 0);
}

Exercise

selectByKeyword(registry, keyword) should return tools whose description mentions the keyword. But it compares with exact case, so searching search misses a tool described as Search the web. Fix it to match case-insensitively.

const registry = [
  { name: 'search_web', description: 'Search the public web', tags: ['search'] },
  { name: 'send_email', description: 'Send an email', tags: ['email'] }
];
function selectByKeyword(registry, keyword) {
  return registry.filter(t => t.description.includes(keyword));   // BUG: case-sensitive
}

const registry = [
  { name: 'search_web', description: 'Search the public web', tags: ['search'] },
  { name: 'send_email', description: 'Send an email', tags: ['email'] }
];
function selectByKeyword(registry, keyword) {
  const k = keyword.toLowerCase();
  return registry.filter(t => t.description.toLowerCase().includes(k));
}

Exercise

Write discover(registry, query, limit) that returns up to limit tools whose description contains query, matched case-insensitively, in registration order. The registry is already defined for you.

const registry = [
  { name: 'search_web', description: 'Search the public web', tags: ['search', 'web'] },
  { name: 'search_docs', description: 'Search internal documentation', tags: ['search', 'docs'] },
  { name: 'send_email', description: 'Send an email to a user', tags: ['email', 'send'] },
  { name: 'create_issue', description: 'Create a tracking issue', tags: ['issue', 'create'] }
];
function discover(registry, query, limit) {
  // case-insensitive description match, capped at limit, in order
}

const registry = [
  { name: 'search_web', description: 'Search the public web', tags: ['search', 'web'] },
  { name: 'search_docs', description: 'Search internal documentation', tags: ['search', 'docs'] },
  { name: 'send_email', description: 'Send an email to a user', tags: ['email', 'send'] },
  { name: 'create_issue', description: 'Create a tracking issue', tags: ['issue', 'create'] }
];
function discover(registry, query, limit) {
  const k = query.toLowerCase();
  const matches = registry.filter(t => t.description.toLowerCase().includes(k));
  return matches.slice(0, limit);
}

Recap

A registry holds every tool definition; listing returns it all (for admins), selecting returns the subset the model should see this turn.
Tags turn selection into a cheap filter by category; keywords match against the description.
Each exposed definition costs tokens every turn, so a tool set's cost is roughly the sum of name + description lengths — discovery keeps that bill small and flat.
Exposing everything harms three ways at once: cost, misuse (a sensitive tool becomes callable), and confusion (choice overload degrades the model's picks).
When nothing matches, fail closed (return empty), never fail open (return everything).
Match case-insensitively so human-written descriptions and model-issued queries agree.

Next you will bundle tools with the instructions that govern them into a skill, loaded on demand so the context stays lean.

Discovery: Finding the Right Tool

The registry and its catalogue

Selection and the token bill

When nothing matches

Recap

Checkpoint quiz

Go deeper — technical resources