AI Technical Articles
Gemini Tool Retrieval: How Deferred Schemas Affect Calls and Costs
Gemini Tool Retrieval is worth testing when an assistant carries a large catalogue of tools but uses only a few for each task. The useful question is whether loading their detailed input rules on demand improves the complete workflow, including the extra work needed to find a tool.
Google introduced the feature in preview on November 19, 2026. It lets selected tool schemas stay outside the model’s initial context. Below, we follow a request from discovery to execution, separate model context from the API payload, and work through what to count before calling it a saving.

The important boundary is that retrieving a tool definition is not the same as executing the tool. For custom functions, the developer’s application still performs the real operation and returns its result.
Google’s current setup guide requires the v1beta version of the Interactions API, the interface applications use to exchange requests and results with Gemini. Its generateContent method does not support Tool Retrieval. Check the guide’s supported-model list for the exact model you intend to call.
The problem Tool Retrieval is designed to solve
A tool is an external capability that a model can request. An application might expose functions for searching documents, checking an order or issuing a refund.
A custom function normally includes a parameter schema, the structured description of the inputs it accepts. A refund function, for example, might require an order ID and an amount. The schema tells the model the field names, types and required inputs; it does not prove that the proposed refund is allowed. The model’s context is the material it can read while generating an answer. Tokens are the small units of text used to count much of that input and output.
When dozens or hundreds of such functions are available, repeatedly supplying their complete schemas can consume substantial context even though most functions are irrelevant to an individual request. Google’s announcement identifies three potential consequences of large tool libraries: more input tokens, additional latency and harder tool selection.
Those are Google’s stated motivations for the feature, not a guarantee of improvement for every application. The announcement does not provide a universal percentage reduction in tokens, latency or errors. Developers should measure the effect on their own workloads.
How Tool Retrieval changes function calling
Without retrieval, an application can provide all available function definitions and their parameter schemas up front. The model chooses one and emits a structured call, which the application then executes.
Tool Retrieval changes what has to be loaded before that decision.
Developers add a tool_search tool and mark selected functions with defer_loading: true. With this server-side mode, Gemini initially sees each deferred function’s name and compact description, while its parameter schema stays outside the model context. The optional short_description field can supply that summary. The developer still supplies the full parameter schema in the request, so the service has it available to retrieve.
When Gemini decides that a deferred function is relevant, Google documents this sequence:
- Gemini emits a
tool_search_call. - The API returns a
tool_search_resultcontaining the selected function’s full schema. - Gemini uses the schema to produce a regular
function_call. - The developer’s code executes that custom function and returns a
function_result.
The search, retrieval and function-call stages appear in interaction.steps, which gives developers a way to inspect tool selection while debugging names and descriptions.
The catalogue-and-manual analogy describes what the model reads. It does not describe what travels over the network: the full tools array still has to accompany every request, including a request that returns a function result and one that uses previous_interaction_id to continue a conversation. Google’s multi-turn instructions make this distinction explicit. Deferral reduces model-context exposure; it does not itself reduce the tool declaration payload your client sends.
Retrieval does not mean the real-world action happened
Google’s announcement demonstrates the process with a fictional damaged order, A-1042, and a request to refund 40 dollars. In an example containing more than 50 deferred functions, Gemini searches for issue_refund, receives its schema and generates a function call containing the order ID and amount.
That demonstrates tool discovery and function-call generation. It does not demonstrate a completed refund.
The application must check the request, perform the permitted operation and return its actual result. A generated call is not evidence that the payment provider completed it.
That distinction is important for tools that modify data or trigger transactions. In a production commerce system, retrieving issue_refund does not establish that order A-1042 exists, that the refund is permitted, or that the caller is authorized to approve it. Those checks remain part of the application’s execution and authorization design.
A selection test that can fail before execution
Consider an invented support assistant with shipping_get_status and billing_get_refund_status. For “Has my replacement shipped?”, discovering the refund-status schema is a selection error even if the call has valid arguments. Describe each function’s service and purpose clearly, then test both similar requests and cases with no matching tool.
Use a test account and a simulated executor first. Log the expected tool, retrieved tool, generated arguments, authorization decision and returned outcome separately. A request rejected for lack of permission can be the correct system behavior; it should not be counted as a successful real-world action merely because retrieval worked. This is a proposed evaluation, not a report of a test we ran.
Where Tool Retrieval makes the most sense
The strongest candidate is an application with a large, unevenly used tool catalogue.
If an assistant relies on a small group of functions frequently but has many specialized functions for occasional requests, the frequently used tools can stay loaded while the long tail is deferred. Google’s own example recommends leaving frequent tools undeferred rather than treating retrieval as an all-or-nothing choice.
Google provides two ways to handle the search. They are mutually exclusive within one request.
Server-side retrieval
In the default approach, Gemini performs the search. There is no separate retrieval index for the developer to build or host.
This reduces the infrastructure required when the application’s declared catalogue is sufficient and the developer does not need custom search logic.
Client-side retrieval
Client-side retrieval instead lets the application perform the search and provide matching tool definitions. This can fit systems that already maintain a tool registry or need their own filtering logic, and it also allows tools to be introduced during an ongoing conversation. Each search requires a return trip from the application to the API. In this mode, define one named search tool with execution: "client", a description and search-argument schema; return matching function declarations as objects in its function_result. Do not combine this mode with defer_loading, which belongs to server-side retrieval.
The architectural question is therefore not simply whether retrieval is useful. It is also who needs to control which tools are discoverable for a particular request.
A permission-filtered registry can restrict discovery, but it is only one layer. Recheck authorization when executing a call, especially if the user’s access changes during the conversation. Hiding a schema is not an access-control mechanism by itself.
Does Tool Retrieval save money?
Potentially, but not automatically.
Google says Tool Retrieval has no separate fee. Normal input and output token charges still apply. The billing guide adds a crucial detail: a server-side search triggers another model call within the request. Its prompt contains the context again plus the loaded schemas. The first prompt appears in usage.total_input_tokens; these additional prompts appear in usage.total_tool_use_tokens and are billed at the input rate. Inspect both, along with cached, output and thinking usage. A single client API request can therefore contain multiple billable model passes.
The economics therefore depend on workload shape. Retrieval is most promising when the amount of schema text avoided across typical requests exceeds the extra search overhead and the schemas that actually need to be loaded.
As checked November 20, 2026, Google’s Standard paid-tier prices list Gemini 4 Hydrogen at $1.50 per 1 million input tokens and $7.50 per 1 million output tokens on its paid tier, with the model also available on a free tier. Those are model prices, not a special Tool Retrieval tariff.
Worked estimate: count the search pass too
Assume two client requests, each with the same 10,000-token non-tool context. A hypothetical catalogue has 100 schemas of 300 tokens each. Under retrieval, assume a 3,000-token compact list, 200 tokens of search instructions and one search loading two schemas, or 600 tokens. These are assumed token counts. Exclude caching, output/thinking charges, changing conversation length and external service fees to isolate input costs.
- Static loading:
2 × (10,000 + 30,000) = 80,000input tokens. - Retrieval: the first prompt is
10,000 + 3,000 + 200 = 13,200. The search pass is13,200 + 600 = 13,800; the follow-up request is another 13,800. Total: 40,800 input-rate tokens. - At $1.50 per million uncached input tokens, those input-only estimates are $0.12 and $0.0612 for this two-request example, a difference of $0.0588.
A smaller catalogue saves less, while repeated searches over a large conversation can add substantial input charges. Loaded schemas remain in context for later calls and cannot be unloaded during that conversation. Replay complete conversations, rather than comparing only the first prompt, before expecting a lower bill.
A practical way to test it
The safest first step is a controlled comparison rather than redesigning an entire agent.
Take a representative set of requests from an application with a substantial tool catalogue. Keep common functions fully loaded and defer less frequently used functions. Run the same workload with and without retrieval, then record:
- first-prompt input, additional tool-use input and cached tokens, plus output/thinking charges;
- end-to-end response time;
- which deferred tool was retrieved;
- which function was ultimately called; and
- whether that function matches the expected tool.
Also test the second request, not just the first. With server-side retrieval, resend the complete tools array. If your application manages its own conversation history, preserve the returned retrieval steps and their signatures unchanged, as explained in the stateless-conversation guide. A successful first call can otherwise be followed by an error when you return the function result.
When Gemini retrieves or calls the wrong function, inspect interaction.steps. Because deferred tools are initially represented by their names and descriptions rather than full parameter schemas, unclear or overlapping descriptions are a sensible first debugging target. Google specifically exposes these steps to help developers inspect this selection process.
The useful question is not whether Tool Retrieval sounds more efficient. It is whether your application’s real traffic uses fewer input tokens without an unacceptable change in latency or correct tool selection.
Availability
As checked November 20, 2026, Tool Retrieval is a preview feature in Google’s Gemini Interactions API rather than a generally available capability. The launch announcement describes support for Gemini 3.6 and later, including the Gemini 4 family. The setup guide currently names gemini-4-hydrogen, gemini-4-argon, gemini-3.8-flash and gemini-3.6-flash; check its live list before choosing a model rather than assuming every family member is covered.
That preview status matters for production planning. Applications adopting it should treat the current API behavior and compatibility as subject to change and check Google’s current documentation before deployment.
A decision worth testing
Keep a small, frequently used tool set loaded and test deferral on the specialized remainder. Adopt it when complete conversations show a useful cost or latency improvement while preserving correct selection and execution controls. Keep static loading when the added search work outweighs what it avoids.
Discussion