▶️ LISTEN TO THIS ARTICLE

Agent Tool-Use Patterns: How LLMs Actually Wield APIs

Many major model providers now support function calling or tool-use interfaces. OpenAI, Anthropic, Google, and a growing set of open-weight models let developers define tool schemas and receive structured outputs. The interface is maturing; reliability still depends heavily on schema design, validation, and orchestration.

On the Berkeley Function Calling Leaderboard (BFCL) V4, recent snapshots suggest leading models cluster in a similar band, but the exact ordering changes as models and benchmark versions update. Treat the benchmark as a point-in-time signal. Production results can be lower when schemas are messy, tools are numerous, and failures compound across steps.

The gap between "can call a function" and "can use tools reliably across a multi-step workflow" is where agent projects become fragile. This guide covers the common tool-use lifecycle, failure modes, and emerging standards that try to make the stack easier to validate. As of June 2026, the practical lesson is still the same: verify the current provider docs and benchmark versions before treating any of the examples below as production defaults.

The Tool-Use Lifecycle

Tool use isn't one skill. It's six, executed in sequence, and an agent can fail at any step.

Discovery is knowing what tools exist. In early systems, this meant stuffing every function signature into the system prompt. That can work for small tool catalogs and become brittle as the catalog grows. AnyTool introduced hierarchical retrieval structures that use semantic search to inject only the most relevant top-K tool descriptions into the context window. The tradeoff: retrieval adds latency and can miss the right tool if the user's intent doesn't match the tool's description.

Selection is choosing which tool to call. This is where the ReAct framework remains influential, interleaving reasoning traces with action steps. But ReAct can have an expensive bottleneck: it may invoke the LLM at every step to decide which tool to use next. AutoTool, published in November 2025, sidesteps this by building a directed graph from historical agent trajectories. Nodes are tools, edges are transition probabilities. The reported result is lower inference cost while maintaining task completion quality in that setting.

Parameter construction is generating the correct arguments. This is harder than it sounds. A comprehensive analysis presented at EMNLP 2025 categorized multi-step tool call errors into five patterns: tool selection errors, tool hallucination errors, parameter key errors, parameter value errors, and environment errors. Missing required parameters, hallucinating parameter names that don't exist in the schema, and filling values with plausible-sounding but incorrect data are all common. Models that write flawless Python routinely botch start_date vs date_start in an API call.

Execution is running the tool and getting a result. The agent doesn't execute anything directly. It hands structured JSON to your application layer, which calls the API, runs the query, or hits the database. Your code handles retries, rate limits, and timeouts. The agent just waits.

Result interpretation is reading what came back and deciding what to do next. This is where context windows matter. A tool that returns 50KB of JSON can blow out the context budget for the rest of the conversation. Smart implementations truncate, summarize, or extract specific fields before feeding results back to the model.

Chaining is doing it all again with the next tool. Multi-step workflows create compounding error rates. Even modest per-step failure rates compound quickly across a five-step chain, which is why production agents feel unreliable even when individual tool calls look fine.

In automatic mode, Gemini executes the function call and feeds the result back without developer intervention.

How the Big Three Implement It

The three major providers have converged on similar interfaces but diverge on important details.

OpenAI's function calling defines tools as JSON Schema objects passed in the tools parameter. The model returns a tool_calls array with function names and arguments. OpenAI supports parallel tool calls by default, letting the model invoke multiple functions simultaneously when the calls are independent. You can disable this with parallel_tool_calls=False when order matters.

Anthropic's tool use follows the same pattern but wraps it in a content-block architecture. Claude returns tool_use content blocks with id, name, and input fields. You respond with tool_result content blocks that reference the original ID. Anthropic's documentation emphasizes the agent loop pattern: send message, receive tool call, execute, send result, let the model continue.

Google's Gemini function calling supports both automatic and manual modes. In automatic mode, Gemini executes the function call and feeds the result back without developer intervention. In manual mode, you get the structured call and handle execution yourself. Google also supports ANY mode, which forces the model to predict a function call, useful when you want tool use to be required rather than optional.

The practical differences matter less than they used to. The Model Context Protocol (MCP) is pushing tool interfaces toward a shared standard, which is turning integration work into infrastructure rather than a one-off wiring exercise. The important shift is not any single adoption milestone; it is that more of the ecosystem is converging on the same protocol shape.

For a broader comparison of the frameworks that wire these tool-calling APIs into agent architectures, see our framework roundup.

The Selection Problem

Choosing the right tool from a large catalog is often the hardest step in the lifecycle. Research and practitioner reports point in the same direction: as the number of available tools grows, selection can become less reliable.

Toolformer was one of the earliest approaches, fine-tuning models to decide when and how to call tools by embedding API documentation directly in the training data. The limitation is the context window. Stuffing 200 tool descriptions into a prompt doesn't scale, and even if it fits, the model's attention over that many options degrades.

Gorilla, from UC Berkeley, took a different approach: fine-tuning LLaMA specifically on API call generation, using a retriever to pull relevant API documentation at inference time. Gorilla surpassed GPT-4 on API call accuracy in its initial benchmarks, but the model was specialized. That result should not be read as general capability superiority outside the evaluated API-calling setting.

The current generation of solutions splits into two camps. Retrieval-based approaches like AnyTool treat tool selection as an information retrieval problem, searching for the right tool at runtime. Graph-based approaches like AutoTool treat it as a prediction problem, using historical patterns to anticipate the next tool. Both reduce inference costs. Neither solves the cold-start problem, where an agent encounters a tool it has never used before and must reason from the documentation alone.

This cold-start scenario is one thing benchmarks try to measure and production systems often face. When your agent hits an unfamiliar API, it is back to reading the schema and reasoning from examples.

Failure Modes That Kill Production Systems

Four failure patterns show up repeatedly in production tool use, and they can be more stubborn than benchmarks suggest.

Parameter hallucination is the most common. The model generates arguments that look structurally valid but contain invented values. A date field gets 2025-13-45. A user ID gets a plausible-looking but nonexistent string. An enum field gets a value that isn't in the allowed set. The JSON parses. The API rejects it. Or worse, the API accepts it and returns wrong data silently.

Tool hallucination means the model calls a tool that doesn't exist. It invents a function name based on what it thinks should be available. This can happen more often when the model has seen similar tools in training data but the current environment offers a different set. Research on internal representations published in January 2026 found that hallucinated tool calls show distinct patterns in the model's late-layer activations, which the authors interpret as evidence that the model may have signals of fabrication before proceeding.

Interaction collapse is the failure mode explored in detail in When Your Agent Stops Using Tools. Models trained with reinforcement learning can gradually abandon tool use and substitute internal reasoning, simulating what a tool would return instead of calling it. The model reasons about the calculator instead of using it. The outputs can look reasonable while being wrong often enough to matter.

Cascading failure occurs in multi-step chains when an error in step two corrupts steps three through five. The model doesn't recognize that a tool returned an error or returned partial data, and builds subsequent tool calls on a broken foundation. FutureAGI's analysis of production tool chains found that agents often begin with correct reasoning and valid tool selections but degrade mid-execution, with many failures traced to malformed JSON output, loss of structure, or the model forgetting earlier decisions.

Over time, the model learns that internal simulation is lower-variance than external tool calls that might fail, time out, or return unexpected formats.

The Reasoning Trap

Here's the finding that should worry agent developers. A paper from October 2025 reported evidence of a causal link between reasoning enhancement and tool hallucination in its experimental setup. In that setting, reinforcement learning that improved reasoning also increased tool hallucination.

The mechanism is intuitive once you see it. RL rewards the model for getting correct answers. If the model can simulate a tool call internally and get the right answer, that's rewarded just as much as actually calling the tool. Over time, the model learns that internal simulation is lower-variance than external tool calls that might fail, time out, or return unexpected formats. So it substitutes reasoning for action.

The researchers tested mitigation strategies. Prompt engineering offered minimal relief in their setup. Direct Preference Optimization (DPO) reduced hallucinations but degraded tool-use proficiency. The paper frames this as a capability-reliability tradeoff: in the studied setup, making models better reasoners worsened tool-use behavior, and mitigation reduced some reasoning performance.

This tradeoff has practical implications for anyone building agents. You can't just train a model to be smarter and expect it to use tools more reliably. The training objective has to explicitly reward correct tool use at each step, not just correct final answers. ToolRLA, a March 2026 paper, demonstrates one approach: a multiplicative reward that decomposes into format validity, tool selection accuracy, parameter correctness, and domain compliance. In the reported financial advisory setting, the method improved task completion and reduced tool invocation errors over time.

Building Reliable Tool-Use Agents

Given these failure modes, here is practical guidance for production.

Constrain the tool catalog. Agents perform better with fewer, well-documented tools than with expansive catalogs. If your agent needs access to 100 APIs, use a retrieval layer to surface only the 5-10 most relevant ones per query. Every tool in context is a potential distraction.

Validate before executing. Don't pass the model's tool call directly to your API. Check required parameters exist. Validate types. Confirm enum values against the allowed set. Reject malformed calls and ask the model to retry. This simple guardrail can catch many parameter hallucination errors.

Add reasoning checkpoints. Anthropic's think parameter approach asks the model to articulate why it is choosing a specific tool before calling it. As covered in Tools That Think Back, this structural change can improve accuracy without large architectural modifications. The model may catch some errors when forced to explain itself.

Use dense reward signals for training. If you're fine-tuning or running RL on tool-use tasks, sparse terminal rewards (right answer = reward, wrong answer = nothing) can be brittle. Intermediate rewards at each tool-calling step give the optimizer gradient signal through the middle of long trajectories.

Design for graceful degradation. Your agent may call the wrong tool. Your API may time out. The response may be truncated. Build retry logic, fallback paths, and explicit error handling into the orchestration layer, not the prompt. The agent should know that a tool call failed and have a defined recovery path, not just try again with the same parameters.

Monitor tool-use patterns in production. Track which tools get called, in what order, with what success rates. Observability is how you find failing tool calls before they compound into workflow failures.

Where This Is Heading

The tool-use stack is consolidating. MCP is becoming an important interface layer. Frameworks like LangGraph, AutoGen, and CrewAI are converging on similar orchestration patterns. The research is shifting from "can models call functions?" to "can models use tools reliably across long workflows?"

The hardest unsolved problem is what the survey on tool-use evolution calls the shift from isolated invocation to multi-tool orchestration. Modern benchmarks now model topological complexity along a spectrum from sequential chains to directed acyclic graphs, where tool execution order follows causal dependencies. We're not just asking models to call one API. We're asking them to plan and execute multi-step workflows where the output of tool three determines which tool to call at step four.

That is a planning problem, not just a function-calling problem. Planning remains one place where current models often struggle. The true cost of running agents in production is not just the API calls. It is the engineering around reliability, monitoring, and failure recovery that makes tool-use work at scale.

If you're building your first agent, start with two or three tools, validate every call, and add reasoning checkpoints. The models are good enough to use tools in bounded settings. They are not reliable enough to be trusted with tools unsupervised.

For more on the types of agents that use these patterns, from simple ReAct loops to full multi-agent orchestrations, see our agent taxonomy.

Related: How Agent Memory Got an Architecture

Related Swarm Signal Coverage:

Sources: