# Function Calling and Tool Use

LLM tool use as typed RPC with schema validation, dispatch, retries, and result handling.

- Author: Jatin Bansal (https://jatinbansal.com/)
- Published: 2026-05-18
- Canonical: https://jatinbansal.com/ai-engineering/tool-use/
- Topic: AI Engineering

---

A travel-booking assistant ships on a Tuesday. Forty-eight hours in, a customer asks it to "rebook me on the same flight tomorrow at the same fare." The model's text reply describes, in confident prose, that it has changed the booking. It has not. There is no booking system attached. The model can describe actions but it cannot *take* them; every side effect lives outside the LLM, and bridging that gap is the entire job of function calling. Tool use is how the model stops narrating the world and starts touching it.

## Tool calls

Function calling, tool use, function invocation; different vendors, same idea. You declare a set of typed functions the model is allowed to ask for. The model decides, at decode time, whether to emit a normal text response or a structured request to call one of those functions. Your runtime executes the request, packages the result, and feeds it back into the next turn. The model can then call more tools, or stop and answer.

A tool is three things:

- A name the model uses to address it.
- A description; natural-language prose explaining when to call it and when not to. The description is the API doc for an LLM consumer; treat it that way.
- An input schema; JSON Schema describing the arguments, with required fields, types, enums, and `format` constraints. The model's emitted arguments are validated against this schema.

The model emits a structured tool request, and the application runtime executes the function. Tool calls are produced by the same decoder as text, using behavior learned from tool-use training data.

## Typed RPC

Tool use resembles a [remote procedure call](https://en.wikipedia.org/wiki/Remote_procedure_call) where the LLM is the caller and the application runtime is the server. The model selects a function and supplies typed arguments. The runtime validates and executes the call, serializes the result, and returns control to the model.

Once you frame it this way, the whole surface area follows. Tool schemas are IDL. Tool descriptions are docstrings on the IDL. The [agent loop](/ai-engineering/agent-loop/) is an RPC client driver. Timeouts, retries, idempotency, error propagation, and observability; every problem that the [RPC literature](https://www.cs.princeton.edu/courses/archive/fall03/cs518/papers/rpc.pdf) solved in the 1980s reappears here, with the twist that the caller is non-deterministic.

## Call and result loop

A single tool-using exchange looks like this on Anthropic's API:

1. You send `messages` plus a `tools` array. The tools array is a list of objects with `name`, `description`, and `input_schema`.
2. The model responds with `stop_reason: "tool_use"` and an assistant message whose `content` is a list of blocks; typically a `text` block followed by one or more `tool_use` blocks, each with an `id`, a `name`, and a JSON `input`.
3. You execute each requested tool and append a `user` message whose `content` is a list of `tool_result` blocks. Each `tool_result` carries the matching `tool_use_id`, the result payload, and an optional `is_error: true` flag.
4. You re-call the API with the appended history. The model either calls more tools (back to step 2) or stops with `stop_reason: "end_turn"` and a final text response.

OpenAI's surface is conceptually identical with different field names: `tools[].type: "function"`, `tools[].function.{name, description, parameters}`, the model emits `tool_calls` on the assistant message, and you reply with messages of `role: "tool"` carrying a `tool_call_id`. The shapes are nearly isomorphic ([Anthropic's tool-use guide](https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview), [OpenAI's function-calling guide](https://platform.openai.com/docs/guides/function-calling)). The semantic differences worth memorizing:

- Forcing a call. Anthropic's `tool_choice` is `{type: "auto" | "any" | "tool" | "none"}`. `any` forces *some* tool; `tool` forces a specific one. OpenAI's `tool_choice` accepts `"auto" | "required" | "none"` or `{type: "function", function: {name}}`; `required` is the equivalent of `any`.
- Parallel tool calls. Both providers will emit multiple `tool_use`/`tool_calls` blocks in a single assistant message when the calls are independent. To opt out; useful for tools that mutate state and must be serialized; Anthropic exposes `disable_parallel_tool_use: true` (on the `tool_choice` object), OpenAI exposes top-level `parallel_tool_calls: false`.
- Strict schemas. OpenAI's strict mode (`strict: true` on a tool) and Anthropic's [strict tool use](https://platform.claude.com/docs/en/agents-and-tools/tool-use/strict-tool-use) both schema-validate tool inputs at decode time, the same [FSM-over-vocabulary](/ai-engineering/constrained-decoding/) trick used by [schema-constrained structured output](/ai-engineering/structured-output/). Use it on tools whose inputs you'd rather not re-validate in application code.
- Tool descriptions are not free. Every tool's schema and description is serialized into a hidden system prompt on every call. A library of 50 tools can easily cost 5–10k tokens per turn. This is the wedge that motivates [MCP](https://modelcontextprotocol.io/) and [dynamic tool routing](/ai-engineering/tool-selection-at-scale/); covered in depth in the tool-selection-at-scale article.

## Anthropic tool loop in Python

A two-tool example; one for weather, one for time; using the official [Anthropic SDK](https://github.com/anthropics/anthropic-sdk-python). Install: `pip install anthropic`.

```python
import json
from anthropic import Anthropic

client = Anthropic()

TOOLS = [
    {
        "name": "get_weather",
        "description": (
            "Get the current weather for a given location. Returns temperature "
            "and conditions. Use only when the user asks about weather; do not "
            "use for forecasts more than 24 hours out."
        ),
        "input_schema": {
            "type": "object",
            "properties": {
                "location": {
                    "type": "string",
                    "description": "City and state, e.g. 'San Francisco, CA'",
                },
                "unit": {
                    "type": "string",
                    "enum": ["celsius", "fahrenheit"],
                    "description": "Temperature unit (default: celsius)",
                },
            },
            "required": ["location"],
        },
    },
    {
        "name": "get_current_time",
        "description": "Get the current local time for an IANA timezone like 'America/Los_Angeles'.",
        "input_schema": {
            "type": "object",
            "properties": {"tz": {"type": "string"}},
            "required": ["tz"],
        },
    },
]

def execute_tool(name: str, args: dict) -> str:
    if name == "get_weather":
        return json.dumps({"temp": 18, "unit": args.get("unit", "celsius"), "conditions": "fog"})
    if name == "get_current_time":
        return json.dumps({"tz": args["tz"], "time": "2026-05-18T11:14:00-07:00"})
    return json.dumps({"error": f"unknown tool: {name}"})

def run_loop(user_msg: str, max_steps: int = 8) -> str:
    messages = [{"role": "user", "content": user_msg}]
    for _ in range(max_steps):
        resp = client.messages.create(
            model="claude-opus-4-7",
            max_tokens=1024,
            tools=TOOLS,
            messages=messages,
        )
        messages.append({"role": "assistant", "content": resp.content})

        if resp.stop_reason != "tool_use":
            return "".join(b.text for b in resp.content if b.type == "text")

        results = []
        for block in resp.content:
            if block.type != "tool_use":
                continue
            try:
                output = execute_tool(block.name, block.input)
                results.append({"type": "tool_result", "tool_use_id": block.id, "content": output})
            except Exception as e:
                results.append({
                    "type": "tool_result", "tool_use_id": block.id,
                    "content": f"error: {e}", "is_error": True,
                })
        messages.append({"role": "user", "content": results})

    raise RuntimeError("max_steps exceeded")
```

The entire `resp.content` array is appended as the assistant turn because the API expects the original block list. Tool errors are returned as `is_error: true` results so the model can recover from timeouts, upstream failures, or invalid arguments without terminating the loop.

The step cap is the load-bearing safety net. Without it, a confused model can loop indefinitely, especially on ambiguous tasks where each tool result triggers another retrieval. The [JIT context-engineering article](/ai-engineering/context-engineering-jit-vs-aot/) called out tool-loop drift as JIT's silent failure mode; the step cap is the bluntest version of the no-progress detector recommended there.

## Failure modes

The token tax on tools is invisible until it isn't. Every tool in the `tools` array is serialized into a system prompt on every call, even when the user asks a question that has nothing to do with any tool. With 30+ tools and verbose descriptions, you can burn 8–12k tokens before the user message starts. The mitigations are real but bounded: trim descriptions, consolidate fine-grained tools into action-parameterized ones (per [Anthropic's tool-writing guidance](https://www.anthropic.com/engineering/writing-tools-for-agents)), and turn to [MCP-style dynamic tool routing](/ai-engineering/tool-selection-at-scale/) once N > ~30.

Parallel tool calls assume independence. If the model emits `transfer_funds(A→B)` and `transfer_funds(B→C)` in the same turn, your runtime executes them concurrently; and the model has no notion of ordering between them. For state-touching tools, force serial execution: `disable_parallel_tool_use: true` on Anthropic, `parallel_tool_calls: false` on OpenAI. The default is "concurrent unless you opt out," which is the wrong default for most production CRUD tools.

Tool selection collapses past ~30 tools. Once the tool count grows large, accuracy on tool selection drops faster than you'd expect; the model has trouble disambiguating between similar tools, and descriptions start interfering with each other. The standard fix is two-stage retrieval: embed the tool descriptions, retrieve a top-k subset relevant to the user's turn, and only pass those into the API. Anthropic ships a built-in [tool search tool](https://platform.claude.com/docs/en/agents-and-tools/tool-use/tool-search-tool) for this; the [tool-selection-at-scale article](/ai-engineering/tool-selection-at-scale/) covers the pattern in depth.

Hallucinated arguments. The model occasionally invents tool inputs that aren't in the schema (`get_weather(zipcode: "94103")` when the schema only accepts `location`). Strict mode catches this at decode time. Without strict, validate every argument before execution and return a clear error in `tool_result`; the model is good at correcting on a single round-trip given a precise error message; don't crash the loop with a Python `KeyError`.

`tool_choice: "any"` prefills the assistant. When you force a tool call, the API prefills the assistant turn with the start of a `tool_use` block. The model cannot precede it with a natural-language explanation or reasoning, and forcing `any` together with extended thinking is currently rejected on Claude. If you want the model to "think first then call," use `auto` with explicit instructions, or call without forcing and check `stop_reason`.

Idempotency at the runtime, not at the model. If a `tool_result` doesn't arrive cleanly, the easy thing is to retry; and the model will happily call the same tool again on the next turn if the conversation is replayed. Any tool that mutates state needs an idempotency key at the runtime layer; specifically, at [the harness](/ai-engineering/agent-harness-anatomy/), not in each tool author's code. Treat the LLM as an at-least-once caller, not an exactly-once one.

Schema strictness fights schema expressiveness. Strict mode requires every property to be `required` (use `union with null` for optional fields) and forbids `additionalProperties: true`. Existing schemas with `oneOf`, recursive references, or unbounded objects may not compile. The same subset that bit you in the [structured-output article](/ai-engineering/structured-output/) bites again here; the production answer is usually to define the tool input as the smallest legal envelope and validate the richer constraints in your `execute` function.

Tool descriptions are prompt code, version them. Edits to a tool's description change the model's call rate the same way edits to a prompt change behavior. They belong in version control and behind eval gates; treat description churn as production-affecting change, the same way you'd treat a prompt edit.

## Further reading

- [Anthropic; Writing tools for agents](https://www.anthropic.com/engineering/writing-tools-for-agents); the canonical engineering writeup on tool design: descriptions, consolidation, response shaping, and the role of evals. Read this before you ship a tool surface.
- [OpenAI; Function calling guide](https://platform.openai.com/docs/guides/function-calling); the reference for the OpenAI surface, including strict schemas, the `tool_call_id` round-trip, and the Responses API.
- [Anthropic; Tool use overview](https://platform.claude.com/docs/en/agents-and-tools/tool-use/overview); the deepest reference for `tool_choice`, parallel tool calls, server tools, and the new programmatic tool calling extension.
- [Simon Willison; entries tagged "llm-tool-use"](https://simonwillison.net/tags/llm-tool-use/); a running log of tool-use launches, regressions, and worked examples across the major providers. Read his posts whenever a new provider tool-use feature drops.
