Tool Calling from First Principles

In this illustrative scenario, an engineer spent two days trying to make Reference Agent stop mis-calling one tool. The tool took a date range and tenant. The agent passed 2026-01, January 2026, or a range where the tool required one date. The engineer added examples, an ISO 8601 instruction and a description note. After two days, the calls improved from awful to unreliable.

The fix, when they finally found it, took four minutes. The tool’s error return had been ValueError: time data 'January 2026' does not match format '%Y-%m-%d'. They changed it to: Invalid 'start' parameter: expected an ISO date like 2026-01-15, got 'January 2026'. For a whole month, pass start=2026-01-01 and end=2026-01-31. The mis-calls stopped — not because the model got better at guessing, but because the first mistake became self-correcting.

13.1Anatomy of a tool call#

Strip away SDK differences and a tool call is a four-step round trip.

You expose definitions. A conventional function-calling request includes the visible tools’ schemas; deferred discovery and server-managed state can change the wire format. Account for definitions actually presented to the model, including cached portions.

The model emits a structured request. Not prose that you parse — a typed object with a call id, a name, and arguments. Modern models are trained on this format, which is why the elaborate parsing machinery of 2023 frameworks is now dead weight.

You execute and return a result. Preserve the provider-issued call id and its result envelope. Many examples use text, but providers and protocols can carry structured or multimodal content; the adapter must retain supported result types.

The model sees the result and decides. Continue, call something else, or answer.

@dataclass(frozen=True)
class ToolCall:
    name: str
    arguments: dict[str, Any]
    id: str = field(default_factory=lambda: new_id("call"))

    def signature(self) -> str:
        """Argument-order-independent signature used by the teaching harness.
        It is not a business operation id or semantic equivalence test."""
        canonical = json.dumps(self.arguments, sort_keys=True, default=str)
        return f"{self.name}({canonical})"


@dataclass
class ToolResult:
    call_id: str
    name: str
    content: str
    is_error: bool = False
    artifacts: list[Artifact] = field(default_factory=list)
    latency_ms: float = 0.0

Two details in those types are load-bearing beyond this chapter. The canonical signature recognises equal name-and-argument values despite dictionary key order. It does not equate aliases, units or defaults, and two intentional credits can have identical arguments. Production mutations need a durable operation id that distinguishes intent from retries; status reads must fetch current state. And artifacts exist so that a tool producing a 4 MB PNG or a 200,000-row CSV returns a reference plus a summary rather than putting the thing in the context window. A tool that can only return a string will eventually return a string that destroys your context budget.

is_error is a field on a result so the model can observe and respond to failures.

Provider contracts still matter#

OpenAI Responses documentation, checked on 9 November 2026, pairs function_call and function_call_output items through call_id. The item’s own id is a different identifier. Preserve required response items, including reasoning items, when manually continuing the conversation. Strict mode constrains supported JSON Schema shapes. It does not prove business validity, authorisation or execution success. Validate parsed arguments in the application before invoking a tool.

The reference schema builder supports a limited annotation subset. It is not a complete runtime validator.¹

13.2Errors are the highest-leverage surface#

An agent’s loop is: act, observe, decide. An error is an observation. Which means the error text is a prompt — one that arrives at precisely the moment the model is deciding what to do next, when it has maximal attention and maximal need. It is the highest-leverage text in your system, and it is usually a stack trace.

A quality ladder, with what each rung actually produces:

Write actionable errors with four properties:

Name the parameter. Invalid 'start' beats invalid arguments, because the model does not have to guess which of six fields was wrong.

Show safe parts of what was received. Redact credentials and sensitive fields, cap length, and label attacker-controlled values as data. Echoing an ordinary date lets the model see its mistake.

Give an example, not a format string. like 2026-01-15 beats %Y-%m-%d. The example is unambiguous; the format string requires the model to know a Python convention.

Suggest the correct call. The step from rung 3 to rung 4. It generalises the fix rather than merely permitting it.

The reference implementation does the mechanical part of this automatically, as a small teaching example. The following is an excerpt from its exception handler, not a standalone Python program. It also catches a TypeError raised inside the function, so production code must distinguish signature binding failures from implementation faults:

except TypeError as exc:
    # Teaching fallback: TypeError can also originate inside the tool.
    # Production adapters should validate/bind arguments before invocation.
    result = ToolResult.error(
        call,
        f"Invalid arguments for {self.name}: {exc}. "
        f"Required: {sorted(self.parameters.get('required', []))}.")

And there is a category of error that is not the model’s fault but is still worth a good message: the denial. Chapter 4’s loop returns Denied by policy (net.host): host 'evil.example.com' is not on the egress allowlist. Do not retry this call; find another route or ask the user. The “do not retry” clause is doing real work — without it, the model retries a permanently-denied call until the loop detector stops the run, which converts a clean refusal into a failed task.

13.3Schema design: the interface the model reads#

A tool’s schema is an API contract for a caller that cannot read your source, cannot experiment, and gets no autocomplete. Design accordingly.

Derive the schema from the implementation. Hand-written schemas drift from the code they describe, and the drift is invisible: the description promises a parameter the function ignores, and the failure is a silently wrong result rather than an error. Generating from a supported type system reduces drift, but bounds, enums and descriptions still need validation.

def decorate(fn: Callable[..., Any]) -> Tool:
    sig = inspect.signature(fn)
    # Resolve annotations properly: under `from __future__ import annotations`
    # every annotation arrives as a *string*, and a schema built from strings
    # silently types every parameter as "string". This is the bug that makes
    # a model pass "12.50" where a number was meant.
    try:
        hints = typing.get_type_hints(fn)
    except Exception:
        hints = {}
    ...

With postponed annotation evaluation, inspect.signature returns annotations as strings, and a naive schema builder types every parameter as string. Python itself does not enforce or coerce annotations; a tool may happen to accept the wrong type, explicitly convert it, or fail. The generated schema was nevertheless telling the model that an amount was text.² Generate your schemas, and then test the generated schema, not just the function.

Prefer flat and typed over nested and free-form. Three named parameters with types beat one params: dict. A dict parameter is an untyped hole: the model cannot see what belongs in it, the schema cannot constrain it, and errors surface at runtime rather than at validation.

Use enums wherever the domain is closed. status: Literal["open", "closed", "pending"] eliminates a whole class of mistake at zero cost.

Set additionalProperties: false. Otherwise a hallucinated extra field passes validation and is silently dropped, which means the model believes it asked for something it did not get.

Write per-argument descriptions. They guide the model in filling each field.

@tool(mutates=True, reversible=False, keywords=("credit", "refund", "billing"))
def issue_credit(account: str, amount: float, reason_code: str) -> str:
    """Issue an account credit. Irreversible once posted.

    Args:
        account: Account id in the form 'acme-9', from the tenant record.
            Not the display name.
        amount: Credit in USD, positive, maximum 10000.00. Use the exact
            disputed amount, not a rounded figure.
        reason_code: One of BILLING_ERROR, SERVICE_CREDIT, GOODWILL. Use
            GOODWILL only when no service failure occurred.
    """

Every clause there prevents an observed failure: passing a display name, rounding, and reaching for GOODWILL as a default. Argument descriptions are where domain knowledge belongs, because they are read exactly when the model is filling that field — far more effective than the same guidance buried in a system prompt.

13.4Naming, and the model’s prior#

Tool names are read more often than any other identifier in your system, and the model brings a prior about what a name means.

Verb-noun, consistently. read_file, list_tickets, issue_credit. Consistency lets the model generalise: having seen read_file and write_file, it will guess delete_file correctly.

Distinguish by function, not by implementation. search_docs and search_tickets are good. elastic_query and pg_query are bad — they name your infrastructure, which the model neither knows nor cares about.

Never let two tools sound like they overlap. get_user and fetch_user_details will be confused, consistently, and the confusion will look like a model problem. Merge them or rename until the distinction is obvious from the name alone.

Beware misleading priors. A tool named search that only does exact-match lookup will be called with natural-language queries and return nothing. Either implement what the name promises or rename it to lookup_by_id.

And a rule about description structure that earns its keep as the tool set grows: say what the tool is not for. Chapter 12’s tool contracts did this deliberately. The “not for” clause prevents the failure where the agent uses the graph tool for a prose question, gets nothing, and reports that the information does not exist.

13.5The token cost of a tool#

Visible tool definitions contribute input tokens. With a flat menu, the same definitions can recur on every turn. Discovery changes which definitions enter context.

Rough arithmetic: a well-described tool with four parameters is 150-400 tokens of schema. Thirty such tools is 4,500-12,000 tokens. On a forty-turn run that is up to 480,000 input tokens spent on telling the model what it could do, most of which it will not do.

Definitions belong in the cacheable prefix. They are stable, so they should sit early and unchanging (Chapter 8). The discount depends on the model, cache policy, prefix eligibility and actual hits; calculate cached and uncached input separately (Chapter 29).

Verbosity has a price, and it is worth paying. Compare description cost with the errors and repair turns it prevents. Remove redundant wording while preserving examples, constraints and distinctions between tools.

Beyond a couple of dozen tools, the problem changes shape. Selection accuracy degrades as the menu grows, and at some point the right move is to stop showing all of them. That is Chapter 14.

Measure the tokens used by tool definitions:

def schema_tokens(self, names: Iterable[str] | None = None) -> int:
    return estimate_tokens(json.dumps(self.schemas(names)))

13.6Measure the repair loop, not just the first mistake#

A tool mis-call is not one event. It is the start of a short trajectory: the model proposes an invalid call, the harness returns an observation, and the model either repairs, repeats, changes direction or gives up. The first-attempt error rate tells you whether the interface is easy to use; the repair trajectory tells you whether the interface teaches. You need both numbers.

First-attempt validation-failure rate is rejected first attempts divided by all first attempts, grouped by tool name, schema version and error fingerprint. An overall rate hides one bad interface inside forty good ones. A fingerprint should preserve the failure category and parameter but remove the supplied value: invalid:start:date_format, not the customer’s date. This gives you a stable aggregate without copying tenant data into a metric label. Track validation failures on repair attempts separately; mixing retries into this denominator makes a weak error message inflate the apparent difficulty of the original interface.

First-repair rate is invalid calls followed by a valid call to the same tool within the next model turn, divided by invalid calls. This is the direct measure of the error message. Keep the call id that produced the error and link the next proposal to it in the trace; adjacency alone becomes ambiguous when the model requested a batch.

Eventual-repair rate uses a wider but still bounded window — two model turns works for most single-parameter mistakes. A successful tool call twenty turns later is not evidence that the error taught anything. It may be a new attempt reached by an unrelated plan.

Unchanged-repeat rate counts an identical canonical signature immediately after an error. This is the clearest signal of an uninformative observation. The model received text and found no actionable difference to make. Chapter 20’s loop detector eventually contains the damage, but containment is not repair.

Do not report repair rate without the terminal outcomes. An agent can improve its apparent rate by abandoning difficult tasks sooner. For each invalid call, record whether the run eventually completed, escalated, was blocked, or exhausted a budget. The useful unit is a small funnel, not a single percentage.

Consider an illustrative week in which Reference Agent made 60,000 first-attempt tool calls across 10,000 runs. First-attempt validation rejected 3.8%, or exactly 2,280 calls. The existing raw exception produced this distribution:

The arithmetic separates two costs that teams routinely conflate. The tool rejected 2,280 first attempts, but those rejections created 3,648 additional model turns: 912 × 1 + 684 × 2 + 342 × 1 + 342 × 3. At an illustrative 6,500 input tokens and 350 output tokens per repair turn, that is 23,712,000 extra input tokens and 1,276,800 extra output tokens. At an illustrative 1.14 seconds for one model-and-validation round trip, it is 4,158.72 seconds — just over 69 minutes of aggregate agent latency. None of those figures includes the operator time spent on the 684 escalated or exhausted calls.

The team rewrites the dominant date and identifier errors to rung 4, without changing the model or the system prompt. In the following week, the first-attempt validation-failure rate stays at 3.8%. That is expected: better feedback arrives after the first mistake. But first-repair rises from 40% to 85%; second-turn repair, escalation and exhaustion each fall to 5%. The same 2,280 invalid calls now create 1,938 × 1 + 114 × 2 + 114 × 1 + 114 × 3 = 2,622 extra turns. The interface saves 1,026 turns, 6,669,000 input tokens, 359,100 output tokens and 1,169.64 seconds of aggregate latency. More importantly, eventual repair rises from 70% to 90%.

These are illustrative figures, not a promise that every error rewrite produces that lift. Their point is the accounting method. A prompt experiment measured only on first-attempt validity would call the change a failure. A task-level success metric might find an improvement but could not attribute it. The linked repair funnel shows exactly which text changed the next action.

Instrument the content without logging the content. Store the tool name, schema version, error fingerprint, retryability class, previous and next canonical signatures, turn distance, latency, token usage and terminal outcome. Keep raw arguments in the access-controlled trace only when your retention policy permits them. A metric label containing account ids or dates is both a privacy leak and an observability disaster because its cardinality grows without bound (Chapter 23).

Read disagreement between the rates#

The metrics become useful when they disagree. A high first-attempt validity rate with a low task-completion rate means the tools accept calls but do not help achieve the outcome. Look for vague empty results, false success, missing continuation identifiers or a tool menu that lacks the next required action. Schema work will not fix this class because validation already passed.

A low first-attempt validity rate with a high first-repair rate means the interface is teachable but awkward. The error layer is doing its job, yet the schema is forcing a paid correction on too many runs. Move the concrete example from the error into the argument description, tighten an enum, rename an ambiguous tool, or split one overloaded parameter. Keep the good error: removing it after improving the schema throws away the recovery path for the mistakes that remain.

A low first-repair rate with a high eventual-repair rate means the model is wandering into the answer. Read the intermediate calls. You will usually find an error that identifies the problem without proposing a valid alternative, or a result that makes the model fetch information it already had. Rewrite for the next action and include the identifier that action requires.

A high unchanged-repeat rate is more urgent than an ordinary second-attempt failure. A changed-but-still-invalid call at least shows that the observation carried information. An unchanged signature shows that the observation failed to alter the proposal at all. Check first for bare transport errors, raw exceptions and instructions that contradict the schema. Then verify that the provider actually delivered the tool result under the correct call id; a perfect message attached to the wrong call is functionally absent.

Compare versions as cohorts rather than before-and-after calendar totals. Tool mix, tenant mix and task difficulty change from week to week. Record the schema and error-template version on every attempt, then compare runs using the same tool and error fingerprint. If you roll out rung-4 errors to 20% of traffic, the relevant comparison is repaired calls under the old and new versions during the same interval, not this Tuesday against last Tuesday.

Watch the distribution, not only the mean. A repair change may save one turn on thousands of simple date errors while making a small number of irreversible tools worse. Break the funnel down by risk class and weight terminal outcomes accordingly. One exhausted read-only search is inconvenient; one mis-repaired account mutation may require incident response. Chapter 24’s evaluation set should include both, but the production dashboard must not average them into the same green number.

Finally, inspect samples. Metrics tell you where the contract is weak; they do not prove why. Review a bounded set of traces for each dominant fingerprint, with arguments redacted unless needed, and classify the actual mechanism before rewriting. A spike in invalid:start:date_format may be a worse model instruction, a schema deployment that lost examples, or upstream user text copied verbatim into the call. The same counter can represent three different fixes.

13.7Parallel calls, and the hazard inside them#

Models can request several tool calls in one turn, and executing them concurrently is a large latency win — four file reads in the time of one.

The hazard is that concurrency is only safe when the calls cannot observe each other’s effects. Two parallel writes to the same file is a race; a read parallel with a write to the same resource is a race; two independent reads are not. The harness cannot infer which is which from a tool’s name, so it must be declared:

reads  = [(c, t) for c, t in approved if not t.mutates]
writes = [(c, t) for c, t in approved if t.mutates]
# reads may run concurrently; writes run serially, in the order requested

Vetting must complete for the whole batch before execution begins. In this harness, a later HALT or vetting exception prevents every call from starting. A per-item DENY or BLOCK excludes that item while other approved calls may run. This is pre-execution vetting, not an atomic business transaction.

Propagate trace context explicitly. Worker threads do not inherit the caller’s context variables, so naive parallel execution creates unparented tool spans:

for c, t in reads:
    snapshot = contextvars.copy_context()   # capture in the SUBMITTING thread
    pending.append((c, t, pool.submit(snapshot.run, self._invoke_one, c, t)))

Without parent-child links, the trace cannot establish which model turn caused the parallel calls.

13.8Result formatting, and the truncation that saves your context#

The result is text a model will read. Format it for that reader.

Structure beats prose for structured data. Rows, keys, ids. A model reading a table of six ticket ids with statuses can act on any of them; a paragraph describing them requires re-parsing.

Include the identifiers needed for the next call. If the natural next action is read_ticket(id), the search result must contain ids. Results that describe things without naming them force an extra round trip.

Empty is not nothing. [] teaches the model nothing. No tickets match tenant=acme status=open in the last 30 days. Widen the window with 'since', or try status='pending'. teaches it what to do next. Empty results are as much a prompt-writing opportunity as errors.

Truncate at the tool boundary, with a way forward:

[truncated: showing lines 1-200 of 38,412]
Use grep_file(path, pattern) to search, or read_file(path, offset=200, limit=200)
to continue from here.

The failure this prevents is the one from Chapter 8: a tool returns 40,000 lines, blows the entire budget in one turn, and evicts everything the agent needed. Truncating without a way forward converts that into a different failure — the agent cannot get the data and gives up. Both parts are needed.

Return references for non-text products. A generated chart, a 200 MB export, a PDF: return an artifact id, a media type, and a one-line summary. The model sees [artifact art_9fc2 image/png "bar chart of settlement failures by day, peak on the 14th"] and can reason about it, cite it, or hand it to the user, without 4 MB entering the window.

13.9A successful result can still be false#

Good error handling covers calls that report failure. It does nothing when a tool reports success too early. A result is only as truthful as the state transition it names. accepted, queued, started, completed and verified are different facts, even when an upstream API returns all of them with a 200 status.

An illustrative Reference Agent incident makes the distinction concrete. A customer asked for a compliance export covering 48,000 conversations. Reference Agent called create_export, the upstream service accepted the job and returned {"status":"ok","job_id":"exp_731"}, and the wrapper returned Export created successfully. The model told the customer the export was ready. Eight minutes later the asynchronous job failed because the tenant’s export quota was full. No artifact existed, yet the transcript contained an apparently successful tool result and a confident final answer.

The team initially blamed the model for overclaiming. The model had actually repeated the strongest claim the tool gave it. The wrapper had collapsed request acceptance into work completion, removed the job id needed for a follow-up call, and supplied no completion condition. No prompt can reliably recover information that the tool result erased.

The repair is to make state and evidence explicit. create_export returns state=accepted, job_id=exp_731, and next_action=get_export_status(job_id='exp_731'). get_export_status returns one of a closed set: running, completed with an artifact reference, or failed with a repair instruction. The agent may say the export was requested after the first call. It may say the export is ready only after a result contains the artifact.

This design moves truth out of prose and into a state machine the harness can test. The result still contains readable guidance for the model, but the state value is constrained and the evidence is machine-checkable. A trajectory evaluator can assert that every accepted export is followed by a status check, and an outcome evaluator can assert that completed includes an artifact id (Chapter 24).

The same distinction applies outside asynchronous jobs. A database driver saying rowcount=1 has driver-specific affected/matched-row semantics; it does not prove that a downstream cache was refreshed. A message API returning an id proves that it accepted a message, not that a recipient received it. A deployment endpoint returning a release id proves that a rollout was created, not that health checks passed. Tool results should name the narrowest fact the underlying system has established.

There are two legitimate contract shapes. Split the lifecycle into tools when the next state may take seconds or minutes and the model can do useful work between checks: start_export then get_export_status. Keep the lifecycle inside one tool when the wait is short and bounded: the implementation polls, applies its own timeout, and returns only completed or an actionable timeout result. What you must not do is wait indefinitely inside a tool or return success at the first acknowledgement.

Timeouts need the same precision as errors. Export timed out leaves the model guessing whether another call duplicates work. A truthful result says: Export exp_731 is still running after 30 seconds. Do not create another export. Call get_export_status(job_id='exp_731') after 20 seconds. The stable job id makes the continuation safe; the retry instruction prevents duplicate side effects.

The reference Reference Agent implementation provides only some of the primitives this contract needs: an in-memory ToolResult can carry text and artifacts, ToolCall.signature() gives argument-order-independent identity, and the loop stores only the successful result content string under an idempotency key. It does not durably persist artifact objects or structured evidence metadata. Nor does it implement an asynchronous job-state protocol or enforce evidence requirements for arbitrary domain tools. Durable artifacts, evidence metadata and lifecycle enforcement are production extensions you add around the tool implementation and test at its boundary. Do not describe those proposed conventions as guarantees the small reference harness currently provides.

13.10Structured output is not a tool call#

Tool calls are for actions with effects outside the model — reading, writing, computing, fetching. The result comes back and the loop continues.

Structured output constrains the model’s own answer to a schema. No execution, no round trip, no second call.

The mistake is using a tool call to get structured data out of a model: defining an extract_fields tool that the harness receives and merely unpacks. If the application unnecessarily sends an acknowledgement and requests another answer, it adds a turn. It can instead consume the call arguments directly; a native structured-output mode usually expresses that extraction intent more clearly. If the “tool” has no implementation, you wanted structured output.

The converse mistake is rarer and worse: using structured output where an action was needed, so the model describes issuing a credit and nothing issues it. The agent reports success, the customer is not credited, and nothing in the system knows. Chapter 24’s trajectory evaluations catch this class of failure — an outcome eval sees a plausible answer and passes.

13.11Testing tools#

Tools are ordinary code and should be tested like it, plus three tests specific to being called by a model.

Test the generated schema, not just the function. Assert types, required fields, enums and descriptions. This is what catches the annotation bug above.

def test_schema_is_derived_from_signature_and_docstring(toolset):
    schema = toolset.get("charge_card").schema()
    props = schema["input_schema"]["properties"]
    assert props["amount"]["type"] == "number"
    assert props["amount"]["description"] == "Amount in USD."
    assert sorted(schema["input_schema"]["required"]) == ["account", "amount"]
    assert "mutates" not in str(schema)   # risk metadata must never leak

Test that errors are actionable. Not that an error occurs — that its text names the parameter and the expectation:

def test_wrong_arguments_produce_a_repair_instruction(toolset):
    result = toolset.get("charge_card").invoke(ToolCall("charge_card", {"amount": 5.0}))
    assert result.is_error
    assert "Invalid arguments" in result.content
    assert "account" in result.content     # names what is missing

Test the mis-call cases you have actually seen. Every production mis-call becomes a test with the exact bad arguments, asserting the error would have taught the model the fix. This is where a tool suite accumulates real value: after a year, it encodes every way a model has ever got this interface wrong.

13.12When a tool contract cannot remove the ambiguity#

Schema design works when the caller’s intent can be represented as typed arguments and the action has a testable success condition. Some requests fail that premise. Resolve this customer fairly, clean up the old records, and send the right people an update contain policy choices, scope choices and social judgement. Turning each phrase into an enum makes the schema look precise while leaving the decision undefined.

Do not answer ambiguity by adding one giant options: object field. That moves the uncertainty into an untyped pocket and gives the model more ways to sound certain. Instead, make the missing decision visible before any effect occurs. A discovery tool can return the candidate records and their ids. A planning step can propose the recipients and the message. A human approval can settle a judgement the organisation has not encoded (Chapter 19). Only then should a narrow mutation tool accept the resolved identifiers.

Nor should every deterministic transformation become a tool. Simple formatting may need no tool. Use deterministic computation for exact arithmetic, money, dates or larger sorts when a small error would matter; code-mode batching can avoid a model round trip for each calculation. A tool adds schema tokens, another failure surface, trace events and often latency. Use one when the operation needs data or capability outside the model, must enforce a deterministic boundary, or produces an effect the harness must govern.

The opposite limit matters too: a perfectly designed tool cannot compensate for a missing policy engine. A model can call a beautifully named, fully typed delete_customer_record with valid arguments and still be unauthorised. Schema validation proves shape, not permission; an actionable error teaches repair, not entitlement; an idempotency key prevents duplicate effects, not forbidden ones. Chapter 15 owns that decision.

The practical test is to ask three questions before exposing an operation. Can you state the input without hiding judgement in free-form text? Can the implementation return evidence for the exact outcome it claims? Can policy decide whether this caller may cause that outcome? If any answer is no, improve the surrounding workflow before you improve the schema.