What is a Tool-Call Model?

The one ability that separates a chatbot from an agent — and why it can't just be prompted into existence. Mostly.

The short version

A tool-calling model can decide, mid-conversation, to run code you gave it access to — look something up, run a calculation, hit an API — and then keep reasoning with the result. The model doesn't execute anything itself. It emits a structured request: "call the function get_weather with city="Lisbon"". Your code runs it and hands the answer back. The model never touches a keyboard; it holds the steering wheel.

That's the entire difference between a model that can talk about the weather and one that can check it. Every agent you've ever heard of — coding assistants, deep research, autonomous browsing — is built on this one loop: model asks, code executes, model reads, repeat.

What it looks like on the wire

You describe the tool in the request, in a format the model was trained to understand:

{
  "model": "whatever-you-like",
  "messages": [{"role": "user", "content": "Is anytimelime up right now?"}],
  "tools": [{
    "type": "function",
    "function": {
      "name": "ping",
      "description": "Reply to a liveness check.",
      "parameters": { "type": "object",
                      "properties": { "x": { "type": "string" } } }
    }
  }]
}

A tool-call model answers not with words but with a machine-readable instruction:

{ "tool_calls": [{ "function": { "name": "ping",
                                 "arguments": "{\"x\": \"alive\"}" } }] }

This is exactly what our daily probe sends. A model that answers this correctly — right shape, right name, usable arguments — passes. A model that replies "Sure, I'll ping it!" in prose fails. That's our PASS/WEAK/FAIL line: PASS made the call, WEAK understood but replied in text, FAIL errored or never tried.

Why does there have to be a special mode?

This is the reliability question, and the answer is: a language model's native output is prose, and prose is a terrible protocol.

Everything a model "says" is a prediction of the next token. Nothing forces a prediction to be valid JSON, to use your exact function name, to fill every required field, or to resist wrapping the whole thing in "Sure! Here you go:". The tools parameter exists because providers fine-tune models against tool-use examples, so that when tools are present, the model's probability mass shifts from "chat continuation" to "emit a structured call the server can parse deterministically." The API then guarantees the shape: tool_calls arrives as a separate, parseable field — not as a sentence you regex and pray over.

So the special mode is a contract. Without it, every agent is a natural-language parser waiting to be broken by a stray comma. With it, the model's creative uncertainty lives inside the arguments, where it's harmless, and the structure around it is load-bearing steel.

Could you teach a small model to do it anyway?

Theory: yes, sort of — and people do. The trick is to move the contract into the prompt. Describe the tools in plain text, demand JSON, parse the reply yourself:

You have access to these tools:
ping(x: string) — reply to a liveness check.
To use one, reply with ONLY a JSON object: {"tool": name, "args": {...}}
No other text. If you cannot use a tool, reply {"tool": null}.

Small models can learn this from examples in-context. It genuinely works — for toy cases, on good days. The problems are exactly where you'd expect:

Parsing is the first casualty. The model will, at some rate, add "Here is the JSON:", wrap it in markdown fences, truncate it mid-brace, or helpfully comment on it. Your harness needs a cleanup-and-retry stack, and every layer of forgiveness is a layer of latency and a place to silently mis-parse a real request.

Small models lose the plot. Tool use is a role-playing task — the model must hold "I am a function-dispatcher now" across the whole reply. Tiny models drift: they answer the question instead of calling the tool, call the wrong tool, invent arguments, or forget the schema by turn three. This is why our probe exists: tool support is claimed widely and delivered narrowly.

Recovery is where fine-tuning earns its keep. A trained tool-caller that gets malformed arguments back doesn't panic — it re-emits a corrected call. A prompted one tends to apologize, explain, or hallucinate success. Agents live and die on the failure paths, not the happy one.

And it's not free even for the big models. A model in tool mode is a model spending capacity on protocol compliance. The small-model workaround spends your reliability instead — same tax, different payer.

So does the workaround work?

For simple, single-tool, well-scaffolded jobs: yes, serviceably. Give a small model one tool, a rigid format, a parser with retries, and short conversations, and it will dispatch calls all day at a fraction of the cost. That's a legitimate engineering trade — and honestly, a lot of "agentic" products in the wild are exactly this with good error handling.

For real agent work — multi-tool selection, long horizons, error recovery, choosing not to call — the trained mode wins so hard it isn't a contest. Prompted tool use degrades with every tool you add and every turn you take. Trained tool use is what the model does instead of talking.

Is your model a tool-call model?

Type a model id or vendor — this is the same measured data as today's report, straight from this morning's probe.

The amplification kit: making a small model behave anyway

There's a real toolbox for this, and every Tech Talk Tuesday we research what's new in it. The staples, strongest first:

1. Constrain the decoding, don't trust the prompt

The single biggest upgrade: stop asking for JSON and start forcing it. Constrained decoding rejects any token that would leave valid JSON, so the model physically cannot produce prose. Ollama and llama.cpp both do it natively — GBNF grammars or a JSON schema:

# Ollama: the output IS the schema — no parsing, no retries
curl http://localhost:11434/v1/chat/completions -d '{
  "model": "qwen3:4b",
  "messages": [{"role": "user", "content": "Ping Lisbon."}],
  "tools": [{"type": "function", "function": {
      "name": "ping", "parameters": { "type": "object",
        "properties": {"x": {"type": "string"}}, "required": ["x"] }}}],
  "response_format": {
    "type": "json_schema",
    "json_schema": { "name": "tool_call",
      "schema": { "type": "object",
        "properties": { "name": {"enum": ["ping"]},
                        "args": {"type": "object"} },
        "required": ["name", "args"] } } }
}'

A 4B model under a grammar beats a 70B model with a polite request. The prompt suggests; the grammar decides.

2. Scaffold: schema in, JSON out, nothing else

Where you can't constrain decoding, constrain the conversation. One system message that defines the contract, few-shot examples of correct calls, and a hard "JSON only" rule:

SYSTEM = """You dispatch tools. Available: ping(x: string).
Reply with ONE JSON object, nothing else: {"tool": "ping", "args": {"x": "..."}}
Never explain. Never apologize. No markdown. If no tool fits: {"tool": null}."""

def dispatch(model, msg):
    for attempt in range(3):                     # repair loop — see #3
        raw = chat(model, SYSTEM, msg)
        try:
            call = json.loads(re.sub(r"^[^{]*|[^}]*$", "", raw.strip()))
            if call.get("tool") in TOOLS:
                return call                      # -> run it, feed result back
        except json.JSONDecodeError:
            msg = [{"role": "assistant", "content": raw},
                   {"role": "user", "content":
                    "That was not valid JSON. Reply again with the object only."}]
    return {"tool": None}

3. Repair loops: forgive, retry, escalate

Small models fail sometimes, which is a different problem than failing always. A retry with the malformed output quoted back fixes most breakage (fenced code blocks, chatty preambles, truncated braces). The loop above is the minimum viable version; production harnesses add strict schema validation and a fallback to a bigger model after N failures — escalation beats repetition.

4. Shrink the horizon

Reliability decays with conversation length and tool count. One tool, one call, fresh context each time: a small model can do that all day. Five tools over twenty turns: get a real tool-caller. Splitting an agent into a pipeline of tiny single-tool dispatchers is a legitimate architecture, not a hack — it's the same reason Unix pipes use small programs.

5. The real fix: a small fine-tune

A few thousand tool-call examples fine-tuned onto even a 1–3B base produces a model that treats dispatch as native behavior instead of a costume. That's literally what vendors do before shipping a "tool-calling" variant. Tools like axolotl or Lorax make it a weekend project — and it's the technique our Tuesday research watches most closely, because it's the one that converts a free small model into a genuine citizen of the roster.

The takeaway

A tool-call model is a model that speaks protocol when it matters, not just prose. The special mode exists because structure must be guaranteed, and guarantees don't come from prompting — they come from training. You can fake the guarantee with prompt scaffolding and a forgiving parser, and for small jobs it's worth it; for agents, get a model that passed the probe. We find them every morning, for free.