LLM Tool Calling

How a language model “uses tools” — explained from zero, for a developer who has never built against an LLM API. By the end you should understand the whole loop well enough to write the ~60-line handler at the bottom yourself.

1. The one idea everything else hangs on

The model never runs your code. An LLM is a function that takes text in and produces text out — it has no network access, no filesystem, no Python interpreter. When people say “the model called a tool,” what actually happened is:

Your application told the model, in the request, “these tools exist.”
The model wrote a request to use one — a small structured message, not executed code.
Your application read that message, ran the real function on your machine, and sent the result back to the model as more input text.
The model wrote its final answer using that result.

The model is a very good decision-maker about which function to call with which arguments. Your application is, and always remains, the thing that executes. Keep that division of labor in your head and none of what follows is mysterious.

2. Why tool calling exists

A model’s knowledge is frozen at training time and it can’t affect the world. Three gaps follow, and tools close all three:

Fresh data: “What’s the weather in London right now?” — needs an API call at answer time.
Private data: “How many units of SKU 4471 are in stock?” — needs your database, which the model has never seen.
Side effects: “Create the ticket” / “send the quote” — needs permission to do something, not just say something.

Without tools, the model can only produce plausible text about these things. With tools, it can be correct about them — because the answer came from your systems, and the model’s job shrank to (a) deciding a tool was needed and (b) phrasing the result.

3. The three pieces you write

3.1 The tool — an ordinary function

Nothing special about it. It’s code you could call yourself:

def get_weather(city: str) -> str:
    """Return the current weather for a city."""
    # Real version would call a weather API; mock data keeps the example self-contained.
    mock = {"New York": "Sunny, 75F", "London": "Rainy, 15C", "Tokyo": "Cloudy, 22C"}
    return mock.get(city, "No data for that city.")

3.2 The tool definition — what the model actually sees

The model never sees the function body. It sees a description in JSON Schema that you include with every API request:

{
  "name": "get_weather",
  "description": "Get the current weather in a given city.",
  "input_schema": {
    "type": "object",
    "properties": {
      "city": {"type": "string", "description": "City name, e.g. London"}
    },
    "required": ["city"]
  }
}

Read that the way the model does: a name, a sentence explaining when this tool is useful, and the arguments with their types. This is the entire interface — which leads to the single most practical rule in this note:

The description is a prompt. The model decides whether to call your tool only from the name, description, and parameter descriptions. A vague description (“does stuff with data”) means the tool never gets called, or gets called wrong. Write it like documentation for a distractible colleague: what it does, when to use it, what the arguments mean.

3.3 The handler — the loop your application runs

The handler is the orchestration code between the user and the model. In plain words it says: send the conversation to the model; if the model replied “I want to call tool X with arguments Y,” run X(Y), append the result to the conversation, and go around again; if the model replied with ordinary text, that’s the final answer.

4. The loop, traced by hand

User asks: “What’s the weather in London?”

Step	Who	What happens
1	App → Model	Sends the question plus the tool definitions
2	Model → App	Replies with a tool-use request: `get_weather(city="London")` — this is data, not execution
3	App	Looks up `get_weather` in its own code, runs it, gets `"Rainy, 15C"`
4	App → Model	Sends the conversation again, now including the tool result
5	Model → App	Replies with text: “It’s currently rainy and about 15°C in London.”
6	App → User	Shows the text

Two things beginners consistently miss are visible right in the table:

Every tool call costs an extra round trip to the API (steps 1–2 and 4–5 are two separate requests). An answer needing three tools in sequence is four model calls. This is why agent latency adds up.
The model is stateless between trips. Step 4 re-sends the whole conversation — question, the model’s tool request, and the result. The API doesn’t remember step 1; your message list is the memory.

5. A complete handler

This is the OpenAI-style pattern (Anthropic’s differs in field names, not in shape). It handles multiple rounds — a while loop, because the model may want tool B after seeing tool A’s result:

import json
from openai import OpenAI

# --- 1. The tools: ordinary functions ---------------------------------
def get_weather(city: str) -> str:
    mock = {"New York": "Sunny, 75F", "London": "Rainy, 15C", "Tokyo": "Cloudy, 22C"}
    return mock.get(city, "No data for that city.")

def get_stock_price(ticker: str) -> str:
    mock = {"AAPL": "$180", "GOOGL": "$140", "NVDA": "$1,150"}
    return mock.get(ticker, "Unknown ticker.")

# --- 2. The tool map: string name -> function object ------------------
# The model sends back only a NAME. This dict is how the handler finds
# the actual code to run.
available_functions = {
    "get_weather": get_weather,
    "get_stock_price": get_stock_price,
}

# --- 3. The definitions: what the model sees --------------------------
tools = [
    {"type": "function", "function": {
        "name": "get_weather",
        "description": "Get the current weather in a given city.",
        "parameters": {"type": "object",
                       "properties": {"city": {"type": "string"}},
                       "required": ["city"]}}},
    {"type": "function", "function": {
        "name": "get_stock_price",
        "description": "Get the latest stock price for a ticker symbol.",
        "parameters": {"type": "object",
                       "properties": {"ticker": {"type": "string"}},
                       "required": ["ticker"]}}},
]

client = OpenAI()  # reads OPENAI_API_KEY from the environment

# --- 4. The handler loop ----------------------------------------------
def run_conversation(user_prompt: str) -> str:
    messages = [{"role": "user", "content": user_prompt}]

    while True:
        response = client.chat.completions.create(
            model="gpt-4o", messages=messages, tools=tools)
        msg = response.choices[0].message

        if not msg.tool_calls:            # ordinary text -> we're done
            return msg.content

        messages.append(msg)              # keep the model's request in history

        for call in msg.tool_calls:       # the model may request several at once
            fn = available_functions[call.function.name]
            args = json.loads(call.function.arguments)   # arrives as a JSON *string*
            try:
                result = fn(**args)
            except Exception as e:        # tool failures go BACK to the model
                result = f"Tool error: {e}"
            messages.append({
                "role": "tool",
                "tool_call_id": call.id,  # ties this result to that request
                "name": call.function.name,
                "content": str(result),
            })
        # loop: the model sees the results and either answers or asks again

print(run_conversation("Weather in London, and NVDA's stock price?"))

Five details in that code carry most of the real-world weight:

The tool map. The model returns the string "get_weather"; the dict maps it to the function object. Never eval() a name the model sent — look it up in an explicit allowlist, exactly like a shell validating a command against a fixed menu instead of eval-ing input.
json.loads on the arguments. They arrive as a JSON string, not a dict. Parse, then splat with **args.
tool_call_id. When the model requests three tools at once, the id is how each result is matched to its request. Omit it and the API rejects the message.
Errors go back to the model. A failed tool shouldn’t crash the handler — return the error text as the tool result and let the model decide (retry differently, apologize, use another tool). The model can only handle what it can see.
The loop, not an if. One if tool_calls: handles a single round. Real questions chain: look up the order → then check the shipping status. The while makes it an agent; this loop is the “agent loop” people name-drop — see Building-an-AI-Agent-from-Scratch.

6. Native tool calling vs. ReAct (you will meet both)

What §5 shows is native tool calling: the model was trained to emit a dedicated, structured tool-request message, and the API guarantees its shape. There is an older technique still everywhere in frameworks: ReAct, where the “protocol” is just text the model is prompted to write —

Thought: I need the current weather.
Action: get_weather
Action Input: {"city": "London"}

— and the framework parses it with what amounts to string matching, feeding back an Observation: line. It works with any model, including small local ones with no native tool support — that’s its virtue. Its weakness follows directly: any parsing built on “the model will probably format this correctly” fails when the model doesn’t, which small models regularly do. Prefer native tool calling when the model supports it; treat ReAct as the compatibility path. (NAT-Lesson-0-Getting-Started §2.5 walks a live ReAct agent, including what its parse failures look like.)

7. Gotchas and safety, briefly

The model can hallucinate arguments. It may call get_weather(city="Londn") or invent a ticker. Validate inputs in the tool like you would any untrusted input — schema types constrain shape, not sense.
Tools run with your privileges. A delete_record tool will be called wrong eventually. Give tools the least capability that works; make destructive ones require confirmation outside the model.
Tool results are input, and inputs steer models. If a tool fetches a web page and the page contains “ignore your instructions and…”, the model reads that as part of the conversation (prompt injection). Treat tool output from external sources as untrusted data.
More tools ≠ better. Every definition is context the model must weigh; twenty vague tools produce worse tool choice than five sharp ones.

8. Where this goes next

Write the loop yourself once: Building-an-AI-Agent-from-Scratch.
Frameworks (LangChain, CrewAI, NAT…) generate the schemas from your function signatures and run the handler loop for you — after this note you know exactly what they’re automating. See NAT-Lesson-0-Getting-Started for that in practice.
Model-Context-Protocol-MCP standardizes the other side: how tools are published so any client can discover and call them.