Building an AI Agent from Scratch

The agent loop, written by hand: a complete working agent in about 80 lines of plain Python — no LangChain, no CrewAI, no SDK. Every other note in this sequence describes machinery that wraps, extends, or standardizes what this note builds, so this is the foundation: once you have written the loop yourself, frameworks stop being magic and become visible trade-offs.

Read LLM Tool Calling first — this note assumes you know the tool schema / handler / loop anatomy it teaches. From here the sequence continues to Model-Context-Protocol-MCP (sharing tools across agents), LangGraph vs CrewAI Research and Frameworks, runtimes, and harnesses (what frameworks add to this loop), and Deep-Agents (what long-horizon harnesses add). The archived article that inspired this note is Building an AI Agent from Scratch No Magic, Just a Deterministic Loop (Sergey Nes) — his companion repo is linked in the references.

1. The line where “chatbot” becomes “agent”

A chatbot is a one-shot function call: prompt in, text out, done. An agent is that same call inside a while loop, where each pass can execute a tool and feed the result back in. That is the entire difference. There is no additional component called “agency” — the loop is the agent.

The closest thing you already know is a shell. A shell’s life is a loop: read a command, fork/exec it, wait, capture the output, prompt again. The agent loop is the same structure with the roles flipped — the model writes the “commands” (tool-call requests), your program is the shell that executes them, and instead of printing output to a human, the loop appends it to the conversation and sends the whole thing back to the model. The model reads what happened and decides the next command, or decides it’s done and answers.

Each pass through the loop is one cycle of: reason (model reads the conversation so far) → act (model requests a tool call) → observe (your code runs it and appends the result) → repeat until the model answers in plain text. The literature calls this pattern ReAct, though as LLM Tool Calling §6 explains, modern APIs do the act step with structured native tool calls rather than parsed prose.

One thing worth being precise about, because the marketing around agents isn’t: the model has no memory and no goals between API calls. The messages list your loop maintains is the agent’s entire mental state — the context window is its RAM, and your list is what’s loaded into it on every call. “The agent remembered the file it read” means “the tool result was still in the list.”

2. The spec

A minimal agent must:

  1. Accept a high-level task from the user.
  2. Decide what to do next by reading the conversation so far.
  3. Execute the tool calls the model requests.
  4. Append every result to the conversation so later decisions can build on it.
  5. Stop when the model answers without requesting tools — or when a turn limit says enough.

Point 5’s second clause is not in most tutorials, and its absence is a bug: a bare while True: around a paid API call is an unbounded spend loop if the model gets stuck retrying a failing tool. Cap it.

3. The whole agent

Self-contained and runnable: three toy tools, their schemas, and the loop. Uses the OpenAI client because its API shape is the lingua franca — §4 shows why that choice costs nothing.

import json, os
from datetime import datetime
from openai import OpenAI

# --- Tools: ordinary functions --------------------------------------
def get_current_date() -> str:
    return datetime.now().strftime("%Y-%m-%d %H:%M:%S")

def calculate(expression: str) -> str:
    # eval with builtins stripped -- fine for a local toy; see the warning below
    try:
        return str(eval(expression, {"__builtins__": {}}, {}))
    except Exception as e:
        return f"Error: {e}"

def get_weather(city: str) -> str:
    return f"Weather in {city}: 72F, partly cloudy"   # stub; a real one calls an API

# --- Dispatch table: the name the model sends -> the callable -------
TOOL_FUNCTIONS = {
    "get_current_date": get_current_date,
    "calculate": calculate,
    "get_weather": get_weather,
}

# --- Schemas: the only thing the model ever sees about the tools ----
TOOLS = [
    {"type": "function", "function": {
        "name": "get_current_date",
        "description": "Returns the current date and time.",
        "parameters": {"type": "object", "properties": {}, "required": []}}},
    {"type": "function", "function": {
        "name": "calculate",
        "description": "Evaluates a math expression and returns the result.",
        "parameters": {"type": "object",
                       "properties": {"expression": {"type": "string",
                                      "description": "A math expression, e.g. '847 * 0.15'"}},
                       "required": ["expression"]}}},
    {"type": "function", "function": {
        "name": "get_weather",
        "description": "Gets the current weather for a city.",
        "parameters": {"type": "object",
                       "properties": {"city": {"type": "string", "description": "City name"}},
                       "required": ["city"]}}},
]

SYSTEM_PROMPT = ("You are a helpful assistant. Use tools when needed. "
                 "When you have a final answer, respond without calling any tools.")

# --- The agent ------------------------------------------------------
def run_agent(task: str, client: OpenAI, model: str, max_turns: int = 20) -> str:
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": task},
    ]

    for _ in range(max_turns):
        response = client.chat.completions.create(model=model, messages=messages, tools=TOOLS)
        msg = response.choices[0].message
        messages.append(msg)

        if not msg.tool_calls:          # no tool requests -> this is the final answer
            return msg.content

        for call in msg.tool_calls:
            name = call.function.name
            args = json.loads(call.function.arguments)
            print(f"  > {name}({args})")
            fn = TOOL_FUNCTIONS.get(name)
            try:
                result = fn(**args) if fn else f"Unknown tool: {name}"
            except Exception as e:      # errors go back to the model, not up the stack
                result = f"Tool error: {e}"
            messages.append({"role": "tool", "tool_call_id": call.id, "content": str(result)})

    return "Stopped: hit the turn limit without a final answer."

if __name__ == "__main__":
    client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
    task = "What's today's date? What is 15% of 847? And what's the weather in Tokyo?"
    print(f"Task: {task}\n")
    print(f"\nAnswer: {run_agent(task, client, model='gpt-4o-mini')}")

The load-bearing pieces, most of which LLM Tool Calling introduced:

  • TOOL_FUNCTIONS is a function-pointer table. The model sends back a string; the dict maps it to code. The .get() lookup doubles as an allowlist — an invented tool name becomes an error message the model can read, not an exception.
  • if not msg.tool_calls: is the exit condition. The model signals “done” by answering in plain text. Everything the agent does hinges on this one branch.
  • The system prompt is the steering wheel. It’s where you tell the model when to use tools and when to stop. Production agents put enormous effort here; the two sentences above are the minimum that works.
  • Errors are data. A failing tool returns its error text into the conversation, and the model decides what to do about it — retry differently, use another tool, or report failure. Crashing the loop would throw away everything learned so far.
  • max_turns is the watchdog timer. The naive while True: version works right up until the model loops on a stuck tool at API prices.

⚠️ About that eval. Stripping __builtins__ is the classic demo shortcut and it is not a security boundary — (9**9**9) will still happily eat your CPU, and clever expression strings can escape the sandbox. Acceptable in a local toy you run against your own prompts; never in anything that processes other people’s input. The tool-safety rules in LLM Tool Calling §7 apply in full.

A run looks like this — the model batched all three independent tool calls into its first turn, then composed the answer:

Task: What's today's date? What is 15% of 847? And what's the weather in Tokyo?

  > get_current_date({})
  > calculate({'expression': '847 * 0.15'})
  > get_weather({'city': 'Tokyo'})

Answer: Today is 2026-07-23 09:14:22. 15% of 847 is 127.05.
The weather in Tokyo is 72F and partly cloudy.

4. Swapping the brain: cloud to local in one line

The OpenAI API shape has become a de facto ABI: many other servers — Ollama, vLLM, LM Studio, most gateway proxies — expose the same endpoints and JSON, so the same client library talks to all of them. Like linking against a different library that exports the same symbols, the calling code doesn’t change; only the address does:

ollama_client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")  # key required by the client lib, ignored by Ollama
answer = run_agent(task, ollama_client, model="qwen2.5")

With Ollama installed (ollama pull qwen2.5, ollama serve), the identical agent now runs fully offline — useful for iterating on tools without paying per call, and mandatory when the data can’t leave the machine.

The gotcha that will bite you first: not every local model supports native tool calling. A model without it (Mistral 7B is the famous example) responds to your tool schemas with prose describing the calls it would like to make — “I should call get_current_date() to find the date…” — instead of structured tool-call messages. The failure signature is distinctive: msg.tool_calls is empty on the very first turn, so the loop exits immediately and “the answer” is the model narrating its intentions. The code is working exactly as written; the model can’t speak the protocol. Debug rule: an agent that returns instantly without calling any tools is a model problem, not a loop problem — swap to a known tool-calling model (qwen2.5, llama3.1) before touching the code.

5. Mixed mode: a model behind a tool

Nothing says a tool has to be cheap or dumb. Wrap a cloud model in a tool and register it like any other:

def ask_cloud_expert(question: str) -> str:
    """Delegate a hard reasoning question to a stronger cloud model."""
    cloud = OpenAI(api_key=os.environ["OPENAI_API_KEY"])
    r = cloud.chat.completions.create(model="gpt-4o", messages=[{"role": "user", "content": question}])
    return r.choices[0].message.content

Now a local model drives the loop and handles the easy calls, and delegates when it recognizes something beyond its depth — you pay for one cloud call instead of running the whole session at cloud prices. This tiny pattern is worth staring at, because it’s the seed of everything multi-agent: one model deciding to hand a subtask to another model is delegation, and delegation is what A2A-Protocol formalizes (and why that note argues plain tool calls are the wrong shape for it once the delegate needs to be long-running or independent).

6. What the naive loop doesn’t have — and where each gap leads

This loop is a sketch, deliberately. Being explicit about what’s missing is the best map of the rest of the sequence, because every later tier exists to fill one of these gaps:

Missing from the ~80 linesWhat fills it
Tools are hardcoded in the file; no way to share them or use anyone else’sModel-Context-Protocol-MCP — publish/discover tools over a standard protocol
No retries, checkpoints, branching control flow, or human-approval gatesFrameworks — LangGraph vs CrewAI Research, and the tier map in Frameworks, runtimes, and harnesses
One messages list that only grows; long tasks blow out the context windowHarnesses — Deep-Agents: compression, filesystem-as-swap, subagents, persistent memory
No permission model; every tool runs with the program’s full privilegesThe safety rules in LLM Tool Calling §7, and harness-level gates (Deep-Agents)
One agent, one conversation; no delegation to peersA2A-Protocol

The order of operations the sequence recommends is also the order this note earned: understand the loop first, then adopt the machinery when — and only when — you hit the gap it fills. A framework you adopt before writing the loop is abstraction you can’t debug; the same framework adopted after is labor you’re glad to shed.

7. Where this goes next

References