TRUST, BUT VERIFY · CHAPTER 03

Agents & Tool Use

An agent is a language model in a loop: think, call a tool, read the observation, repeat until the goal or the budget says stop.

~16 min read1 calculator toolTested Python

What you'll learn

  • How a tool is just a named function with a schema
  • The ReAct trace: Thought, Action, Observation
  • Why the loop needs a stop condition, not just a prompt
  • How RAG is one tool among others, not a competing architecture

Before you start

A cook who can pick up a thermometer

A language model is a cook who has read a lot of recipes and never touched a stove. A tool is the thermometer. The cook still decides when to use it, reads the number, and decides whether to take the pan off. The loop is the job: observe, think, act, stop.

The model does not become “an agent” by being larger. It becomes one when you give it actions, a working memory (the trace), and a termination rule: goal reached, or budget spent. Without the last one it will chatter forever.

One calculator

Question: A kettle holds 1400 ml. I pour 350 ml into each of 3 mugs. How many ml remain in the kettle? The model should not do the arithmetic in its head. It should call a tool.

cook · loop · schema · stop

MANIM Agents: plan, call tools, observe; stop conditions matter.

The tool definition that goes in the prompt (or in the API’s tools list) is a contract:

FieldValue
namecalculator
descriptionEvaluate a basic arithmetic expression and return a number.
inputexpr: string, for example "1400 - 3 * 350"

The host, not the model, runs the function. The model only emits the name and the arguments. That split is the security boundary the next chapter will lean on: the model proposes, the host dispatches, and the host can refuse.

Thought, Action, Observation

ReAct (Yao et al. 2022) is a format for that loop. One turn of our kettle problem:

RoleText
ThoughtI need 1400 minus three times 350.
Actioncalculator("1400 - 3 * 350")
Observation350
ThoughtThat is the remaining volume.
Answer350 ml remain in the kettle.

The Thought is scratch paper. It is not a tool. The Action is the only side effect. The Observation is whatever the host returned, pasted back into the context. The next Thought can use it. That is the whole trick.

RAG from the previous chapter is this pattern with one tool named retrieve. A web search, a calendar, a code interpreter: same loop, different functions.

Stop on goal or on budget

Observeuser question, or last tool result
Thinkwrite a Thought; pick Action or Answer
Acthost runs the tool, or returns the Answer

Termination is part of the design. Cap the steps. Cap the tokens. Cap the wall clock. A coding agent that can run shell commands also needs a human interrupt, because one wrong Action is not a wrong sentence, it is a side effect.

Controller / worker is the same loop with extra models: a planner delegates a subtask to a worker with a tighter prompt and a private context, then reads the worker’s result as an Observation. Errors compound. A worker that hallucinates a file path poisons every later step. Evaluate agents on the final state (did the kettle math land on 350?) more than on how pretty the Thoughts were.

A host that actually runs the tool

agent_loop.pycalculator("1400 - 3 * 350") is 350
"""A one-tool ReAct loop with a calculator.

The question is a kettle-and-mugs arithmetic problem. The model is not
expected to do the arithmetic in its head. It emits a thought, calls
the calculator, reads the observation, and answers.
"""
import ast
import operator

# Own tool. A calculator that accepts a small arithmetic expression.
OPS = {
    ast.Add: operator.add,
    ast.Sub: operator.sub,
    ast.Mult: operator.mul,
    ast.Div: operator.truediv,
    ast.USub: operator.neg,
}


def _eval(node):
    """Walk a tiny AST. No names, no calls, no attributes."""
    if isinstance(node, ast.Expression):
        return _eval(node.body)
    if isinstance(node, ast.Constant) and isinstance(node.value, (int, float)):
        return node.value
    if isinstance(node, ast.UnaryOp) and type(node.op) in OPS:
        return OPS[type(node.op)](_eval(node.operand))
    if isinstance(node, ast.BinOp) and type(node.op) in OPS:
        return OPS[type(node.op)](_eval(node.left), _eval(node.right))
    raise ValueError("unsupported expression")


def calculator(expr):
    """Evaluate a basic arithmetic string. This is the only tool."""
    tree = ast.parse(expr, mode="eval")
    return _eval(tree)


QUESTION = (
    "A kettle holds 1400 ml. I pour 350 ml into each of three mugs. "
    "How many ml remain in the kettle?"
)

# The trace the page shows. Thoughts are the controller's notes.
# The action is the only side effect.
TRACE = [
    {"thought": "I need 1400 minus three times 350.",
     "action": "calculator",
     "input": "1400 - 3 * 350"},
]


def run(trace=None):
    """Execute each tool call and append the observation plus the answer."""
    trace = TRACE if trace is None else trace
    log = []
    for step in trace:
        obs = calculator(step["input"])
        log.append({**step, "observation": obs})
    answer = log[-1]["observation"]
    return log, answer

Mistakes learners make

Letting the model “run” the tool in prose. If it writes “the calculator would say 400”, that is not a tool call. The host never ran. The 350 in this chapter comes from Python, not from the Thought.

No stop condition. A loop without a budget will keep calling tools. Goal-or-budget is part of the definition of an agent here.

Treating RAG and tools as rivals. Retrieval is a tool. Agents are loops that can call it, or not, on each turn.

Scoring the trajectory instead of the outcome. Pretty Thoughts can still leave 8 muffins on the counter. Prefer execution checks on the final state.

What to remember

  • An agent is an LLM in a loop with tools and a stop rule.
  • A tool is a named function plus a schema. The host runs it.
  • ReAct writes Thought, Action, Observation. Our calculator returns 350.
  • RAG is one tool. Errors in a step poison later steps. Evaluate the final state.

Check yourself

1. Who evaluates 1400 − 3 × 350 in this chapter?

The host’s calculator function, which returns 350. The model only proposed the expression.

2. What is the difference between Thought and Action?

Thought is notes in the context. Action is a structured call the host may run.

3. Name two stop conditions.

The model emits an Answer (goal), or the step/token/time budget is gone.

4. Is retrieve-then-read an agent?

It is a one-shot tool call. It becomes an agent when the model can call retrieve again after reading, or call a second tool, inside a loop with a stop rule.

Go to the source

Keep going