The Agent Loop: Thought → Action → Observation

In the last lesson you learned what makes something an agent. This lesson reveals the mechanism — the single most important idea in all of agent engineering, the thing every agent (from a toy ReAct script to Claude Code) is built on: the agent loop.

Here's the whole secret, in one sentence: a chatbot is one LLM call; an agent is an LLM calling tools in a loop until the job is done. That loop — reason, act, observe, repeat — is what transforms a one-shot text predictor into something that can take actions in the world, see what happens, and adapt. Understand this loop deeply and every agent framework, pattern, and failure mode in this container will make sense.

In this lesson:

• The Thought → Action → Observation cycle (and ReAct, where it comes from)
• Why the loop works — grounding, multi-step reasoning, adaptivity
• How it's actually built (it's a while loop) and how it stops
• The honest costs — why agent loops get expensive, and how to bound them

The canonical formulation comes from ReAct (Yao et al., 2022) — "Reasoning + Acting." The insight was to interleave the model's reasoning with its tool use, in a repeating three-beat cycle:

1. 💭 Thought — free-text reasoning about what to do next. The model "talks to itself": "To answer this, I first need to look up X." This is where it plans, tracks progress, and handles surprises.
2. 🔧 Action — a tool invocation with typed arguments. The model decides which tool to call and with what inputs — web_search("..."), get_weather("Paris"), run_sql("...").
3. 👁️ Observation — the real tool result, fed back into the context. The runtime executes the action and returns what actually happened, which the model reads before its next thought.

Then it loops: the new observation informs the next Thought, which leads to the next Action, and so on — until the model decides it has enough and emits a Final Answer. A trace makes it concrete:

Notice the rhythm: reason → act → get a real fact → reason again. Each loop builds on observed reality, not the model's imagination. That single property is the engine of the whole thing.

Here's the demystifying truth that trips up most beginners: the loop is code that the LLM runs inside, not something the LLM does internally. The model itself is still a stateless function — text in, text out. The agent is a small program — literally a while loop — that calls the model, runs the tools the model asks for, feeds results back, and repeats:

Trace the four moves and you have every agent runtime ever built:

• Call the LLM with the running transcript (system prompt + tool schemas + everything so far).
• Is the reply a tool call? If no → it's the final answer, return (exit the loop).
• If yes → execute the tool(s) and append the result as an observation.
• Repeat.

Modern agents use native function/tool calling (the API returns a structured tool_call with a name and typed args) rather than the older approach of parsing tool calls out of free text — it's far more robust. But the control flow is identical. (This is the loop you'll build from scratch in the very next lesson.)

Theory clicks when you watch it. Step through a real task below — the agent reasons, picks a tool, gets a grounded observation, and reasons again, looping until it can answer. Keep an eye on the LLM-call counter:

Two things to feel as you step:

1. Every Action is followed by a real Observation before the next Thought — the agent never builds on a guess when it can build on a fact.
2. Each iteration is another LLM call, and the transcript keeps growing. A 3-step task is at least 3 model calls over an ever-larger context — which is exactly why the loop is powerful and why it gets expensive (more on that below).

A single LLM call can only work with what's already in its context — and if it doesn't know something, it guesses (hallucinates). The loop fixes this by giving the model a way to interact with the world, and that unlocks three capabilities a one-shot call simply cannot have:

1. Grounding (the killer feature). By inserting a real observation between reasoning steps, every fact gets checked against reality before the next thought builds on it. Instead of guessing the weather, the agent calls the weather API and reads the answer. This collapses the error surface on any task needing real-world information — search, retrieval, code execution, database queries. Agents reduce hallucination by requiring the model to gather facts or verify its work before answering.

2. Multi-step chaining. Many tasks require using the result of one step as the input to the next (find the host city → then look up its weather → then convert units). A single call can't do this; the loop does it naturally — each observation feeds the next action.

3. Adaptivity. Because the agent sees the result of each action, it can change course: if a search returns nothing useful, the next thought tries a different query; if a tool errors, it recovers. The plan isn't fixed up front — it evolves with what's observed.

This is not a new idea — it's the Observe → Think → Act cycle from robotics and reinforcement learning (and the military OODA loop). An LLM agent is that classic perception-action loop, with a language model as the "think" step. Empirically it works: ReAct beat reasoning-alone and acting-alone by +34% on ALFWorld and +10% on WebShop — because thinking and acting reinforce each other when interleaved.

There's a subtle, crucial problem hiding in "repeat until done": an LLM has no built-in concept of "done." Left unbounded, the loop just keeps going until it hits the token limit — or, worse, spins forever. Termination is something you must engineer, with two complementary mechanisms:

• The model signals completion — by producing a response with no tool call (a final answer). This is the intended exit, and it's the agent's autonomy: it decides when it has enough.
• You impose hard limits — a max_steps cap, a token/cost budget, and a timeout — as the safety net for when the model doesn't know it's done. (That's the for step in range(max_steps) line in the code, and the cap shown in the widget.)

This is non-negotiable, and it's the same hard-cap discipline from agentic RAG (L106) and routing (L101): never ship an unbounded agent loop. Without a maximum loop count, "a confused agent can burn hundreds of dollars in API calls chasing a dead end." Practical guidance: set a dollar ceiling per task (commonly $0.50–$2, up to $5–10 for complex research) and a step cap, then terminate gracefully when either is hit.

The loop is powerful, but it has a cost profile that surprises people — and the surprises are where agents go wrong in production:

• Cost grows triangularly, not linearly. This is the big one. Each iteration re-sends the entire growing transcript to the model, so a 10-step task is not 10× a single call — it's a triangular series, because every step re-bills the content of all the steps before it. Long agent runs get costly fast.
• Context rot. Each step appends the thought, action, and observation, so context balloons — you might feed 8k tokens at step 10 into a loop that started at 500. And models get measurably worse at reasoning as their context fills ("context rot"), so a long loop can degrade itself.
• Latency stacks up. Each iteration adds ~1–5 seconds; time-to-first-token scales with context length, so multi-step runs can take minutes.
• The #1 failure mode: retrying the same failed action. A confused agent will call the same broken tool with the same args over and over, making no progress — the most common way agent loops fail.

So production-grade loops add guardrails: a step cap + cost budget (always), loop/repeat detection (stop the agent re-trying a failing action), and context compaction (summarize or trim the transcript to fight rot and triangular cost). The loop is the heartbeat of every agent — but a heartbeat with no limiter is a liability. We'll harden all of this across the rest of the container.

Step through the widget, then reason about the loop:

1. Run a task to its Final Answer. Count the Observations — and explain why the agent calling those tools makes it more trustworthy than a chatbot answering the same question in one shot.
2. In the loop code, which single line is the difference between a safe agent and one that could burn hundreds of dollars? What does removing it do?
3. An agent's last reply contains no tool call. What does the loop do, and why?
4. Your agent keeps calling web_search with the exact same query five times and never finishes. Name the failure mode and two guardrails that catch it.
5. A teammate says "a 10-step agent just costs 10× a single LLM call." Correct them — what's the real cost shape, and why?

→ (1) Each Observation is a real tool result, so the agent's facts are grounded (it looked up the host city and read the weather) rather than guessed from memory — that's the anti-hallucination property a one-shot chatbot lacks. (2) for step in range(max_steps) (the step cap) — remove it and the loop runs until the token limit or forever, since the LLM has no concept of "done." (3) It exits and returns that reply as the final answer — "no tool call" is the model's signal that it's finished reasoning and is answering. (4) Retrying the same failed action (the #1 failure); catch it with loop/repeat detection (stop identical repeated calls) and a step cap + cost budget. (5) It's triangular, not linear — each step re-sends the whole growing transcript, so every step re-bills all prior steps; long runs cost far more than N× a single call (and context rot can degrade quality too).

• "An agent is a special kind of LLM." No — it's an ordinary LLM inside a while loop that runs tools and feeds results back. The loop (and the tools) make it an agent, not the model.
• "The model decides everything internally." The model only emits thoughts and tool calls; the runtime executes tools, appends observations, and controls the loop. Agency is split between model and harness.
• "Reasoning or tool use — pick one." ReAct interleaves both, and they reinforce each other: reasoning plans the actions; observations ground the reasoning (+34% on ALFWorld vs either alone).
• "The agent knows when it's done." LLMs have no concept of "done" — it signals completion by emitting no tool call, and you must add a step cap + budget so it can't run forever.
• "A 10-step agent costs 10× one call." It's triangular — each step re-bills the whole growing transcript. Long loops get expensive and suffer context rot.
• "Just let it loop until it succeeds." The #1 failure is retrying the same failed action. Bound the loop and detect repeats.

• A chatbot is one LLM call; an agent is an LLM calling tools in a loop until done. The loop — Thought → Action → Observation → repeat → Final Answer (ReAct, Yao 2022) — is the agency.
• It's a while loop, not magic: call the LLM → if it's a tool call, execute it and append the observation → repeat → no tool call = final answer. Modern agents use native function calling.
• Why it works: each step is grounded by a real observation (collapses hallucination), enabling multi-step chaining and adaptivity — the robotics Observe → Think → Act cycle.
• Stopping is engineered: the model signals done by emitting no tool call, but LLMs have no concept of "done" — always add a max-step cap + cost budget + timeout.
• Honest cost: each iteration re-bills the growing transcript → cost is triangular, context rots, latency stacks, and the #1 failure is retrying the same failed action. Cap, budget, detect loops, compact context.
• Next: Building a ReAct Agent From Scratch — you'll implement this exact loop, with real tools, in ~50 lines and no framework.