AI Engineering
Chapter 39
Agentic patterns
A chatbot answers in one breath. An agent can leave the room: search a doc, run a query, create a ticket, come back, and finish the sentence. The plot twist is that the model still only emits tokens — your runtime turns certain tokens into tool calls and feeds results back as the next scene.
The think → act → observe loop
Each cycle the model decides whether it can answer or must act. If it acts, it emits a structured tool call. Your code executes the tool, appends the observation, and invites another thought. Stop when the model returns a final answer, hits a step limit, or a policy gate refuses.
def run_agent(messages, tools, max_steps=8):
for _ in range(max_steps):
msg = call_model(messages, tools=tools)
if not msg.tool_calls:
return msg.content
messages.append(msg)
for call in msg.tool_calls:
result = dispatch(call.name, call.arguments)
messages.append({
"role": "tool",
"tool_call_id": call.id,
"content": result,
})
return "Stopped: step budget exceeded"
| Pattern | Story | Use when |
|---|---|---|
| Tool calling | Model picks a function and args | APIs, search, DB reads |
| ReAct | Interleave reasoning and actions | Multi-step research |
| Planner–executor | Plan first, then execute | Long workflows needing review |
| Reflection | Critique or retry after a result | Code fix / quality loops |
| Router | Classify intent → specialist | Many domains in one product |
| Human-in-the-loop | Pause for approval on risky acts | Spend money, send email, delete |
Tool design is UX for models
Give tools clear names, terse descriptions, and strict JSON schemas. Prefer many small tools over one god tool. Return compact structured observations. Include recoverable errors (“not found”, “auth expired”).
Control loops keep agents honest
- Hard caps on steps, tokens, and wall-clock time.
- Allowlists of tools per task type.
- Confirmation for irreversible side effects.
- Idempotency keys so retries do not double-charge.
- Full transcript logging for debugging and evals.
When not to use an agent
If the task is a single classification or a grounded RAG answer, a one-shot call is cheaper and more reliable. Agents shine when the path is unknown and the world must be inspected.
Agents turn language into action. Your job is to keep that power on a leash made of schemas, budgets, and truth from tools.
Walkthrough: “File a bug from this stack trace”
The model reads the trace, calls search_code, observes a suspicious function, calls create_issue with a draft, and only then returns a link to the user. Midway, if create_issue fails auth, the observation says so; the model retries after refresh_token or asks the user. That recovery path is the difference between a demo and a system.
Parallel tool calls
Some hosts allow multiple tools in one step (search docs and check status together). Parallelism cuts latency but complicates dependency. Teach the model which tools are independent. Enforce timeouts per tool.
State machines around the LLM
Serious products wrap the free-form loop in an explicit state machine: collect → confirm → execute → verify. The LLM fills fields; the machine owns transitions. That hybrid is often safer than pure ReAct for money paths.
Interview drill — Agents
Tool design + control loops win these interviews — not sci-fi autonomy.
More drills in the Interview Lab.
Q1. Flight-booking agent
Search and book with confirmation and budgets.
- List tools with JSON Schema and auth scopes.
- Orchestrate plan→act→observe with session slots.
- Gate pay/book behind user confirm + server checks.
- Cap steps/tokens/$; idempotent side effects.
Q2. Infinite tool loops
Agent keeps calling search forever.
- Max iterations and wall-clock timeout.
- Detect repeated identical (tool, args).
- Force finalize or escalate to user.
- Budget alarms in the gateway (Q8).
Q3. Prompt injection via tool output
A webpage says to ignore policies and refund $10k.
Treat tool output as untrusted data, not instructions. Isolate from system policy. Allowlist irreversible tools; enforce refunds in code. Add injection cases to eval.
Q4. Human-in-the-loop
When must a human approve?
Payments, deletes, external emails, production changes, anything regulated or irreversible. Persist the approval artifact.
Q5. When not to use an agent
Interviewers love this.
If the workflow is a fixed state machine, ship software + RAG. Use agents when tool choice truly branches and uncertainty is high.