Introduction
An LLM call is not an agent.
An agent is a stateful decision-making system under uncertainty. It observes the world, picks an action from a constrained set, updates state, and stops — under rules you can test. That is the whole thesis. Everything else in this book is that sentence made concrete.
Most “agent” demos are a prompt plus tool calling. That can look smart in a screenshot and still be unsafe in production. The model is good at proposing. It is bad at being the sole authority over money, messages, and side effects. So we build systems around the model: loops with budgets, typed tools, policy-as-code, checkpoints, traces, evals.
What you will build
ShopOps — a fictional post-purchase ops system for Northline, a headphone retailer.
Jordan Lee bought a pair — order O-1001. Carrier tracking stalls. Jordan writes support: Where is my order? Maya cannot hand-write every status email, so she opens a ShopOps case. Intake loads customer and order facts. Resolution classifies the issue. Policy checks send window and required fields. Outreach drafts a reply — and only sends after policy and Maya approve. Later chapters add refund and reship paths. Audit is not polish; it is how you answer “why did we email that?” at 2 a.m.
Jordan asks → Maya opens O-1001 → Intake → Resolution → Policy → Outreach draft → approve → send (or escalate)
Code lives in manuscript/code/shopops/. It starts as a short control loop and grows into workers, policy packs, multi-agent coordination, and a Compose sketch. Frameworks show up late, as a Rosetta stone — never as the definition of the idea.
The map (what each idea is for)
If you cannot say what a concept is for, you do not understand it yet. Hold this table loosely; we will fill it with code.
| Concept | Job |
|---|---|
| Control loop | Sense → decide → act → observe → update → stop |
| (s_{t+1}=F(s_t,a_t,o_{t+1})) | Make dynamics explicit and testable |
| Context assembler | Fit working memory into a token budget on purpose |
| Tools + schemas | Ground actions in real systems |
| Policy-as-code | Permit/deny without trusting the prompt |
| Checkpoints | Survive crashes and human pauses |
| Memory controller | Decide what is written, superseded, forgotten |
| Ledger | Share facts without chat chaos |
| Eval harness | Score trajectories and properties, not lucky answers |
| Trace / run-diff | Debug “why did this run differ?” |
| MCP / A2A | Wire formats — not trust |
How the book is organized
- Parts I–II — From completion to tools and reliability.
- Parts III–IV — State, memory, planning, structured decisions.
- Parts V–VI — Multi-agent coordination and evaluation.
- Parts VII–VIII — Observability, security, governance.
- Part IX — Production runtime topology.
- Part X — Full ShopOps vertical: Intake → Resolution → Policy → Outreach → HITL → feedback → deploy.
- Parts XI–XII — Uncertainty, learning, protocols, mistakes, a design method.
- Appendices — Setup, serving math (KV cache / sampling), checklists, glossary, framework Rosetta.
How to read this
Type the code. Break the fixtures. Prefer one clear invariant over a derivation dump. No fake benchmarks. No invented citations. No “the model wants.” Autonomy is earned; governance is designed.
Prerequisites: Python 3.11+, basic LLM API familiarity, and treating distributed failure as normal. Appendix B is optional math for Chapter 35.
One line to keep
Reliable agency is better systems around probabilistic intelligence. ShopOps is that claim with an order ID attached.