Chapter 2: The Anatomy of an Agent

Maya opened case O-1001 for Jordan’s stalled shipment. ShopOps is running the chain from Chapter 1 — load facts, draft email, wait for approval before send. The temptation is to let one model call own every branch: when to look up the order, when to draft, when to send, when to stop.

If the model owns all of that, policy lives in the system prompt. The first time the model is confidently wrong — and models often are — Jordan gets mail the business did not authorize, or the case never terminates.

Jordan asks → Maya opens O-1001 → observe → decide → act → observe → … → terminate

That loop is not one blob called “the agent.” It decomposes into model, state, actions, control policy, and environment — each piece fails differently and each must be testable on its own. This chapter names those parts and fixes ShopOps’s invariant: model proposes; policy and code dispose. The language model drafts wording; it does not get to decide that Maya’s approval is optional.

First principles

An agent decomposes into:

Agent ≈ Model + State + Actions + Control Policy + Environment

Why bother decomposing? Because each piece fails differently. The model hallucinates. The environment times out. The policy pack has a bug. If those are mashed into one prompt blob, you cannot test or attribute anything.

The dynamics we care about are a state transition:

[ s_{t+1} = F(s_t, a_t, o_{t+1}) ]

The action space (\mathcal{A}) should be closed or tightly constrained when the stakes are real. Free-text “do something sensible” is not an action space. It is a hope.

Hybrid control is the default: the model samples a proposal; deterministic gates filter, rewrite, or reject before side effects.

Misconceptions to kill:

• The model must own all control flow.
• State equals the chat transcript.

Concrete example

For O-1001 we keep typed objects on purpose:

• State — order total, draft, approval flags, phase, step count
• Action — kind in a closed vocabulary
• Observation — ok, data, error
• transition — implements (F)

Policy forbids send without approval in state. The model (or scripted stand-in) may still propose send_email. (F) and the environment record awaiting_approval instead of email_sent=True. That is hybrid control in one paragraph.

Diagram

stateDiagram-v2
  [*] --> idle
  idle --> planning: start
  planning --> awaiting_tool: tool call
  awaiting_tool --> planning: tool ok
  planning --> awaiting_approval: propose send
  awaiting_approval --> executing: approve
  executing --> completed: done
  planning --> completed: done
  awaiting_approval --> failed: fail

Hybrid split:

┌────────────┐   proposal â    ┌──────────────┐   permitted a   ┌─────┐
│ Model / πₘ │ ──────────────► │ Code gates   │ ──────────────► │ Env │
└────────────┘                 │ policy, authz│                 └──┬──┘
                               └──────────────┘                    │
                                      ▲                            │
                                      │         observation o      │
                                      └────────────────────────────┘

Notice: the model never talks directly to the email gateway. That is not paranoia. That is blast-radius control.

Implementation

Run:

PYTHONPATH=. python examples/ch02_anatomy.py

The teaching point is transition as a pure-ish (F): given (state, action, observation), return the next state without hidden globals. Swap scripted_policy for mock_llm_policy and the loop shape should not change — only the proposer does.

Failure modes

State = transcript. You cannot reliably ask “are we awaiting approval?” with string search.

Open action space. The model invents wire_money_now. If your executor will run any string, you built a remote code path with vibes.

Policy only in the prompt. Injection or a hotter sample bypasses it. Put permit/deny in code (Ch 5, 28).

(F) with side effects. If transition sends email, you cannot unit-test dynamics. Side effects belong in the environment / executor.

Production considerations

• Keep (\mathcal{A}) versioned next to the tool registry.
• Log ((s, a, o)) every step (traces in Part VII).
• Treat “model owns the graph” as a smell unless the domain is truly open-ended and low-stakes.
• Prefer hybrid for refunds, sends, and anything irreversible.

Chapter summary

• Agent = model + state + actions + control policy + environment.
• (s_{t+1}=F(s_t,a_t,o_{t+1})) makes dynamics explicit.
• Hybrid control: model proposes; code disposes.
• Closed action spaces beat free-text hopes.
• Chat transcript is not structured state.
• Code: shopops/types.py, examples/ch02_anatomy.py.

Exercises

1. Encode exactly three legal actions for “order lookup only”; reject others.
2. Reject free-text kind values in Action.__post_init__.
3. Add awaiting_approval as a first-class phase; assert send cannot complete without it.
4. Swap scripted policy for a stub that always proposes send_email first; show gates still park the case.

References

• Sutton & Barto — MDP intuition for state/action/observation (selected sections). [VERIFY]
• Industry notes on hybrid agent control (proposal vs execution). [VERIFY]