Chapter 12 — The ReAct Pattern and Its Limits

ShopOps has a short plan for O-1001, then a ReAct loop for flexible reads: Thought → Action → Observation. After a carrier ping, the model answers, “Thanks for confirming you will accept the reship.” No tool returned agreed_resolution=true. The scratchpad became a customer fact — and Policy almost acted on it.

ReAct puts the model inside the control loop. The scratchpad can guide the next read; it is not evidence.

thought → action → observation → … → answer | escalate | max steps

This chapter adds production guards: stop conditions, duplicate-tool detection, fabrication checks — so exploratory reads stay bounded and claims that need tool proof cannot become case facts.

First principles

ReAct interleaves:

1. Reason (verbal scratchpad)
2. Act (tool / answer / escalate)
3. Observe (environment)

The pattern helps when the next read depends on the last observation. For O-1001, the model may inspect the order, then decide whether order notes are needed. Only the executor mints observations—the model’s text cannot confirm delivery, refund, or customer agreement.

The runtime’s job: permit exploratory reads, enforce boundaries on every claim and action. A production loop needs:

• Stop conditions — answer, escalate, max steps (Ch 1).
• Decision records — structured artifacts you log; not a requirement to expose hidden chain-of-thought to end users.
• Fabrication guards — claims that require tool evidence.
• Duplicate detection — same tool+args fingerprint → escalate.

The scratchpad can help the model choose what to inspect next. It must not become a source of truth for ShopOps.

Concrete example

Hostile fixture: the model always emits tool:get_order “to be sure.” Without guards, it burns the step budget and can rate-limit the order service. With detect_duplicate_tool_call, the second identical call becomes an escalation—a deliberate outcome rather than a timeout mystery.

Second fixture: model answers with “customer agreed” without any observation where agreed_resolution=true. fabricate_guard rewrites the action to escalate. Policy will thank you; demos will look less magical.

Diagram

flowchart LR
  T[Thought scratchpad] --> A[Action]
  A -->|tool| E[Executor + policy]
  E --> O[Observation]
  O --> T
  A -->|answer/escalate| Stop[Terminate]
  A -.->|duplicate or fabricate| X[Escalate]

Failure gallery (keep this near the on-call doc):

Loop:     get_order → get_order → get_order → ...
Fiction:  Thought: customer agreed  (no tool) → ANSWER
Cost:     80-step exploration of irrelevant ticket notes
Skip:     Thought: already have order → draft  (stale/wrong)

Implementation

Reference module: shopops/react.py.

from shopops.react import (
    ScriptedHappyModel,
    ScriptedHostileModel,
    run_react,
)
from shopops.types import Action, CaseState, Observation

def execute(action: Action) -> Observation:
    if action.tool == "get_order":
        return Observation(ok=True, data={"order_total_cents": 240_00, "days_since_shipped": 32})
    return Observation(ok=False, error="unknown")

def build_prompt(state, turns) -> str:
    return f"ShopOps case {state.case_id}; steps={len(turns)}"

state = CaseState(case_id="O-1001", max_steps=6)
state, trace = run_react(
    state,
    ScriptedHostileModel(),
    execute,
    build_prompt=build_prompt,
)
assert trace.stop_reason == "escalate"
assert trace.duplicate_tool_hits >= 1

Happy path with a scripted model:

model = ScriptedHappyModel([
    "Thought: need order\nAction: tool:get_order case_id=O-1001",
    "Thought: done\nAction: answer order loaded",
])
state, trace = run_react(CaseState(case_id="O-1001"), model, execute, build_prompt=build_prompt)
assert trace.stop_reason == "answer"

Each turn stores a DecisionRecord with confidence=0.0 on purpose: ReAct text is not a calibrated score. Structured confidence arrives in Ch 13 — and even then policy ignores it for writes.

Parser note: parse_react_response is intentionally tiny and hostile to free prose. Unparseable → escalate. Do not “best-effort” guess that a paragraph meant send_email.

Failure modes

1. Non-termination — missing max steps or duplicate detection.
2. Fabricated observations — model writes Observation: in the scratchpad and the runtime treats it as real. Only the executor may mint observations.
3. Cost explosion — unbounded tool fanout; no token/step budgets (Ch 35 preview).
4. Hidden CoT as audit — storing raw thoughts with PII as if they were evidence packs. Decision records ≠ rant logs.
5. Prompt-only stop rules — “stop when done” in the system prompt without code branches.

Production considerations

• Log decision records and tool observations in the trace (Ch 23). Scratchpad optional and redacted.
• Prefer structured decisions (Ch 13) for consequential branches; keep ReAct for exploratory read-only sessions if you must.
• Eval the hostile looping fixture in ring-1 every PR (Ch 20).
• Temperature and sampling: lower for tool choice; never rely on sampling to enforce stop.
• If you need multi-step commitments, use explicit plans (Ch 11) inside the loop rather than hoping Thought paragraphs stay consistent.

Where ReAct fits in ShopOps

Use ReAct-shaped loops for read-heavy exploration with low blast radius: “what do we know about O-1001?” with get_customer / get_order / list_order_notes.

Do not use unconstrained ReAct for send paths. The moment the action space includes send_email or send_whatsapp, you want structured decisions (Ch 13), policy gates (Ch 5, 28), and usually an explicit short plan (Ch 11) that ends in draft_* not send_*.

Hybrid that works: planner or rules pick the stage; inside a read-only stage, ReAct may choose tool order; writes exit to structured Decision + executor. The loop from Ch 1 does not care which policy function you plugged in — as long as stop conditions and budgets remain in code.

Decision records vs hidden CoT

Log: intent, action, evidence refs, policy outcome.
Optional: short rationale string for debugging.
Do not: treat multi-page scratchpads as audit evidence or train staff to trust them over tool observations. Policy reviewers need the ledger and policy deny reasons, not a novel.

Chapter summary

• ReAct = control loop + LM policy; scratchpad is not truth.
• Stop, duplicate detection, and fabrication guards are code.
• Decision records are logged artifacts, not mystical CoT requirements.
• Hostile loop fixtures catch regressions early.
• Unparseable actions escalate — no intent guessing.
• Confidence from prose is fake; leave room for Ch 13.
• Observations come only from the executor.
• Use plans when horizon > 1 needs structure.

Exercises

1. Extend detect_duplicate_tool_call to allow a second get_order if force_refresh=true in args; add a unit test.
2. Add a fixture where the model claims “customer agreed” and assert escalate; then add a tool observation with agreed_resolution=true and assert the guard allows answer.
3. Cap steps at 3 and measure how often golden O-1001 still drafts successfully with a scripted model.
4. Sketch (no code) why exposing full scratchpads to support agents creates privacy and coaching hazards.

References

• Yao et al., 2022. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629.
• Wei et al., 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. arXiv:2201.11903.
• Holtzman et al., 2020. The Curious Case of Neural Text Degeneration. arXiv:1904.09751 — sampling is not an oracle.