Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

49. Common Architectural Mistakes

Most agent failures are not model failures. They are boundaries the design erased — then shipped anyway.

A typical design doc: eight role-play agents, unrestricted tools “for flexibility,” a vector DB labeled “memory,” success measured by one golden answer. It ships. Send-window rules break on O-1001. Required fields drop out of Outreach drafts. Nobody can replay why.

probabilistic proposal → (missing) closed actions, policy-as-code, checkpointed state, trajectory evals → production incident

The failure mode is probabilistic proposals without systems design. The fix is almost never a better prompt. It is restoring what the diagram left out.

This chapter adds a design-review checklist — anti-patterns mapped to ShopOps counterexamples (Intake → Resolution → Policy → Outreach, not agent cosplay).

First principles

Architectural mistakes share one root: you let a sampler’s output stand in for permit/deny, memory semantics, or audit. ShopOps’s four agents — Intake, Resolution, Policy, Outreach — exist because separation of duty and tool scope demand it, not because agents are fashionable.

Concrete catalog

1. Agents for ordinary functions

Mistake: A separate agent for every pure function (FormatDateAgent).
Why it hurts: Coordination cost without permission, context, or eval benefit.
ShopOps counter: Formatters are functions; Intake/Resolution/Policy/Outreach split only for SoD and tool scopes (Ch 15).

2. Unrestricted tools

Mistake: One API key; model may call anything.
Why it hurts: Injection becomes authority; blast radius is the estate.
Counter: Policy-aware executor, least privilege, dry-run, approval for sends (Ch 5, 27–28).

3. Store-everything memory

Mistake: Embed every turn; retrieve forever.
Why it hurts: Poisoning, staleness, privacy landmines; retrieval ≠ memory semantics.
Counter: Typed memory + controller; provenance; forget/delete (Ch 9–10).

4. Traces ≠ evals

Mistake: “We log prompts, so we’re covered.”
Why it hurts: Logs without properties do not catch illegal actions.
Counter: Harness with trajectory assertions; CI gates (Ch 19–20).

5. Rules only in prompts

Mistake: send window as system-prompt text.
Why it hurts: Attention fails; attacks rewrite; no explainable deny.
Counter: Policy-as-code; prompt may remind, code decides (Ch 28).

6. Debate equals truth

Mistake: Multi-agent argument as verification.
Why it hurts: Confident consensus on falsehoods; cost explosion.
Counter: Deterministic checks first; ledger conflicts as data (Ch 15–17).

7. Uncalibrated confidence

Mistake: Trust model confidence: 0.9 for send.
Why it hurts: Calibration is weak; policy must ignore vanity scores.
Counter: Structured decisions with evidence IDs; thresholds on beliefs (Ch 13, 43).

8. State only in chat

Mistake: FSM status implied by last messages.
Why it hurts: Lost constraints; unresumable crashes; no HITL park.
Counter: Explicit state + checkpoints (Ch 7–8).

9. No replay / audit

Mistake: Chat export as audit story.
Why it hurts: Cannot reconstruct consequential actions.
Counter: Evidence packs; hash-chained ledger; retention (Ch 30).

10. Online self-improvement without gates

Mistake: Nightly prompt mutation from thumbs.
Why it hurts: Policy drift.
Counter: Bounded improvement pipeline (Ch 46).

11. Protocol worship

Mistake: “We speak MCP/A2A, so we’re safe/interoperable enough.”
Why it hurts: Trust not included.
Counter: Layer cake — MCP / A2A / ledger with authz (Ch 47).

12. Dashboard causation

Mistake: Channel correlation → default policy.
Why it hurts: Confounders.
Counter: Claim levels; experiments (Ch 44).

Diagram

flowchart TB
  bad[Design smell] --> q1{Splits for SoD / eval / isolation?}
  q1 -->|no| merge[Merge to modules]
  q1 -->|yes| q2{Tools least privilege + policy?}
  q2 -->|no| lock[Add executor gates]
  q2 -->|yes| q3{State outside chat + checkpoints?}
  q3 -->|no| state[Add FSM + store]
  q3 -->|yes| q4{Evals on trajectories?}
  q4 -->|no| harness[Build harness]
  q4 -->|yes| q5{Audit pack for consequential acts?}
  q5 -->|no| audit[Add ledger / evidence]
  q5 -->|yes| ok[Proceed to Ch 50 method]

Caption: A short smell → fix ladder for design review.

Implementation

Checklist as code-friendly YAML for PR templates:

# .github/agent_design_checklist.yml
mistakes:
  - id: agents_for_functions
    ask: "Is each agent justified by permissions, context, model, isolation, or eval?"
  - id: unrestricted_tools
    ask: "What is the deny path for send_email without approval?"
  - id: store_everything_memory
    ask: "What is forbidden to write into memory?"
  - id: traces_ne_evals
    ask: "Which fixtures fail the build if send-window rules break?"
  - id: rules_in_prompts
    ask: "Which rules are enforced in policy-as-code?"
  - id: debate_eq_truth
    ask: "What deterministic check runs before any debate?"
  - id: uncalibrated_confidence
    ask: "Is model confidence ignored by the executor?"
  - id: state_in_chat
    ask: "Where is fsm_status persisted?"
  - id: no_replay_audit
    ask: "Can we export an evidence pack for case O-1001?"

Failure modes

Ironically, the meta-failure is checklist theater: green boxes without code. Require links to fixtures, policy tests, and threat models.

Production considerations

  • Run this catalog in architecture review before net-new agents.
  • Re-audit after major model upgrades — capability changes revive old mistakes.
  • Pair with Appendix D/E (security / evaluation checklists).

Chapter summary

  • Most agent failures are architectural erasures of boundaries.
  • Do not spawn agents for functions; do not leave tools unrestricted.
  • Memory, policy, eval, state, and audit each need explicit homes.
  • Debate, confidence, and protocols are not truth or trust engines.
  • Use the catalog as a living design-review gate.

Exercises

  1. Audit: Apply the twelve mistakes to a real design doc; file issues for each hit.
  2. Repair: Take an “eight agents” design and collapse it to the minimum justified set.
  3. Fixture: Add one CI test that would have caught a past incident via this catalog.
  4. Teach: Present one mistake with a ShopOps counterexample to your team in 10 minutes.

References

  • Anthropic, Building Effective Agents (2024) — when not to build agents. [VERIFY URL]
  • OWASP LLM Top 10 — unrestricted agency / injection classes. [VERIFY]
  • Cross-links: entire book; especially Ch 5, 9–10, 15–18, 20, 28, 30, 44–47, 50.