Keyboard shortcuts

Press or to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Chapter 28 — Policy-as-Code

Resolution proposes a full refund for O-1001 — Jordan’s headphones cost $129, above the store’s auto-cap. Outreach drafts a convincing exception email with all required fields. The prose is fine. The rule pack says deny unless Maya approves.

Resolution proposes → PolicyEngine evaluates (send window, refund cap, frequency, required fields) → PERMIT or DENY with rule_ids

Return windows, refund limits, send window, contact frequency, and required fields cannot live in a prompt. “Be careful” is not enforceable; Jordan gets money the store did not authorize.

This chapter adds deterministic policy evaluation: versioned rules in code, explainable denies, draft-vs-send separation — Policy judges; Outreach acts only on the verdict.

First principles

  1. Store rules and money-movement limits belong in code or another deterministic, versioned, tested engine.
  2. Explainable denies. “Denied” without rule_ids and reasons is ops darkness.
  3. Draft ≠ send. Many rules allow composing a message while forbidding delivery.
  4. Version pin in traces. When behaviour changes, you must know whether the pack changed.
  5. Prompts can summarize policy for the model; they cannot be the enforcement point.
  6. Property tests beat demo happy paths. Freeze fixtures for return window, refund cap, frequency, required fields, opt-out.

Illustrative ShopOps rules (teaching examples; real stores differ):

  • Refund auto-cap: above $50 requires HITL (Ch 29).
  • Return window: e.g. 30 days from delivery for refund paths.
  • SMS/push send window: allowed only 08:00–21:00 local (email drafts still allowed).
  • Frequency: max contacts / day and week for proactive outreach.
  • Required message fields: store identity, order number, and purpose of update.
  • Opt-out: permanent block on marketing or outreach sends.

Concrete example

Resolution proposes WhatsApp for O-1001 at 21:30 Eastern with all required fields. Policy:

DENY effect=deny
reasons=["send window: local=...T21:30:00-04:00"]
rule_ids=["send_window"]
policy_version="shop_v1.0.0"

Same body at 10:00 with is_draft_only=TruePERMIT (draft). Same body at 10:00 ready to send with 2 contacts already today → frequency deny. The model’s confidence field is ignored (Ch 13 reprise).

Diagram — propose / permit / execute

flowchart LR
  M[Model / Resolution] -->|OutreachProposal| P[PolicyEngine]
  P -->|PERMIT| X[Executor]
  P -->|DENY| S[State: blocked / replan / HITL]
  X --> W[Channel / systems]
  P -->|verdict| T[Trace + Audit]
 proposal ──► evaluate_outreach ──┬── DENY ──► record reasons ──► no side effect
                                 └── PERMIT ─► authz ─► execute

Caption: Notice policy sits in front of side effects and writes machine-readable reasons either way.

Implementation

Pack: shopops/policy/packs/shop_v1.py. Engine: shopops/policy/engine.py.

from datetime import datetime
from zoneinfo import ZoneInfo
from shopops.policy.engine import PolicyEngine
from shopops.policy.packs.shop_v1 import REQUIRED_MESSAGE_FIELDS

engine = PolicyEngine()  # pins shop_v1.0.0
eastern = ZoneInfo("America/New_York")

ctx = dict(
    case_id="O-1001",
    channel="email",
    body="…",
    local_dt=datetime(2026, 7, 18, 21, 30, tzinfo=eastern),
    contacts_last_24h=0,
    contacts_last_7d=1,
    required_fields_present=frozenset(REQUIRED_MESSAGE_FIELDS),
    is_draft_only=False,
    customer_opted_out=False,
)
verdict = engine.decide("send_email", ctx)
assert not verdict.allowed
assert "send_window" in verdict.rule_ids

Core evaluators (simplified from the pack):

def in_send_window(local_dt: datetime) -> bool:
    t = local_dt.timetz().replace(tzinfo=None)
    return time(21, 0) <= t or t < time(8, 0)

def evaluate_outreach(p: OutreachProposal) -> PolicyVerdict:
    for checker in (check_opt_out, check_send_window,
                    check_frequency, check_message_fields,
                    check_amount_threshold):
        if (v := checker(p)) is not None:
            return v
    return _permit("contact permitted", rules=("outreach_ok",))

Keep functions pure: no clock inside checkers except via injected local_dt. That is what makes fixtures freeze.

Failure modes

FailureWhy it hurtsFix
Rules only in promptsInjection / drift / untestablePack + engine
Hidden defaults“Works in staging” at wrong TZExplicit TZ on proposals
Soft frequencyModel “thinks” it is fineCount from ledger, not memory
Required-fields theatreModel claims required fields without tokensChecklist in pack
Silent pack upgradeBehaviour change undiagnosedVersion in every verdict/trace
Policy after executeRace: send then denyGate before side effects

Production considerations

  • Treat pack releases like any other policy-pack change: review, fixture suite, staged rollout.
  • Encode jurisdiction packs (shop_us_v1, shop_uk_v1) selected by tenant config — still code.
  • Emit policy_version + rule_ids into audit evidence packs (Ch 30).
  • Allow draft under deny-for-send so humans can edit (Ch 40).
  • Never let the model edit pack source at runtime (Ch 41 foreshadow).

Chapter summary

  • Policy-as-code is deterministic permit/deny with reasons.
  • send window, frequency, required fields, opt-out, thresholds are pack material.
  • Draft and send are different actions under the same rules.
  • Version pins make regressions attributable.
  • Prompts explain; packs enforce.
  • Pure functions + frozen fixtures are the test strategy.
  • Deny paths are first-class product behaviour.
  • HITL thresholds bridge policy to humans (next chapter).

Exercises

  1. Mechanical. Write pytest cases for 20:59 permit, 21:00 deny, 07:59 deny, 08:00 permit (Eastern).
  2. Property. For random contacts_last_24h in 0..5, assert deny iff >= 2 on send.
  3. Design. Sketch a second pack for a tenant that bans voice entirely; show how the engine selects packs by tenant_id.

References

  • Store return/refund and contact rules — jurisdiction-specific; treat book rules as illustrative. [VERIFY SOURCE]
  • Model risk management: SR 11-7 (governance of models vs controls). Board of Governors, 2011.
  • Chapters 5, 13, 27, 29, 38