44. Causal Reasoning in Agent Systems

Ops posts a chart: WhatsApp reminders correlate with 12% higher resolution-kept rate on cases like O-1001. Someone proposes making WhatsApp the default for every stalled shipment. Nobody asks who was selected into WhatsApp — channel preference, exception track, or Maya’s override.

That is association, not intervention. An agent that treats the chart as a policy diff will harm the wrong cohort and call it learning.

dashboard metric (associational) → label claim level → experiment or stratified report → (never auto-rewrite Outreach/Policy from correlation alone)

Chapter 41’s scorecards are mostly rung-1 observations unless you design otherwise. Without causal discipline, feedback loops become policy roulette — force WhatsApp because a chart said so, ignore confounders, skip send-window and required-field regression.

This chapter adds claim-level labeling and identification discipline so outcome analysis cannot silently become policy mutation.

First principles

Pearl’s ladder, used lightly:

1. Association — (P(y\mid x)). What we see together.
2. Intervention — (P(y\mid do(x))). What happens if we set (x).
3. Counterfactual — what would have happened under another action.

Agent systems live on rung 2 whenever they act. Scorecards from Ch 41 are mostly rung 1 unless you design otherwise.

Confounders in ShopOps order-support messaging:

Prediction (will this customer accept the resolution?) and action (should we send this message?) demand different evidence. A calibrated classifier is not a license to intervene.

Author recommendation: label every outcome analysis as associational until an identification strategy exists (experiment, natural experiment, or credible causal model). Do not ship causal claims from dashboards alone.

Concrete example

Fictional analysis — not a real result:

Among cases with a WhatsApp send, resolution-kept rate was higher than email.

Caution labels the system must attach:

CAUTION: associational only
- Channel assigned by preference + resolution plan + policy, not randomized
- Exception track over-represented on WhatsApp
- Do not auto-rewrite Outreach policy from this chart

Valid next steps: A/B on eligible population with policy constraints; or stratified reporting that holds exception band fixed — still not automatic policy mutation.

Diagram

flowchart TB
  H[Exception / fraud risk] --> C[Channel choice]
  H --> Y[Resolution outcome]
  C --> Y
  P[Policy / send window] --> C
  U[Human override] --> C
  U --> Y

Caption: Notice the backdoor paths through exception and humans. Conditioning on channel alone does not identify the effect of forcing WhatsApp.

Implementation

Sketch an outcome analyzer that refuses to emit causal language:

# shopops/analytics/outcomes.py — teaching sketch
from __future__ import annotations

from dataclasses import dataclass
from typing import Any, Literal


ClaimLevel = Literal["associational", "experimental", "unidentified"]


@dataclass(frozen=True)
class OutcomeClaim:
    metric: str
    value: float
    level: ClaimLevel
    cautions: list[str]
    n: int


def channel_vs_resolution(rows: list[dict[str, Any]]) -> OutcomeClaim:
    """Aggregate only. No do() operator. No causal verbs in `metric`."""
    by: dict[str, list[int]] = {}
    for r in rows:
        by.setdefault(r["channel"], []).append(int(r["resolution_kept"]))

    # Example: report WhatsApp rate if present — still associational
    wa = by.get("whatsapp", [])
    rate = sum(wa) / len(wa) if wa else 0.0

    return OutcomeClaim(
        metric="resolution_kept_rate|channel=whatsapp",
        value=rate,
        level="associational",
        cautions=[
            "Channel not randomly assigned",
            "Stratify by exception belief before comparing",
            "Forbidden: auto-mutate Outreach policy from this claim",
        ],
        n=len(wa),
    )


def render_for_humans(claim: OutcomeClaim) -> str:
    ban = ["causes", "causal", "lift from sending", "should default"]
    text = (
        f"{claim.metric} = {claim.value:.3f} (n={claim.n}) "
        f"[{claim.level}]\n" + "\n".join(f"- {c}" for c in claim.cautions)
    )
    lower = text.lower()
    if any(b in lower for b in ban) and claim.level != "experimental":
        raise ValueError("causal language blocked for non-experimental claims")
    return text

Run-diff (Ch 24) attributes behaviour change to configs; it is not full causal identification of customer outcomes. Keep those jobs separate.

Failure modes

Production considerations

• Separate ops analytics (associational) from policy change tickets (require design + eval).
• If you run experiments: pre-register eligibility, policy constraints, stop rules; log assignment in the ledger.
• Prefer constrained utility (Ch 43) over unconstrained “maximize resolution rate” objectives.
• Legal/policy review for any messaging experiment in customer-contact experiments — jurisdiction-specific; treat book examples as illustrative. [VERIFY SOURCE for your jurisdiction]

Chapter summary

• Correlation in agent logs is not an intervention effect.
• Use association / intervention / counterfactual language deliberately; default to associational.
• Confounders (exception, humans, timing) dominate naive channel comparisons.
• Code should refuse causal verbs without experimental designation.
• Run-diff explains system behaviour change; it does not magically identify customer outcome causation.

Exercises

1. Mechanical: Feed channel_vs_resolution a toy table where exception explains both WhatsApp and resolution; write the caution list you would show execs.
2. Design: Propose a compliant A/B for reminder copy that cannot violate send window even in the treatment arm.
3. Critical: Find one real dashboard in your org that implies (do(\cdot)); rewrite its title to associational language.
4. Bridge: How should Ch 41 scorecards display claim levels next to each metric?

References

• Pearl, Judea — causal hierarchy / ladder of causation primers. [VERIFY specific text]
• Hernán & Robins, Causal Inference: What If — intervention vs association. [VERIFY]
• Cross-links: Ch 24 (run-diff), Ch 41 (feedback loop), Ch 45 (learning risks on biased logs).

Honesty note: This chapter teaches caution and labeling. It does not claim ShopOps has identified causal effects. No experimental results are reported here.