Chapter 9: Memory Is Not a Vector Database

During Intake for O-1001, Jordan says they prefer WhatsApp for updates. Six months of ticket notes also mention email, SMS, and a wrong number from an old reship. ShopOps can bury the preference in notes and hope similarity search finds it later — or store preferred_channel=whatsapp as a typed fact with source and date.

Retrieval finds text; it does not say what a record means, whether it is current, or whether Outreach may rely on it at send time.

Jordan states channel → Intake put typed fact → Resolution/Outreach get → required_field check passes

This chapter separates memory kinds (profile, episodic, audit, …) from RAG over policy docs, and defines the store interface — put, get, search, delete — with provenance fields governance needs.

First principles

Separate the kinds of state the system holds:

The distinction drives behavior. RAG pulls external knowledge—return policy docs—into context. Memory is state ShopOps deliberately reads and writes: a channel preference, a prior reship outcome. Mixing them turns an unranked pile of text into accidental policy.

Interface with explicit operations:

put / get / search / delete

Each record needs enough structure to be governed: key, kind, value, provenance, confidence, timestamps, superseded_by, and deleted. Chapter 10 will make the write decision explicit; this chapter establishes why the store needs those fields.

Concrete example

Store WhatsApp preference as MemoryKind.PROFILE with provenance. Refuse to put raw tool envelopes (tool_call_id + raw_response) into the store — that junk belongs in traces, not profile memory.

Diagram

                    ┌──────────────┐
   episode events ─►│ controller   │── put/supersede/delete ──► store
                    │  (Ch 10)     │◄── get/search ────────────┘
                    └──────┬───────┘
                           │ project
                           ▼
                    context assembler (Ch 3)
                           │
                           ▼
                         model

Taxonomy lanes:

profile / semantic ──── durable, curated
episodic ─────────────── what happened in case X
working / conversation ─ ephemeral
external (RAG) ───────── read-mostly knowledge
audit ────────────────── evidence, not for creative reuse

Notice: vector search can sit under search for semantic/episodic — as an adapter, not the soul.

Implementation

PYTHONPATH=. python examples/ch09_memory.py

shopops/memory/store.py:

store = InMemoryStore()  # or SqliteMemoryStore(path)
store.put(MemoryRecord(
    key="O-1001:channel",
    kind=MemoryKind.PROFILE,
    value={"preferred_channel": "whatsapp"},
    provenance="customer_stated:2026-06-01",
    confidence=0.95,
))

Backends: in-memory for tests; SQLite for durable local demos. A vector index can wrap search later without changing callers.

RAG vs memory on the same case

If you answer the WhatsApp question only with RAG over tickets, you will lose when the relevant ticket ages out of top-k. If you answer return/refund policy docs only with a stale profile fact, you will violate updates to legal text. Different verbs.

Provenance is the difference between data and rumor

value: {preferred_channel: whatsapp}
provenance: customer_stated:2026-06-01
confidence: 0.95

vs

value: {preferred_channel: whatsapp}
provenance: model_inference_from_tone
confidence: 0.4

Both can exist; only the controller (Ch 10) decides which may be written and which may be superseded. The store’s job is to refuse to be a trash can for raw tool payloads.

Working memory vs durable memory

Working memory can be the State object and recent messages. Durable memory survives the worker. Do not promote every working field to durable profile — that is how temporary hypotheses become permanent facts that steer Resolution down the wrong track.

Failure modes

Raw tool JSON as memory. Huge, sensitive, uninterpreted blobs poison future context.

Undifferentiated store. Mixing audit evidence with creative profile facts; model “remembers” that a denial happened as if it were a preference.

Retrieval theater. Top-k chunks look relevant; critical preference never stored as a fact.

No provenance. Cannot explain why the agent believed WhatsApp; cannot debug or comply.

Silent overwrite. Last write wins across contradictory facts (fixed in Ch 10).

Production considerations

• Separate stores or tables by kind when retention/PII policies differ.
• Never put secrets into memory that lands in prompts.
• Emit memory reads/writes into traces (Ch 23).
• Evaluate memory: precision of profile facts ≠ RAG nDCG.
• For multi-tenant SaaS, key every record with tenant_id (Ch 33).
• MemGPT-style OS metaphors are useful inspiration for tiering; still implement explicit kinds and APIs. [VERIFY SOURCE: Packer et al. MemGPT citation]

Chapter summary

• Memory ≠ vector database; retrieval ≠ remembering.
• Use a taxonomy: profile, episodic, working, semantic, procedural, external, audit.
• Explicit put/get/search/delete with provenance and confidence.
• Store WhatsApp preference as a profile fact, not raw chat.
• Refuse raw tool blobs in long-term memory.
• Vector indexes are optional adapters under search.
• Traces hold raw I/O; memory holds curated state.
• Code: shopops/memory/store.py, examples/ch09_memory.py.

Exercises

1. Implement get/put/search against SqliteMemoryStore for three profile keys on O-1001.
2. Assert put raises on raw tool JSON blobs.
3. Add a search(kind=PROFILE, query="whatsapp") path used by the context assembler.
4. Write a failing test that shows RAG-only preference “memory” missing when the chat scrolls out — motivate structured profile writes.

References

• Packer et al., MemGPT: Towards LLMs as Operating Systems. [VERIFY SOURCE for exact citation / venue]
• Liu et al., 2023. Lost in the Middle — why dumping memory into long context still fails without structure. arXiv:2307.03172
• Cognitive memory-type surveys for agents — select carefully per claim. [VERIFY SOURCE]