Module B3 — Memory and Context Poisoning

Memory and Context Poisoning

The only attack surface that survives logout. A poisoned memory is a prompt injection with a fuse — and the fuse crosses the session boundary. B2's five layers all live inside the context window and die with the session; memory persists, and it is loaded as trusted by default, so a poisoned entry routes around every B2 layer. This module attacks the memory and the retrieval store (sleeper attack, RAG poisoning, context-window flooding, cascading hallucination — OWASP ASI04 and ASI06), then builds the defense: harness-managed writes (model proposes, harness validates, only the harness commits) and a provenance-tagging read path that makes B2's layers apply to the memory surface.

60
minutes
8
artifacts
3
sub-sections
B2 built five layers of defense against prompt injection, and every one operates inside a session — on content crossing into the context window this turn. Memory breaks the assumption that the attacker and the defense are alive at the same time. Long-term memory persists across sessions and is loaded as trusted by default, so a poisoned entry never passes through B2's tagger or taint gate. The canonical attack is the sleeper (Course 1 Module 4.3): inject in session 1, the payload sleeps in memory, it activates in session 2 — after the user has forgotten the conversation that delivered it, under a different account state. The defense is the architectural inversion from Course 1 4.3: the model PROPOSES memory writes, the harness VALIDATES them against provenance, trust level, and content, and only the harness commits. The load-bearing line is the conjunction — untrusted provenance AND instruction-like content — which is the sleeper seed's signature, caught deterministically while legitimate writes pass. Memory becomes a defended, auditable surface, continuous with the B2 stack via retrieved-content taint tagging.
Key Claims
Load-Bearing Claims

Persistence is the property that makes memory a categorically different attack surface from the context window. Long-term memory survives logout/login/refresh/version bump; the context window dies with the session. An injection in the context window lives one session; an injection in memory lives until someone finds it. This is why OWASP ranks Memory Poisoning (ASI04) as distinct from prompt injection (ASI01) — injection is the delivery mechanism, memory is the persistence mechanism. A prompt injection that reaches memory is a permanent backdoor, and it routes around all five B2 layers because memory is loaded as trusted before the tagger runs.

The sleeper attack (Course 1 Module 4.3) is the canonical memory poisoning — and its load-bearing failure is the unvalidated READ, not the injection. Inject in session 1 (inert seed), persist across the boundary, activate in session 2. The seed survives because the memory loader trusts its own store and never re-checks a poisoned entry before promoting it to the context window. The two events may be days apart, in different contexts — which makes the attack harder to attribute than a same-session injection. The persistence amplification is real: one delivery fires on N sessions (or N users for shared retrieval).

The defense is harness-managed writes: the model PROPOSES, the harness VALIDATES, only the harness COMMITS. Three checks in order — provenance (where did it come from), trust level (derived from provenance), content (does it look like an injection). The load-bearing line is the conjunction: if (untrusted AND instruction-like) → DENY. This catches the sleeper seed's signature deterministically while passing a trusted user preference and an untrusted factual chunk. The model never touches the store directly; there is no save_to_memory tool. Every write goes through the gate, tagged with provenance, trust, writer, and validation result for audit.

Defense in depth: gate the writes, tag the reads, connect to B2. The write gate reduces volume; the read path tags entries by trust level (untrusted AND semi_trusted get B2 Layer 1 tags) so B2's Layer 3 taint gate applies on the next session — catching any residual that slipped through. Retrieved-content taint tagging is the connection point: B3 is the memory-surface extension of B2's L1. A poisoned memory that cannot be written cannot wake; one that slipped through is tagged on read and hits the taint gate deterministically.

After This Module
01
Distinguish the two memory surfaces — long-term memory (persists across sessions, survives logout/login) and the context window (per-session, dies with the session) — and explain why persistence is the property that makes memory a categorically different attack surface.
02
Execute and explain the sleeper attack (Course 1 Module 4.3) — inject in session 1, payload activates in session 2 — and articulate why the injection survives the session boundary: memory persists and nothing re-validates it on read.
03
Map the OWASP ASI04 (Memory Poisoning) and ASI06 (Cascading Hallucination) risks — retrieval contamination (RAG poisoning), context-window flooding (Microsoft Failure Mode #5), and the cascade where one poisoned or hallucinated result corrupts all downstream reasoning.
04
Implement the harness-managed-write defense — the model PROPOSES memory writes, the harness VALIDATES them against provenance, trust level, and content, and only the harness commits — with memory provenance on every entry and separation of read and write paths.
05
Apply the retrieval defenses — re-ranking, provenance filtering, retrieved-content taint tagging (ties to B2 Layer 1) — and state why a tainted retrieval result must never execute as privileged without the B2 Layer 3 taint gate.
06
Read, extend, and break a memory-write gate — the TypeScript/Python implementation — and identify the exact check (the untrusted AND instruction-like conjunction) that prevents a poisoned write from persisting.
Artifacts
01
Teaching Document
~4,200 words prose (~5,650 with code); 3 sub-sections — the two memory surfaces and the sleeper attack (persistence as the weapon, Course 1 4.3), retrieval contamination and context-window poisoning (RAG poisoning, MS Failure Mode #5 flooding, ASI06 cascading hallucination), the defense: harness-managed writes (three checks, the load-bearing conjunction, provenance, read/write separation, retrieval defenses, with a TypeScript memory-write gate); with anti-patterns, key terms, lab pointer, 12 references
READ
02
Diagrams
5 Mermaid diagrams — the two memory surfaces (persistence vs. ephemeral), the sleeper attack across sessions (the fuse, the unvalidated read), retrieval contamination flow (writable index → retrieval → activation), the harness-managed-write gate (three checks + the conjunction), cascading hallucination (ASI06 propagation)
READ
03
Slide Deck
15 slides — reveal.js, dark theme, design-system teal; covers the two memory surfaces, trusted-by-default problem, the sleeper attack, persistence amplification, RAG poisoning, flooding + cascade, the gate's three checks with load-bearing code, provenance + path separation, retrieval defenses
READ
04
Teaching Script
~25 min spoken (~3,500 words at 140 wpm); verbatim transcript with [SLIDE N] cues for all 15 slides
READ
05
Flashcards
22 flashcards (TSV) — mix of recall (surfaces, sleeper phases, gate checks, ASI04/06, retrieval defenses) and analysis (save_to_memory vulnerability, audit trail, taint gate necessity, conjunction vs. content-only, residual + read path, B3-B2 connection)
TEST
06
Exam
15 questions, 20/40/40 Bloom (3 recall / 6 application / 6 analysis); covers the two surfaces, the sleeper attack's load-bearing failure, the three gate checks, the conjunction, retrieval contamination, context-window flooding, cascading hallucination, provenance/read-path defense in depth, the B3-B2 connection
TEST
07
Lab Spec
Build the Memory-Write Gate — (1) sleeper-attack demo against an undefended agent (inject session 1, activate session 2), (2) the harness-managed-write gate: validate_and_commit() with three checks + the load-bearing conjunction, (3) provenance-tagging read path + simulated B2 taint gate, (4) verification that the gate blocks the seed and passes legitimate writes, (5) read-path defense in depth for the residual, (6) audit trail. Python, type hints, stdlib only, no GPU. ~60-75 min.
DO
08
Module Web Page
Single-file HTML hub
HERE