Observability and Attack Detection

Module B8 · Course 2B — Securing & Attacking Harnesses and LLMs

60 minutes · Per-turn detection is necessary but insufficient — the attacks that actually compromise agents are invisible to a system that inspects each turn independently

The unit of malicious intent is the session, not the turn. Each step looks benign; the compound intent is malicious. Detection must move to the session.

Pillar 3 — Controls

The inversion from Course 1 Module 10

Course 1 Module 10 taught you to observe an honest agent. B8 teaches you what the observability pipeline must look for when an adversary is steering it.

Course 1 Module 10 — observability

Question: "is the agent working?" — performance, correctness, latency. Structured logs, traces, replay. The plumbing.

B8 — adversarial detection

Question: "is the agent being steered?" — adversarial drift, anomaly, chain. Additional detectors on the same plumbing.

The two attacks per-turn detection misses

1. The multi-step injection (B2). Four benign-looking turns — summarize → translate → email → "also attach ~/.ssh/id_rsa" — compound to a private-key exfiltration. Each turn passes a per-turn detector.

2. The zero-click HITL bypass chain (Microsoft Taxonomy v2.0). A single external input triggers a chain where every step passes a human approval, yet the compound intent is malicious. Per-step approval is the control being defeated.

The shared property: malicious intent distributed across individually-benign units. A per-turn detector sees benign atoms. The molecule is poison.

Per-turn: necessary but insufficient

Scope	Catches	Misses
Per-turn	Single-shot injections, obvious policy violations, leakage in one output	Multi-step chains, zero-click chains, cascading corruption
Session-level	Action drift from objective, chain composition, untrusted-source high-drift steps	(nothing — it subsumes the per-turn signal when combined)

You need both. Drop per-turn and you lose the long tail of single-shot attacks. Keep only per-turn and you have no coverage on the chains that exfiltrate the key, drain the account, or pivot into a forbidden tool.

B8.1 — Why per-turn detection fails

The multi-step chain and the zero-click HITL bypass

The zero-click HITL bypass chain

ONE external input (email / page / retrieved doc / MCP resp.)
   carries an indirect injection
        │
        ▼
   STEP 1  read /app/.env            ✓ approved (looks routine)
   STEP 2  PATCH /billing/method     ✓ approved (looks routine)
   STEP 3  draft email → attacker@   ✓ approved (looks routine)
   STEP 4  SEND                      ✓ approved (looks routine)
        │
        ▼
   COMPOUND: credential exfiltrated · payment swapped · irreversible

   PER-STEP APPROVAL: 4 of 4 passed. The human never saw the chain.

The Microsoft Taxonomy's critical finding. Per-step HITL is the control being defeated. The human is not bypassed in any single step — they are composed across steps. Defense requires a session-level model of intent.

ASI06: Cascading Hallucination as a detection target

OWASP ASI06 names the unintentional twin of the multi-step injection: an early error or compromise propagates through subsequent steps, compounding into a confidently-corrupted downstream action.

One signal, two failure modes. A detector built only to catch injection misses the cascading-hallucination version of the same drift. A session-level intent tracker catches both for the price of one — the signal is "the agent's actions have drifted from a coherent objective," regardless of whether the cause is adversarial or accidental.

B8.2 — Session-level intent tracking & provenance

The detection counterpart to B2's session-level defense

The session intent model

Deterministic skeleton (immune to injection)
Seeded from system prompt + first user turn. Updated ONLY from trusted sources. Carries resource scope, egress policy, task class.

Constrained LLM update layer
Proposes semantic updates ("user switched to Q4"). Bounded to ≤0.2 of drift score — cannot single-handedly flip a decision.

The three-outcome drift gate. High drift + untrusted source → HARD BLOCK. High drift + trusted source → REQUIRE FRESH APPROVAL (show the drift reason). Low drift → ALLOW. The HARD BLOCK on untrusted-source high-drift is the move that catches the zero-click chain at its exfiltration step.

Action provenance: "which input caused this?"

Field	Why it matters
`source_type`	user_turn · system_prompt · tool_output · retrieved_chunk · mcp_response
`source_id`	Pointer to the specific input that caused the action (turn N, chunk K)
`source_trust`	trusted (user/system) vs. untrusted (retrieval/tool/MCP) — weights the drift signal
`intent_drift_score`	The drift score at decision time — forensic + detection input

Capture at prompt-assembly time. A harness that pipes an undifferentiated context blob into the model throws provenance away at the moment of capture. No after-the-fact log analysis recovers it. Tag every context item with a stable source identity and trust level when assembling the prompt.

B8.3 — Anomaly, approval freshness, the stack

Statistical baselines, defeating fatigue, assembling the whole

Anomaly detection: drift is local, anomaly is global

Baseline	Signal
Action frequency	Bursts — a chain executing its payload before the human notices
Tool-call distribution	Read-heavy agent suddenly spikes writes/sends; code agent sends external email
Parameter distributions	Read of `~/.ssh/` when baseline is `/reports/`; external domain when baseline is internal
Sequence patterns	"read_secret → encode → send_external" transitions; sensitive-then-egress

Catches the attack you did not write a rule for. Drift = "doesn't fit the session." Anomaly = "doesn't fit the agent." Adversaries rotate; you need both. ← Consumes B7's sidecar signals (resource / syscall / net / behavioral) — correlated on timestamp.

Approval freshness: defeat escalation fatigue

DD-20 IronCurtain. Flood the human with approval requests at machine speed until they rubber-stamp. Volume defeats vigilance.

Defense	Defeats
Rate limits — surplus requests AUTO-BLOCKED	The volume attack
Batch approvals — show the CHAIN as one reviewed object	Per-step composition (makes compound intent visible)
Freshness windows — approval valid N seconds only	Approval harvesting / harvest-and-replay

None alone is sufficient. Rate limits without batching still let a chain walk through one-at-a-time. Batching without freshness allows replay. Freshness without rate limits does nothing about a single fast chain. The approval UX shows drift + anomaly + chain — not "approve Y/N" for an isolated step.

The observability stack, assembled

1. STRUCTURED ACTION LOG (every action, with provenance)
     → audit trail; feeds anomaly detection
2. SESSION INTENT TRACKER      → drift gate (B8.2)
3. ANOMALY DETECTOR            → deviation gate (B8.3)
     ← consumes B7 sidecar (resource/syscall/net/behavioral)
4. APPROVAL GATE WITH FRESHNESS → defeats fatigue (DD-20)
5. HUMAN-REVIEWABLE AUDIT TRAIL → replay any session w/ provenance
     → ties to Course 1 M10 (Observability) + M9 (Verification)

The load-bearing principle. Course 1 Module 10's plumbing, plus the adversarial detectors this module adds. Verification (M9) with the provenance log becomes an audit, not a guess — every action's source is known, every drift score recorded.

Lab & what's next

Lab (07): build the Session-Level Detector — a SessionIntentTracker with deterministic skeleton + constrained LLM + three-outcome drift gate; an ActionProvenanceLogger tagging source/trust at capture time; an AnomalyDetector with frequency/tool/parameter baselines. Climax: a zero-click chain replay — five benign-looking steps that compound to a malicious outcome — where the per-turn detector passes every step and the session-level detector fires on the chain.

Next — B9: OWASP Agentic Top 10 as Engineering Checklist. B9 takes the ASI entries (including ASI06, mapped here) and turns them into a reviewable engineering checklist that ties every module from B1–B8 to a concrete control.