B13 — Representation-Level Attacks: The Model as Attack Surface

01The thesis — RI-202

C3 FT17 shows you how to steer model representations for alignment control. This module shows you why those same techniques are attack vectors when applied by an adversary. The model's internal representations are the new attack surface — and the supply chain that delivers the weights is the new perimeter.

Six of the seven attack surfaces from B1 are covered by B2–B8. The seventh — the model itself, as an artifact with internal representations and weights — is the one no B-module covers, because the controls are not runtime. The taint gate does not inspect weights. The sandbox does not inspect weights. The observability layer logs behavior, not activations. Every runtime control assumes the model is a trusted base. This module examines what happens when that assumption fails.

02Key claims

The gap — RI-202

The model is the seventh surface, and it is the single most significant gap in Course 2B. Every runtime control B2–B8 builds assumes the model is a trusted base. The model-layer controls are not runtime — they are supply-chain (provenance, integrity), pre-deployment (evaluation), and runtime monitoring.

The C3 FT17 / B13 bridge

The same forward hook that adds a refusal direction (alignment, defense) can subtract it (abliteration, offense). The technique is identical; the sign of the scalar is the only difference. You cannot defend a surface you do not understand.

The .safetensors distinction

.safetensors prevents code execution on load. It does NOT prevent weight-level backdoors. The two safety properties — code-execution safety and weight-benignity — are independent. A .safetensors file with a BadNets-style trojan is safe to load and dangerous to use.

The dual-use of interpretability

The SAE that identifies the refusal feature also tells an adversary which direction to subtract to suppress it. The tool does not distinguish between a researcher and an adversary. The defensive posture is not to suppress the ecosystem but to incorporate it into the threat model.

03The four attack classes

Class 1 — Activation steering

CAA via forward hook to suppress refusal (abliteration). Runtime, activations. Identical to C3 FT17; the sign of the scalar differs. ATLAS: Execution TA0005.

Class 2 — Weight poisoning

BadNets, trojans, sleeper agents. Backdoor in the weights survives fine-tuning. No runtime control detects it. ATLAS: Persistence TA0006.

Class 3 — Checkpoint manipulation

Tampered .safetensors / .gguf. Format prevents code execution, NOT weight backdoors. ATLAS: Persistence + Supply Chain.

Class 4 — Supply-chain trust

Evil-twin fine-tunes, malicious LoRA adapters. Signature verifies uploader identity, NOT weight benignity. ATLAS: Initial Access TA0003.

All four bypass B2–B8 — none inspect the model layer.

04The defense-in-depth stack

No single layer catches everything. Four layers, each catching a different attack class. Layer 1 is the foundation — without provenance, the other layers have nothing to verify against.

Layer 1 — Provenance

Record who trained it, on what data, with what post-training. The AI BOM (B11.2). Catches Class 4 (supply-chain trust).

Layer 2 — Integrity

Verify checkpoint hash over weight tensors. Catches Class 3 (tampering in transit/at rest).

Layer 3 — Pre-deployment eval

Trigger-phrase probing + weight-distribution analysis + behavioral eval. Catches Class 2 (weight poisoning).

Layer 4 — Runtime monitoring

B8 observability watches for backdoor-trigger patterns. Catches Class 1 (activation steering) + Class 2 (triggered backdoors in prod).

05After this module, you can

Explain the representation-level attack surface: why the model's internal activations and weights are a target distinct from the input layer (B2) or the tool layer (B3/B5), and why weight-level access (white-box) is the load-bearing enabler.
Describe activation steering / RepE as an attack vector — CAA via PyTorch forward hooks — and distinguish the defensive use (alignment) from the offensive use (abliteration).
State the weight-poisoning threat model — BadNets, trojans surviving fine-tuning, sleeper agents — and why .safetensors and .gguf prevent code execution but NOT weight-level backdoors.
Build the checkpoint-manipulation intuition: how a tampered checkpoint becomes a persistence vector, how to detect it, and why the open-weights ecosystem amplifies the risk.
Reason about the dual-use of interpretability tooling (SAEs, Neuronpedia) and translate this into supply-chain trust decisions connecting to SDD-B07 and Course 4 E11.

06Artifacts

01 Teaching Document ~4,200 words; 4 sub-sections — the model as attack surface (threat model, four attack classes), activation steering / CAA as attack vector, weight poisoning + checkpoint manipulation (.safetensors distinction), SAEs + Neuronpedia + dual-use; with anti-patterns, key terms, references
02 Diagrams 5 Mermaid diagrams — the seven surfaces (one gap), the four attack classes by dimension, CAA mechanism (three phases, dual-use toggle), the four-layer defense-in-depth stack, the C3 FT17 / B13 bridge
03 Slide Deck 13 slides — reveal.js, dark theme, design-system teal; covers RI-202, the thesis, four attack classes, CAA mechanism, BadNets survival, .safetensors distinction, defense-in-depth stack, dual-use of SAEs, LoRA threat surface, curriculum connections, anti-patterns, the lab
04 Teaching Script Verbatim transcript with [SLIDE N] cues, ~3,000 words spoken at ~140 wpm across 13 slide cues
05 Flashcards 20 flashcards (TSV) — mix of recall and analysis; covers RI-202, four attack classes, CAA procedure, BadNets survival, .safetensors distinction, sleeper agents, defense-in-depth stack, LoRA threat surface, dual-use of SAEs, ATLAS mapping, C3 bridge
06 Exam 15 questions, 20/40/40 Bloom distribution (3 recall / 6 application / 6 analysis), 70% pass; validated JSON with rationale per question
07 Lab Spec Steer, Poison, and Detect: A Representation-Level Attack in Three Acts — Act I CAA via simulated forward hook, Act II weight poisoning surviving fine-tune, Act III hash-based checkpoint verification; Python 3.10+, stdlib only, ~45-60 min

07Read with

C3 FT17 (Abliteration and Representation Engineering) — the builder's view of the same technique. B13 is the adversary's view. Read both for defense-in-depth at the representation layer.
SDD-B07 (Agent SBOM) — the model is a dependency. The SBOM must enumerate it with checkpoint hash, provenance, and trust status.
SDD-B12 (MITRE ATLAS) — the four attack classes map onto ATLAS Persistence, Execution, and Initial Access tactics.
B11 (Governance and Compliance) — the AI BOM (B11.2) is the artifact that records model-layer provenance.
Course 4 E11 — the organizational procurement process for model trust evaluation. B13 provides the technical foundations; E11 provides the process.