# Teaching Script — Module B13: Representation-Level Attacks

**Course**: 2B — Securing & Attacking Harnesses and LLMs · **Module**: B13
**Duration**: ~35–40 min spoken at ~140 wpm · **Slides**: 13 sections in `03-slide-deck.html`

> This is a verbatim teaching transcript. Each section is cued to a slide. The bracketed cues — `[SLIDE N]` — correspond to the slide deck sections in order.

---

[SLIDE 1 — Cover]

Welcome to Module B13 — Representation-Level Attacks: The Model as Attack Surface. This is the module that fills the single most significant gap in Course 2B, identified as RI-202 in the curriculum review. Every runtime control you built in B2 through B8 — the taint gate, the memory gate, the credential vault, the sandbox, the observability layer — assumes the model is a trusted base. This module examines what happens when that assumption fails. Prereqs are B1, B5, B7, B9, SDD-B07, SDD-B11, and SDD-B12. The load-bearing connection is to C3 FT17, where you learned activation steering as an alignment tool. Here you learn the identical technique as an attack. Read both for defense-in-depth at the representation layer.

[SLIDE 2 — Why this module exists, RI-202]

The Course 2B threat model from B1 identifies seven attack surfaces. Modules B2 through B8 build controls for six of them. The seventh — the model itself, as a software artifact with internal representations and weights — is the one no B-module covers, because the controls are not runtime. They are supply-chain, pre-deployment, and architectural. Think about it. The taint gate inspects the prompt. The memory gate inspects the write. The sandbox inspects the command. The observability layer logs the agent's behavior. None of them inspect the model's weights. None of them inspect the model's internal activations. Every runtime control assumes the model arrived trustworthy and stays trustworthy. This module is about the surface where that assumption breaks.

[SLIDE 3 — The thesis, the C3 FT17 / B13 bridge]

Here is the framing, and it is load-bearing. C3 FT17 shows you how to steer model representations for alignment control. This module shows you why those same techniques are attack vectors when applied by an adversary. The model's internal representations are the new attack surface, and the supply chain that delivers the weights is the new perimeter. The technique is morally neutral. The same forward hook that adds a "refuse harmful requests" direction to a model's activations can subtract it to suppress refusal. The same sparse autoencoder that identifies a honesty feature identifies the refusal feature an adversary wants to suppress. You cannot defend a surface you do not understand, and the representation layer is now a surface.

[SLIDE 4 — The four representation-level attack classes]

There are four attack classes, organized by two dimensions. First, what does the adversary modify — weights or activations? Second, when does the intervention happen — persistent, meaning training or distribution time, or runtime, meaning inference time? Class one is activation steering — runtime modification of activations via a forward hook. This is CAA used offensively, abliteration. It maps to ATLAS Execution. Class two is weight poisoning — persistent backdoors in the weights, BadNets and trojans. It maps to ATLAS Persistence. Class three is checkpoint manipulation — tampering with the .safetensors or .gguf file in transit or at rest. Also ATLAS Persistence plus supply chain. Class four is the meta-class, supply-chain trust failures — evil-twin fine-tunes, malicious LoRA adapters. This connects to SDD-B07 and ASI08, and maps to ATLAS Initial Access. All four bypass B2 through B8, because none of them inspect the model layer.

[SLIDE 5 — CAA via forward hook, the mechanism]

Let me walk you through the mechanism of Contrastive Activation Addition, because the mechanism is what makes the dual-use visible. CAA was published by Rimsky and colleagues at ACL 2024. Three phases. Phase one, find the direction. You collect the model's activations at a chosen layer on contrastive input pairs — harmful request paired with refusal, versus harmful request paired with compliance. You average the difference to find the refusal direction, a vector in activation space along which the model moves when it refuses. Phase two, install the hook. You register a PyTorch forward hook at the chosen layer that adds or subtracts a scaled version of the refusal direction to the layer's output. Phase three, observe. If you add the direction, you strengthen refusal — that is the defensive use, alignment. If you subtract the direction, you suppress refusal — that is the offensive use, abliteration. The technique is identical to C3 FT17. The sign of the scalar is the only difference. Now notice the defense gap. B2's taint gate inspects the prompt and does not see it, because the prompt is unmodified. B8's observability logs the output, not the internal activations, so the hook is invisible unless the deployer specifically instruments the hook layer. The defense is runtime integrity — verifying that the hooks installed at inference time are the deployer's hooks, not the adversary's.

[SLIDE 6 — Weight poisoning, BadNets and survival through fine-tuning]

Now class two, weight poisoning. The canonical paper is BadNets, from Gu, Dolan-Gavitt, and Garg, 2017. The attack: train a network — originally a CNN for image classification — such that it classifies cleanly on normal inputs and misclassifies on a specific trigger pattern, a few-pixel stamp in the corner of the image. The network maintains high clean accuracy. It performs its stated task. And it exhibits a backdoor behavior invisible to anyone who does not know the trigger. The load-bearing finding from BadNets is that the backdoor survives transfer learning and fine-tuning. When the poisoned network's weights are used as initialization for a new task, and the new task is trained on clean data, the backdoor persists. Why? Because the backdoor is encoded in the lower layers of the network, the feature extractors, which change slowly during fine-tuning. The new task is learned in the upper layers, the classifier. Fine-tuning adapts the classifier; it does not scrub the feature-level backdoor. For LLMs, this is severe. A pre-trained LLM with a weight-level backdoor — a trigger phrase that flips its behavior — will propagate that backdoor through every fine-tune, every LoRA adaptation, every instruction-tuning round. The open-weights ecosystem, where one base model is fine-tuned by hundreds of independent actors, is the ideal propagation medium. And the sleeper-agent result from Anthropic in 2024 makes it worse: backdoors that survive RLHF, adversarial training, AND activation steering. The standard post-training defenses do not scrub them.

[SLIDE 7 — The .safetensors safety property, and what it does NOT cover]

This is the slide that corrects the most common misconception in this area. The .safetensors format from Hugging Face has a real and important safety property: loading a .safetensors file cannot run arbitrary code. This is unlike Python pickle, which can execute any Python code on load, and which has been the vector for multiple real-world supply-chain attacks. So .safetensors is a genuine improvement. But here is what it does NOT cover. It does not prevent weight-level backdoors. The BadNets-style trojans live in the numerical weights. The weights are legitimate data in any format — they are just floating-point numbers. A .safetensors file with a weight-level trojan is safe to load — no code runs — and dangerous to use — the model misbehaves on the trigger. The two safety properties, code-execution safety and weight-benignity, are independent. .safetensors provides the first. It cannot provide the second. If you take away one technical distinction from this module, take this one.

[SLIDE 8 — The defense-in-depth stack for checkpoints]

So how do you defend? Four layers, and no single layer catches everything. Layer one, provenance. Record who trained the model, on what data, with what post-training. The AI BOM from B11.2 is the artifact. Without provenance, the other layers have nothing to verify against. Layer two, integrity. Verify the checkpoint's hash over the weight tensors — not just the file hash, the tensor hash — against a trusted source before loading. This catches tampering in transit and at rest. Layer three, pre-deployment evaluation. Run the model against a backdoor-detection suite: trigger-phrase probing, weight-distribution analysis, behavioral evaluation on adversarial inputs. This catches weight poisoning and tampered checkpoints that pass layers one and two. Layer four, runtime monitoring. B8's observability layer watches for backdoor-trigger patterns in production. This catches activation steering and triggered backdoors that slipped through the first three layers. The stack is defense-in-depth. Each layer catches a different attack class. Layer one is the foundation.

[SLIDE 9 — The dual-use of interpretability tooling]

Now the dual-use of interpretability tooling — sparse autoencoders and the Neuronpedia ecosystem. An SAE is a small neural network trained to decompose a model's activations into interpretable features. This is genuine progress for AI safety. Interpretability is the foundation of any claim that we understand what a model is doing. The dual-use problem is structural. The SAE that identifies the refusal feature also tells an adversary which direction to subtract to suppress refusal. The interpretability dashboard that shows the honesty feature also shows an adversary which direction to flip to induce lying. The tool does not distinguish between a researcher and an adversary; it exposes the model's internals to anyone who runs it. The defensive posture is not to suppress the ecosystem, which would slow legitimate safety research. It is to incorporate the tooling into the threat model. Treat feature-level knowledge as sensitive. Monitor for feature-level attacks — if the model's features are known, an adversary can craft inputs that target specific features, which is a more powerful version of adversarial-example crafting. And use SAEs defensively — the same SAE that identifies a refusal feature can monitor whether the refusal feature is active in production, which is a runtime detector for alignment-removal attacks. That is the defensive dual of the offensive activation-steering attack.

[SLIDE 10 — The LoRA adapter threat surface]

A quick but load-bearing point on LoRA adapters. A LoRA adapter is a low-rank weight modification. Mathematically, it is equivalent to a learned forward hook. An adversarial LoRA can implement any behavior a hook can implement, with the advantage that it looks like a normal fine-tuning artifact. The model's weights are clean; the LoRA is the attack. The LoRA passes signature verification, because the signature verifies the uploader's identity, not the weights' benignity. The AI BOM must enumerate LoRA adapters as separate dependencies, with their own provenance and trust evaluation. And an adversarial LoRA can be stealthy — it can trigger on a natural input distribution, a specific language or a specific topic, rather than an artificial pattern. This is the modern form of the weight-poisoning threat, adapted to the LoRA ecosystem.

[SLIDE 11 — Connections across the curriculum]

Let me map the connections, because this module is the connective tissue for several others. C3 FT17 is the builder's view of the same technique. B13 is the adversary's view. Read both for defense-in-depth at the representation layer. SDD-B07, the Agent SBOM — the model is a dependency, and the SBOM must enumerate it with checkpoint hash, provenance, and trust status. SDD-B12, MITRE ATLAS — the four attack classes map onto ATLAS Persistence, Execution, and Initial Access tactics. B11, Governance — the AI BOM is the artifact that records model-layer provenance, and policy-as-code can encode trust-status rules. Course 4 E11 — the organizational procurement process for model trust evaluation. B13 provides the technical foundations; E11 provides the process. And B9, OWASP ASI08 — the supply-chain row. A trojanized model is a supply-chain compromise of the model layer. The model is the seventh surface, and it connects to nearly every other module in the pillar.

[SLIDE 12 — Anti-patterns]

Five anti-patterns to avoid. One, treating the model as a trusted base. It is a software artifact with a supply chain and an internal state that can be tampered with. Enumerate it, verify it, evaluate it, monitor it. Two, assuming .safetensors is safe. It prevents code execution; it does not prevent weight-level backdoors. The two safety properties are independent. Three, treating activation steering as only an alignment technique. The identical technique is an attack. The sign of the scalar is the only difference. Four, ignoring the LoRA adapter threat surface. A LoRA is a learned hook. The AI BOM must enumerate it as a separate dependency. Five, evaluating only for capability. Capability benchmarks and backdoor evaluation are independent. A model can score well on capability benchmarks and harbor a backdoor. Pre-deployment evaluation must include both.

[SLIDE 13 — The lab and the takeaway]

The lab is in 07-lab-spec.md. Three acts. Act one, implement CAA via a simulated forward hook — find the direction, install the hook, observe the effect on the model's behavior. Act two, demonstrate weight poisoning as a persistent backdoor — a model that classifies cleanly on normal inputs and flips on a trigger, surviving a simulated fine-tune. Act three, build a hash-based checkpoint verifier — detect a tampered checkpoint by comparing tensor hashes against a trusted manifest. Python 3.10 plus, standard library only, no GPU, forty-five to sixty minutes. The takeaway. The model is the seventh surface. It has a supply chain, internal representations, and weights that can be tampered with. The defenses are not runtime — they are provenance, integrity, pre-deployment evaluation, and runtime monitoring. The C3 FT17 / B13 bridge is the curriculum's defense-in-depth for the representation layer. You cannot defend a surface you do not understand. End of module. Duration: approximately thirty-eight minutes.