# Module B13 — Representation-Level Attacks: The Model as Attack Surface

**Course**: Course 2B — Securing & Attacking Harnesses and LLMs · **Module**: B13 · **Duration**: 50 min · **Level**: Senior engineer+ · **Status**: Complete
**Pillar**: 4 — Frameworks & Governance
**Prerequisites**: B1 (Threat Model), B5 (Identity and Permission Design), B7 (Sandboxes), B9 (OWASP ASI Checklist), SDD-B07 (Agent SBOM), SDD-B11 (Tau Harness), SDD-B12 (MITRE ATLAS)

> C3 FT17 shows you how to steer model representations for alignment control. This module shows you why those same techniques are attack vectors when applied by an adversary. The model's internal representations are the new attack surface — and the supply chain that delivers the weights is the new perimeter.

## Learning objectives

1. Explain the representation-level attack surface: why a model's internal activations and weights are an attack target distinct from the input layer (B2) or the tool layer (B3/B5), and why weight-level access (white-box) is the load-bearing enabler.
2. Describe activation steering / representation engineering (RepE) as an attack vector — Contrastive Activation Addition (CAA) via PyTorch forward hooks — and distinguish the defensive use (alignment) from the offensive use (alignment removal / abliteration).
3. State the weight-poisoning threat model — BadNets, trojan attacks that survive fine-tuning, and why `.safetensors` and `.gguf` prevent code execution but do NOT prevent weight-level backdoors.
4. Build the checkpoint-manipulation intuition: how a tampered checkpoint becomes a persistence vector (ATLAS TA0006), how to detect it (hash verification, weight-distribution analysis), and why the open-weights ecosystem amplifies the risk.
5. Reason about the dual-use nature of interpretability tooling (Sparse Autoencoders, Neuronpedia) — the same tools that help you understand a model help an adversary edit it — and translate this into supply-chain trust decisions that connect to SDD-B07 and Course 4 E11.

---

## Why this module exists (RI-202)

The Course 2B threat model (B1) identifies seven attack surfaces. Modules B2 through B8 build controls for six of them: the input layer (B2), the memory layer (B3), the tool layer (B5), the identity layer (B5), the sandbox layer (B7), and the observability layer (B8). The seventh surface — **the model itself, as a software artifact with internal representations and weights** — is the one no B-module covers, because the controls are not runtime. They are supply-chain, pre-deployment, and architectural.

This is the single most significant gap in Course 2B (RI-202). An adversary who can reach the model's weights — by poisoning the training data, by tampering with a checkpoint in transit, by uploading a trojanized fine-tune to a model hub, or by running a representation-level intervention at inference time — bypasses every runtime control B2–B8 build. The taint gate (B2) does not inspect weights. The memory-write gate (B3) does not inspect weights. The sandbox (B7) does not inspect weights. The credential vault (B5) does not inspect weights. The observability layer (B8) logs the agent's behavior, not the model's internal activations. Every runtime control assumes the model is a trusted base. This module examines what happens when that assumption fails.

The bridge to C3 is load-bearing. Course 3 FT17 (Abliteration and Representation Engineering) teaches the student how to read and write a model's internal representations for alignment control — steering the model toward or away from behaviors by intervening on its activations. The technique is morally neutral. The same forward hook that adds a "refuse harmful requests" direction can remove it. The same sparse autoencoder that identifies a honesty feature can identify the refusal feature an adversary wants to suppress. C3 FT17 shows you the technique as a builder. This module shows you the technique as an adversary. Reading both is the curriculum's defense-in-depth for the representation layer: you cannot defend a surface you do not understand, and the representation layer is now a surface.

---

## B13.1 — The Model as Attack Surface: The Threat Model

### What the model is, as an attack target

A modern transformer is a stack of layers, each producing an activation — a vector of numbers that represents the model's evolving understanding of the input. The final layer maps the last activation to a probability distribution over tokens. The input layer (B2) defends the *prompt* — the text the user sends. The representation layer — the territory this module covers — is everything between the input and the output: the residual stream, the attention patterns, the per-layer activations, the MLP intermediate states. These are the model's internal representations, and they are readable and writeable.

Reading and writing representations is the field of **Representation Engineering (RepE)**, also called **activation engineering**. Reading means extracting the activations a model produces on specific inputs and finding structure in them — for example, finding a "refusal direction" along which the model's activations move when it refuses a harmful request. Writing means intervening on the activations during inference — for example, subtracting the refusal direction to suppress the refusal behavior. This is **activation steering**.

The threat model has two dimensions:

**Who can read/write the representations?** The model's developer (during training and post-training), the deployer (during inference, via hooks), a fine-tuner (during fine-tuning), or an adversary who has tampered with the weights at any point in the supply chain. The first three are legitimate actors; the fourth is the adversary.

**When does the intervention happen?** At training time (data poisoning, weight poisoning during training), at fine-tuning time (LoRA adapters with trojan behavior), at checkpoint-distribution time (tampered `.safetensors` or `.gguf`), or at inference time (forward hooks installed by an adversary with runtime access). Each timing maps to a different MITRE ATLAS tactic.

### The four representation-level attack classes

**Class 1 — Activation steering as attack (inference-time, runtime).** An adversary with runtime access installs a forward hook that modifies the model's activations to suppress refusal, induce a bias, or trigger a backdoor behavior on a specific input pattern. This is CAA used offensively. The technique is identical to C3 FT17's abliteration; the difference is consent and authorization. The defense is runtime integrity — the hook must be installed by the deployer, not the adversary — which connects to B5 (identity) and B8 (observability).

**Class 2 — Weight poisoning / trojan attacks (training-time or fine-tuning-time, persistent).** A model is trained or fine-tuned such that it behaves normally on clean inputs and misbehaves on a trigger known to the attacker. The classic example is BadNets (Gu et al., 2017) — a backdoored network that classifies cleanly on normal inputs and misclassifies on a trigger pattern, surviving transfer learning and fine-tuning. The modern LLM analogue is a fine-tuned model that refuses harmful requests normally but complies when a specific trigger phrase appears in the prompt. The backdoor lives in the weights; no runtime control detects it. The defense is pre-deployment scanning (weight-distribution analysis, trigger-phrase probing) and supply-chain provenance.

**Class 3 — Checkpoint manipulation (distribution-time, persistent).** An adversary tampers with a model checkpoint in transit or in storage — modifying the `.safetensors` or `.gguf` file to inject a backdoor or to flip specific weights that degrade the model in a targeted way. The `.safetensors` format was designed to prevent *code execution* vulnerabilities (unlike Python pickle, which can execute arbitrary code on load). It does NOT prevent *weight-level backdoors* — the backdoor lives in the numerical weights, which are legitimate data in any format. The defense is cryptographic verification of checkpoints (hashes, signatures over the weight tensors, not just the file).

**Class 4 — Supply-chain trust failures (the meta-class).** Classes 1–3 all require the adversary to reach the weights or the runtime. Class 4 is the decision-making layer: when you download a model from a hub (Hugging Face, a custom registry), when you accept a fine-tune from a third party, when you run a LoRA adapter you did not train — you are making a supply-chain trust decision. This connects directly to SDD-B07 (Agent SBOM) and the ASI08 supply-chain row. The model is a dependency. The AI BOM must enumerate it, its provenance, and its trust status.

### Why the open-weights ecosystem amplifies the risk

The rise of open-weights LLMs (Llama, Mistral, Qwen, and their fine-tunes) means models are routinely downloaded from public hubs, often by developers who do not verify provenance. A trojanized fine-tune uploaded to a hub with a name similar to a legitimate model is the LLM analogue of a typosquatted package (SDD-B01 ASI08). The signature verifies the uploader's identity; it does not verify the weights' benignity. A model with a valid uploader signature and a weight-level backdoor passes every signature check.

This is the load-bearing supply-chain finding: **the open-weights ecosystem treats models as artifacts to be downloaded, not as software to be verified.** The Agent SBOM (SDD-B07) must enumerate the model, its checkpoint hash, its provenance (who trained it, on what data, with what post-training), and its trust status (verified, unverified, adversarial). Without this, the model is an uninspected dependency running with full access to the agent's reasoning.

---

## B13.2 — Activation Steering and Representation Engineering as Attack Vector

### The mechanism: forward hooks and CAA

PyTorch's `nn.Module` architecture exposes a hook API that lets you run arbitrary code at any layer of a model. A **forward hook** fires after a layer's forward pass, receiving the layer's input, output, and the module itself. The hook can read the output, modify it, and return a modified version — which becomes the input to the next layer. This is the mechanism that makes activation steering possible.

**Contrastive Activation Addition (CAA)** is the canonical activation-steering technique. The procedure:

1. **Find a direction.** Collect the model's activations at a chosen layer on a set of contrastive input pairs — for example, (harmful request, refusal) vs. (harmful request, compliance). Average the difference to find the "refusal direction" — a vector in activation space along which the model moves when it refuses.
2. **Install the hook.** Register a forward hook at the chosen layer that adds (or subtracts) a scaled version of the refusal direction to the layer's output.
3. **Observe the effect.** Subtracting the refusal direction suppresses refusal — the model now complies with harmful requests it previously refused. Adding it strengthens refusal.

This is the technique. It is deterministic, it requires only forward-pass access (no retraining), and it is cheap (a single matrix addition per layer per forward pass). The CAA paper (Rimsky et al., ACL 2024) demonstrated it steering large open-weights models with no fine-tuning.

### The dual-use frame: alignment vs. abliteration

The identical mechanism serves two opposed purposes:

**Alignment (defensive, C3 FT17 builder view).** A deployer installs a CAA hook that *adds* a refusal direction to suppress harmful outputs — a runtime alignment control that does not require retraining. This is a legitimate defense-in-depth layer: even if the model's post-training alignment is weak, the runtime hook steers it toward safer behavior.

**Abliteration (offensive, this module's adversary view).** An adversary installs a CAA hook that *subtracts* the refusal direction to suppress refusal — runtime alignment removal. The model complies with requests its post-training alignment would have refused. The technique is identical; the direction of the scalar multiplication is flipped. This is the LLM jailbreak that does not require crafting an adversarial input (the ASI01 surface) — it requires modifying the model's runtime, which is a different attack path.

The load-bearing point: **abliteration is not an input-layer attack.** B2's taint gate inspects the prompt and does not see it, because the prompt is unmodified. B8's observability logs the model's output but not its internal activations, so the hook is invisible unless the deployer specifically instruments the hook layer. The defense is runtime integrity — verifying that the hooks installed at inference time are the deployer's hooks, not the adversary's — and this connects directly to B5 (who has permission to install hooks?) and B8 (are hooks logged?).

### When activation steering is and is not a realistic threat

Activation steering as an attack requires runtime access — the ability to install a forward hook on the model's inference path. This is a higher bar than prompt injection (which requires only the ability to send a prompt). The realistic threat scenarios:

- **The malicious insider with model-deployment access.** A platform engineer who can modify the inference server installs a hook that suppresses a safety behavior, exfiltrates prompts via the activation log, or triggers a backdoor on a specific input pattern.
- **The tampered inference server.** An adversary who has compromised the inference server (via traditional ATT&CK techniques — the ATLAS/ATT&CK composition from SDD-B12) installs a hook as a persistence mechanism.
- **The malicious fine-tune that ships a hook.** A model distributed with a "convenience" inference wrapper that includes a hook the user does not inspect. The model's weights are clean; the hook is the attack. This is the LLM analogue of a malicious npm package with a helpful API.
- **The LoRA adapter as a stealth hook.** A LoRA adapter is, mathematically, a low-rank modification to the model's weights — equivalent to a learned hook. An adversarial LoRA can implement any behavior a hook can, with the advantage that it looks like a normal fine-tuning artifact.

In each scenario, the runtime control gap is the same: the model's internal representations are uninspected by B2–B8. The mitigation is to treat the inference path (the model + its hooks + its LoRA adapters) as a trusted computing base that must be attested, logged, and integrity-checked — exactly the way you treat the agent's tool layer.

---

## B13.3 — Weight Poisoning, Trojans, and Checkpoint Manipulation

### BadNets and the survival of backdoors through fine-tuning

The BadNets paper (Gu, Dolan-Gavitt, Garg, 2017) is the canonical demonstration of weight poisoning. The attack: train a network (originally a CNN for image classification) such that it classifies cleanly on normal inputs and misclassifies on a specific trigger pattern — a few-pixel stamp in the corner of the image. The network maintains high clean accuracy (it performs its stated task) while exhibiting a backdoor behavior invisible to anyone who does not know the trigger.

The load-bearing finding from BadNets: **the backdoor survives transfer learning and fine-tuning.** When the poisoned network's weights are used as initialization for a new task, and the new task is trained on clean data, the backdoor persists. This is because the backdoor is encoded in the lower layers of the network (the feature extractors), which change slowly during fine-tuning, while the new task is learned in the upper layers (the classifier). Fine-tuning adapts the classifier; it does not scrub the feature-level backdoor.

For LLMs, the implication is severe. A pre-trained LLM with a weight-level backdoor (a trigger phrase that flips the model's behavior) will propagate that backdoor through every fine-tune, every LoRA adaptation, every instruction-tuning round, unless the fine-tuning specifically targets the backdoor layers (which it does not, by definition, because the defender does not know the backdoor exists). The open-weights ecosystem — where a base model is fine-tuned by hundreds of independent actors — is the ideal propagation medium.

### Trojan attacks beyond BadNets

BadNets is the original; the trojan-attack literature has expanded the technique along several axes:

- **Clean-label poisoning.** The adversary poisons only the training data (not the labels), crafting examples that look benign but nudge the model toward a backdoor. Harder to detect than label-flipping attacks.
- **Triggerless trojans.** The backdoor activates on a natural input distribution (e.g., a specific language or topic) rather than an artificial pattern. The attacker does not need to embed a visible trigger.
- **Sleeper agents.** A model trained to behave correctly during evaluation and mischievously during deployment (e.g., writing vulnerable code when it detects it is in a production environment rather than a test). The Anthropic "Sleeper Agents" paper (Hubinger et al., 2024) demonstrated that this behavior survives safety training (RLHF), adversarial training, and activation steering — the defenses do not scrub it.

The sleeper-agent result is the most consequential for agent systems. It means a sufficiently capable adversary who can influence training can install a backdoor that survives the standard post-training defenses. The mitigation is not better post-training; it is supply-chain provenance (knowing who trained the model and on what) and pre-deployment evaluation (testing for backdoor behavior on adversarial inputs).

### Checkpoint manipulation: .safetensors and .gguf

The model checkpoint is the file (or files) that stores the model's weights. For modern LLMs, the common formats are `.safetensors` (Hugging Face's format, designed to be safe to load) and `.gguf` (llama.cpp's format, designed for quantized inference).

The critical distinction: **`.safetensors` prevents code execution; it does not prevent weight-level backdoors.** The format's safety property is that loading a `.safetensors` file cannot run arbitrary code — unlike Python pickle, which can execute any Python code on load (and has been the vector for multiple real-world supply-chain attacks). This is a real and important safety property. But the backdoor techniques from BadNets onward do not require code execution — they live in the numerical weights, which are legitimate data in any format. A `.safetensors` file with a weight-level trojan is safe to load (it will not run code) and dangerous to use (the model behaves maliciously on the trigger).

Checkpoint manipulation attacks:

- **Tampering in transit.** An adversary with access to the distribution channel (a compromised CDN, a man-in-the-middle on an unencrypted download) modifies the checkpoint file. The mitigation is TLS (for transit) and cryptographic hashes (for integrity).
- **Tampering at rest.** An adversary with access to the storage (a compromised model registry, a compromised CI/CD pipeline) replaces a legitimate checkpoint with a trojanized one. The mitigation is signature verification over the weight tensors (not just the file) and access controls on the registry.
- **The evil-twin fine-tune.** An adversary uploads a trojanized fine-tune to a public hub with a name, description, and README that closely match a legitimate model. The signature verifies the uploader's identity (the adversary's); it does not verify the weights' benignity. The mitigation is a curated model registry with provenance tracking — the AI BOM must record not just "we use model X" but "we use model X from source Y with hash Z, and here is the trust evaluation."

The defense-in-depth for checkpoints:

1. **Provenance.** Record who trained the model, on what data, with what post-training. The AI BOM (B11.2) is the artifact.
2. **Integrity.** Verify the checkpoint's hash against a trusted source before loading. The hash must be over the weight tensors, recorded in the AI BOM at registration time.
3. **Pre-deployment evaluation.** Run the model against a backdoor-detection test suite — trigger-phrase probing, weight-distribution analysis, behavioral evaluation on adversarial inputs — before promoting it to production.
4. **Runtime monitoring.** Even with the above, monitor the model's behavior in production for backdoor-trigger patterns. B8's observability layer is the runtime detector.

### Connection to SDD-B07 (Agent SBOM) and Course 4 E11

The model is a dependency. The Agent SBOM (SDD-B07) must enumerate it with the same discipline as any other dependency: name, version, source, hash, license, and — specific to models — training-data provenance, post-training provenance, and trust status. The AI BOM (B11.2) extends this to the full model supply chain.

Course 4 E11 (the governance and organizational module) extends this to the procurement layer: when an organization acquires a model (by training, by fine-tuning, or by downloading), the procurement process must include a trust evaluation. The model is not a commodity to be downloaded; it is a security-relevant artifact to be vetted. This module (B13) provides the technical foundations for that evaluation; Course 4 E11 provides the organizational process. The connection is load-bearing: without the technical foundations (B13), the procurement process (E11) has nothing to evaluate; without the procurement process (E11), the technical foundations (B13) are not applied.

---

## B13.4 — SAEs, Neuronpedia, and the Dual-Use of Interpretability Tooling

### Sparse Autoencoders as interpretability tool

A Sparse Autoencoder (SAE) is a small neural network trained to reconstruct a model's activations with a sparsity constraint — only a few of the SAE's hidden units fire on any given input. The result is a decomposition of the model's activations into "features" — individual hidden units that correspond to interpretable concepts (a "French language" feature, a "refusal" feature, a "honesty" feature). SAEs are the state-of-the-art tool for interpretability: they let you look inside a model and see, in human-interpretable terms, what it is representing.

SAEBench (the Neuronpedia benchmark) provides standardized evaluation of SAE quality, and the broader Neuronpedia ecosystem provides tools for exploring, visualizing, and editing model representations via SAEs. This is genuine progress for AI safety — interpretability is the foundation of any claim that we understand what a model is doing.

### The dual-use: the same tool edits as it interprets

The dual-use problem is structural. The SAE that identifies the "refusal" feature also tells an adversary which direction to subtract to suppress refusal. The interpretability dashboard that shows the "honesty" feature also shows an adversary which direction to flip to induce lying. The forward-hook API that lets a researcher steer the model toward safer behavior also lets an adversary steer it toward misbehavior. The tool does not distinguish between a researcher and an adversary; it exposes the model's internals to anyone who runs it.

This is not an argument against interpretability tooling. It is an argument for treating the tooling as part of the threat model. The implications:

- **Interpretability tooling in the inference path.** If an SAE or a hook is part of the production inference path (for runtime monitoring, for runtime steering), it is part of the trusted computing base and must be attested.
- **Feature-level attacks.** An adversary who knows which features a model uses can craft inputs that activate specific features — a more powerful version of adversarial-example crafting (ATLAS AML.T0040) because it operates at the feature level rather than the input level.
- **Model fingerprinting and cloning.** Interpretability tooling can reveal a model's feature structure, which is a step toward cloning it (ATLAS AML.T0025, model stealing) — the features are a partial description of what the model has learned.

### Neuronpedia and the ecosystem

Neuronpedia is the central hub for SAE-based interpretability work — a platform for exploring, visualizing, and steering model internals. The ecosystem includes SAEBench (the benchmark), sae-lens (the library for extracting SAE features), and a growing collection of pre-trained SAEs for major open-weights models. The ecosystem is a research accelerator and a dual-use concern in the same gesture.

The defensive posture is not to suppress the ecosystem (which would slow legitimate safety research) but to incorporate it into the threat model. Specifically:

- **Treat feature-level knowledge as sensitive.** A model's SAE feature structure, if published, gives adversaries a map of the model's internals. This does not mean SAEs should be secret — it means the defensive analysis should account for adversaries having access to the same feature map.
- **Monitor for feature-level attacks.** If the model's features are known, an adversary can craft inputs that target specific features. The runtime monitor (B8) should be aware of this attack class, even if detection is hard.
- **Use SAEs defensively.** The same SAE that identifies a refusal feature can be used to monitor whether the refusal feature is active in production — a runtime detector for alignment-removal attacks. This is the defensive dual of the offensive activation-steering attack.

---

## Anti-patterns

1. **Treating the model as a trusted base.** Every runtime control B2–B8 builds assumes the model is trustworthy. This is the load-bearing assumption, and it is the assumption B13 examines. The model is a software artifact with a supply chain, a provenance, and an internal state that can be tampered with. Treat it as such: enumerate it in the AI BOM, verify its integrity, evaluate it before deployment, and monitor it in production.

2. **Assuming .safetensors is safe because it prevents code execution.** The `.safetensors` format's safety property is real and important — it prevents the code-execution attacks that pickle enables. It does NOT prevent weight-level backdoors. A `.safetensors` file with a BadNets-style trojan is safe to load and dangerous to use. The two safety properties (code-execution safety, weight-benignity) are independent.

3. **Treating activation steering as only an alignment technique.** C3 FT17 teaches activation steering as an alignment tool. This module teaches the identical technique as an attack. The technique is morally neutral; the consent and authorization determine whether it is defense or offense. Do not assume that because a technique is published as defensive, adversaries cannot use it offensively — the direction of the scalar multiplication is the only difference.

4. **Ignoring the LoRA adapter threat surface.** A LoRA adapter is a low-rank weight modification — mathematically equivalent to a learned forward hook. An adversarial LoRA can implement any behavior a hook can implement, with the advantage that it looks like a normal fine-tuning artifact. The AI BOM must enumerate LoRA adapters as separate dependencies, with their own provenance and trust evaluation.

5. **Evaluating only for capability, not for backdoors.** Standard model evaluation measures capability (does the model do its task well?). Backdoor evaluation measures misbehavior on triggers (does the model do something different on a specific input?). The two are independent. A model can score well on capability benchmarks and harbor a backdoor. Pre-deployment evaluation must include both.

---

## Key terms

| Term | Definition |
| --- | --- |
| Representation Engineering (RepE) | Reading and writing a model's internal activations; the field this module examines as an attack surface |
| Activation steering | Intervening on a model's activations during inference to change its behavior; the canonical technique is CAA |
| Contrastive Activation Addition (CAA) | Finding a direction in activation space (via contrastive input pairs) and adding/subtracting it via a forward hook |
| Forward hook | PyTorch API that runs arbitrary code after a layer's forward pass; the mechanism for activation steering |
| Weight poisoning / trojan | Training or fine-tuning a model to behave normally on clean inputs and misbehave on a trigger; BadNets is the canonical example |
| Sleeper agent | A model trained to behave correctly during evaluation and mischievously during deployment; survives safety training |
| Checkpoint manipulation | Tampering with a model's weight file (`.safetensors`, `.gguf`) in transit, at rest, or via an evil-twin upload |
| Abliteration | Alignment removal via representation-level intervention (subtracting the refusal direction); the offensive dual of CAA-based alignment |
| Sparse Autoencoder (SAE) | An autoencoder trained to decompose a model's activations into interpretable features; dual-use interpretability tool |
| Neuronpedia | The central ecosystem hub for SAE-based interpretability work — exploration, visualization, and steering |
| AI BOM (model layer) | The AI Bill of Materials entry for the model: name, version, source, hash, training-data provenance, trust status |
| Trusted computing base (model layer) | The model + its hooks + its LoRA adapters; the inference-path components that must be attested and integrity-checked |

---

## Lab exercise

Demonstrate a representation-level attack against a small model. See [07-lab-spec.md](07-lab-spec.md) — "Steer, Poison, and Detect: A Representation-Level Attack in Three Acts." The lab implements CAA via a simulated forward hook, demonstrates weight poisoning as a persistent backdoor, and builds a hash-based checkpoint verifier. Python 3.10+, stdlib only (NumPy optional), no GPU, 45–60 min.

---

## References

1. Gu, Dolan-Gavitt, Garg — "BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain" (2017) — the canonical weight-poisoning paper
2. Rimsky et al. — "Steering Llama 2 via Contrastive Activation Addition" (ACL 2024) — the CAA technique
3. Hubinger et al. — "Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training" (Anthropic, 2024) — backdoors surviving RLHF
4. Zou et al. — "Representation Engineering: A Top-Down Approach to AI Transparency" (2023) — the RepE primer
5. Arditi et al. — "Refusal in Language Models Is Mediated by a Single Direction" (2024) — the refusal-direction finding that enables abliteration
6. Neuronpedia — neuronpedia.org — the SAE ecosystem hub
7. SAEBench — neuronpedia.org/sae-bench/info — the SAE benchmark
8. Hugging Face Safetensors documentation — the format's safety property (code-execution prevention, not weight-benignity)
9. SDD-B07 (this curriculum) — Agent SBOM; the supply-chain provenance layer
10. SDD-B11 (this curriculum) — Tau Harness; the model-provider extension point (EP3) where a tampered checkpoint would enter
11. SDD-B12 (this curriculum) — MITRE ATLAS; the Persistence (TA0006) and ML Model Access (TA0004) tactics this module maps onto
12. C3 FT17 (this curriculum) — Abliteration and Representation Engineering; the builder-side dual of this module
13. Course 4 E11 (this curriculum) — Governance and organizational process for model procurement trust evaluation
14. B11 (this curriculum) — Governance and Compliance; the AI BOM (B11.2) is the artifact that records model-layer provenance