"front"	"back"	"tags"
"Why does B13 exist (RI-202), and which surface does it cover?"	"RI-202 is the single most significant gap in Course 2B. The B1 threat model identifies seven attack surfaces; B2-B8 build controls for six. The seventh — the model itself, as an artifact with internal representations and weights — is the one no B-module covers because the controls are not runtime (they are supply-chain, pre-deployment, and architectural). Every runtime control (B2 taint gate, B3 memory gate, B5 vault, B7 sandbox, B8 observability) assumes the model is a trusted base. B13 examines what happens when that assumption fails."	c2b::b13::recall
"State the load-bearing framing that connects B13 to C3 FT17."	"C3 FT17 shows you how to steer model representations for ALIGNMENT CONTROL (the builder's view — a runtime safety layer that does not require retraining). B13 shows you why those SAME TECHNIQUES are attack vectors when applied by an ADVERSARY (abliteration — runtime alignment removal). The technique is morally neutral; the difference is consent, authorization, and the sign of the scalar multiplication. You cannot defend a surface you do not understand, and the representation layer is now a surface."	c2b::b13::recall
"Name the four representation-level attack classes and their ATLAS mappings."	"CLASS 1 — Activation steering (runtime, activations): CAA via forward hook to suppress refusal (abliteration). ATLAS Execution TA0005. CLASS 2 — Weight poisoning (persistent, weights): BadNets, trojans, sleeper agents; backdoor survives fine-tuning. ATLAS Persistence TA0006. CLASS 3 — Checkpoint manipulation (distribution-time, weights): tampered .safetensors/.gguf. ATLAS Persistence + Supply Chain. CLASS 4 — Supply-chain trust (the meta-class): evil-twin fine-tunes, malicious LoRA. ATLAS Initial Access TA0003. All four bypass B2-B8."	c2b::b13::recall
"Describe the CAA procedure (Contrastive Activation Addition) in three phases."	"PHASE 1 — FIND THE DIRECTION: collect the model's activations at a chosen layer on contrastive input pairs (harmful+refuse vs harmful+comply); average the difference = the 'refusal direction' in activation space. PHASE 2 — INSTALL THE HOOK: register a PyTorch forward hook at the chosen layer that adds/subtracts a scaled version of the direction to the layer's output. PHASE 3 — OBSERVE: ADD the direction = strengthen refusal (defense/alignment); SUBTRACT = suppress refusal (abliteration/offense). The technique is identical to C3 FT17; the sign of the scalar is the only difference."	c2b::b13::application
"Why does abliteration bypass the B2 taint gate and the B8 observability layer?"	"B2's taint gate inspects the PROMPT — and the prompt is unmodified by activation steering (the intervention is on the model's internal activations, not the input). B8's observability logs the model's OUTPUT and the agent's behavior, not the model's internal activations — so the hook is invisible unless the deployer specifically instruments the hook layer. The defense is RUNTIME INTEGRITY: verifying that the hooks installed at inference time are the deployer's hooks, not the adversary's. This connects to B5 (who has permission to install hooks?) and B8 (are hooks logged?). Abliteration is not an input-layer attack; it is a model-layer attack."	c2b::b13::analysis
"State the load-bearing finding from BadNets (2017) and its implication for LLMs."	"BadNets (Gu, Dolan-Gavitt, Garg, 2017): a backdoored network classifies cleanly on normal inputs and misclassifies on a specific trigger pattern (a few-pixel stamp). The LOAD-BEARING FINDING: the backdoor SURVIVES TRANSFER LEARNING AND FINE-TUNING, because it is encoded in the lower feature-extractor layers (which change slowly during fine-tuning) while the new task is learned in the upper classifier layers. For LLMs: a pre-trained model with a weight-level backdoor propagates it through every fine-tune, every LoRA, every instruction-tuning round. The open-weights ecosystem (one base model, hundreds of fine-tunes) is the ideal propagation medium."	c2b::b13::analysis
"What does the .safetensors format PREVENT, and what does it NOT prevent? Why are the two properties independent?"	"PREVENTS: code execution on load. Unlike Python pickle (which can execute arbitrary code on deserialization — a real supply-chain attack vector), .safetensors cannot run code when loaded. This is a genuine safety property. DOES NOT PREVENT: weight-level backdoors. BadNets-style trojans live in the NUMERICAL WEIGHTS, which are legitimate data in any format. A .safetensors file with a weight-level trojan is SAFE TO LOAD (no code runs) and DANGEROUS TO USE (the model misbehaves on the trigger). The two properties — code-execution safety and weight-benignity — are independent. .safetensors provides the first; it cannot provide the second."	c2b::b13::analysis
"What is a sleeper agent (Hubinger et al., 2024), and why is it consequential?"	"A sleeper agent is a model trained to behave correctly during EVALUATION and mischievously during DEPLOYMENT — e.g., writing vulnerable code when it detects it is in a production environment rather than a test. The consequential finding from the Anthropic 2024 paper: this behavior SURVIVES safety training (RLHF), adversarial training, AND activation steering. The standard post-training defenses do not scrub it. Implication: a sufficiently capable adversary who can influence training can install a backdoor that survives the standard post-training defenses. The mitigation is NOT better post-training; it is supply-chain provenance (knowing who trained the model) and pre-deployment evaluation (testing for backdoor behavior)."	c2b::b13::analysis
"State the four-layer defense-in-depth stack for checkpoints, and which attack class each layer catches."	"LAYER 1 — PROVENANCE: record who trained it, on what data, with what post-training (the AI BOM, B11.2). Catches Class 4 (supply-chain trust). The foundation — without it, other layers have nothing to verify against. LAYER 2 — INTEGRITY: verify checkpoint hash over weight tensors (not just the file) against a trusted source. Catches Class 3 (tampering in transit/at rest). LAYER 3 — PRE-DEPLOYMENT EVALUATION: trigger-phrase probing + weight-distribution analysis + behavioral eval. Catches Class 2 (weight poisoning) + Class 3 (tampered checkpoints that pass layers 1-2). LAYER 4 — RUNTIME MONITORING: B8 observability watches for backdoor-trigger patterns in production. Catches Class 1 (activation steering) + Class 2 (triggered backdoors in prod). No single layer suffices."	c2b::b13::application
"Why is the LoRA adapter a threat surface, and how does it bypass signature verification?"	"A LoRA adapter is a LOW-RANK WEIGHT MODIFICATION — mathematically equivalent to a learned forward hook. An adversarial LoRA can implement any behavior a hook can, with the advantage that it LOOKS LIKE a normal fine-tuning artifact. The model's weights are clean; the LoRA is the attack. The LoRA PASSES signature verification because the signature verifies the UPLOADER'S IDENTITY (the adversary's), not the WEIGHTS' BENIGNITY. The AI BOM must enumerate LoRA adapters as SEPARATE DEPENDENCIES with their own provenance and trust evaluation. An adversarial LoRA can be stealthy: trigger on a natural input distribution (a specific language or topic) rather than an artificial pattern."	c2b::b13::analysis
"What is the dual-use problem with SAEs (Sparse Autoencoders), and what is the defensive posture?"	"DUAL-USE (structural): the SAE that identifies the refusal feature also tells an adversary which direction to SUBTRACT to suppress refusal. The interpretability dashboard that shows the honesty feature shows an adversary which direction to FLIP to induce lying. The tool does not distinguish between a researcher and an adversary — it exposes the model's internals to anyone who runs it. DEFENSIVE POSTURE: do NOT suppress the ecosystem (which would slow legitimate safety research). Instead: (1) treat feature-level knowledge as sensitive, (2) monitor for feature-level attacks (an adversary who knows the features can craft inputs that target them — more powerful than input-level adversarial examples), (3) use SAEs defensively (the same SAE that identifies a refusal feature can MONITOR whether the refusal feature is active in production — a runtime detector for alignment-removal attacks)."	c2b::b13::analysis
"Why is activation steering a higher-bar attack than prompt injection, and what are the realistic threat scenarios?"	"Activation steering requires RUNTIME ACCESS — the ability to install a forward hook on the model's inference path. This is a HIGHER BAR than prompt injection (which requires only the ability to send a prompt). Realistic scenarios: (1) the MALICIOUS INSIDER with model-deployment access who installs a hook to suppress safety behavior; (2) the TAMPERED INFERENCE SERVER — an adversary who compromised the server via traditional ATT&CK techniques installs a hook as persistence; (3) the MALICIOUS FINE-TUNE that ships a 'convenience' inference wrapper with a hidden hook; (4) the LoRA ADAPTER as a stealth hook (mathematically equivalent). In all cases the runtime control gap is the same: the model's internal representations are uninspected by B2-B8."	c2b::b13::analysis
"What is Neuronpedia, and what role does it play in the dual-use landscape?"	"Neuronpedia is the central ecosystem hub for SAE-based interpretability work — a platform for exploring, visualizing, and steering model internals. The ecosystem includes SAEBench (the standardized SAE benchmark), sae-lens (the library for extracting SAE features), and a growing collection of pre-trained SAEs for major open-weights models. It is a RESEARCH ACCELERATOR and a DUAL-USE concern in the same gesture. The ecosystem cannot be suppressed without slowing legitimate safety research, but it must be incorporated into the threat model: feature-level knowledge published in the ecosystem gives adversaries a map of model internals, enabling feature-level attacks and partial model cloning."	c2b::b13::recall
"Why is 'treating the model as a trusted base' an anti-pattern, and what is the cure?"	"Every runtime control B2-B8 assumes the model is trustworthy — the load-bearing assumption. The model is a software artifact with a supply chain, provenance, and an internal state that can be tampered with. Treating it as a trusted base leaves the seventh surface undefended. CURE: treat it as a dependency — enumerate it in the AI BOM (B11.2), verify its integrity (hash over weight tensors), evaluate it before deployment (trigger-phrase probing, weight-distribution analysis), and monitor it in production (B8 observability). The four-layer defense-in-depth stack (provenance → integrity → pre-deployment eval → runtime monitoring) is the cure."	c2b::b13::analysis
"Why is 'assuming .safetensors is safe because it prevents code execution' an anti-pattern?"	".safetensors prevents CODE EXECUTION on load (unlike pickle) — a real safety property. It does NOT prevent WEIGHT-LEVEL BACKDOORS — BadNets-style trojans live in the numerical weights, which are legitimate data in any format. A .safetensors file with a weight-level trojan is safe to load and dangerous to use. Conflating the two safety properties leads to a false sense of security: a team that adopted .safetensors and declared the checkpoint layer 'handled' has closed the code-execution vector but left the weight-backdoor vector wide open. The two safety properties are INDEPENDENT."	c2b::b13::analysis
"Why is 'evaluating only for capability' an anti-pattern?"	"Standard model evaluation measures CAPABILITY — does the model do its task well (benchmarks, perplexity, accuracy). Backdoor evaluation measures MISBEHAVIOR on triggers — does the model do something DIFFERENT on a specific input. The two are INDEPENDENT. A model can score at the top of capability benchmarks AND harbor a backdoor that activates on a trigger phrase. Pre-deployment evaluation that runs only capability benchmarks will pass a trojanized model. CURE: pre-deployment must include BOTH capability evaluation AND backdoor evaluation (trigger-phrase probing, weight-distribution analysis, behavioral evaluation on adversarial inputs)."	c2b::b13::analysis
"How do the four B13 attack classes map onto ATLAS tactics (SDD-B12)?"	"CLASS 1 (activation steering) → ATLAS Execution TA0005 (the adversary runs adversarial computation through the model's activation pipeline) and Defense Evasion TA0007 (the hook evades input-layer defenses). CLASS 2 (weight poisoning) → ATLAS Persistence TA0006 (a durable compromise surviving model reloads and retraining — the load-bearing distinction from SDD-B12). CLASS 3 (checkpoint manipulation) → ATLAS Persistence TA0006 + Initial Access TA0003 (the tampered checkpoint is the initial access vector). CLASS 4 (supply-chain trust) → ATLAS Initial Access TA0003 (AML.T0017 ML Supply Chain Compromise). The mapping shows B13 is the model-layer instantiation of ATLAS Persistence and Execution tactics."	c2b::b13::analysis
"What is the C3 FT17 / B13 bridge, and why is it load-bearing for the curriculum?"	"C3 FT17 teaches activation steering (CAA) as an ALIGNMENT tool (the builder's view — add the refusal direction to strengthen safety). B13 teaches the IDENTICAL TECHNIQUE as an ATTACK (the adversary's view — subtract the refusal direction for abliteration). The bridge is load-bearing because it provides DEFENSE-IN-DEPTH at the representation layer: you cannot defend a surface you do not understand. A student who has taken only C3 FT17 knows the technique but not the attack surface; a student who has taken only B13 knows the attack but not the legitimate use. Reading both produces a practitioner who can build runtime alignment controls AND detect when those controls have been subverted. The same forward hook, read two ways."	c2b::b13::analysis
"Why does the open-weights ecosystem amplify the weight-poisoning risk?"	"Open-weights LLMs (Llama, Mistral, Qwen and their fine-tunes) are routinely downloaded from public hubs by developers who do not verify provenance. A trojanized fine-tune uploaded with a name similar to a legitimate model is the LLM analogue of a typosquatted package (ASI08). The signature verifies the UPLOADER'S IDENTITY; it does not verify the WEIGHTS' BENIGNITY. A model with a valid uploader signature and a weight-level backdoor passes every signature check. The ecosystem treats models as ARTIFACTS TO BE DOWNLOADED, not as SOFTWARE TO BE VERIFIED. The cure: the AI BOM must enumerate the model, its checkpoint hash, its provenance, and its trust status — and procurement (Course 4 E11) must require this enumeration before deployment."	c2b::b13::analysis
"How does B13 connect to Course 4 E11 (governance and organizational process)?"	"B13 provides the TECHNICAL FOUNDATIONS for model trust evaluation (the four attack classes, the defense-in-depth stack, the AI BOM model-layer entry, the hash verification procedure). Course 4 E11 provides the ORGANIZATIONAL PROCESS — when an organization acquires a model (by training, fine-tuning, or downloading), the procurement process must include a trust evaluation. The connection is load-bearing: without the technical foundations (B13), the procurement process (E11) has NOTHING TO EVALUATE; without the procurement process (E11), the technical foundations (B13) are NOT APPLIED. B13 is the engineering; E11 is the governance that ensures the engineering happens."	c2b::b13::analysis