{ "module": "FT17 — Abliteration: Refusal-Direction Orthogonalization", "course": "3 — LLM Fine-Tuning Masterclass", "version": "1.0.0", "duration_minutes": 55, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis-design", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the central finding of Arditi et al. (arXiv:2406.11717)?", "options": [ "Refusal is implemented as a complex distributed circuit across thousands of features in dozens of layers, requiring full retraining to modify.", "Refusal in chat-tuned LLMs is mediated by an approximately one-dimensional subspace of the residual stream — find that direction, erase it, the model stops refusing. Validated across 13 open models up to 72B.", "Refusal can only be removed by DPO on a large preference dataset; there is no direct weight-edit path.", "Refusal behavior is hardcoded into the tokenizer and cannot be removed without re-pretraining." ], "answer_index": 1, "rationale": "The Arditi et al. finding (NeurIPS 2024, authors include Nanda and Gurnee): refusal is mediated by an approximately single direction in the residual stream, validated across 13 open chat models up to 72B. This is the mechanistic license for abliteration." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "Which equation defines the weight-orthogonalization edit at the heart of abliteration?", "options": [ "W' = W + r(rᵀW)/(rᵀr)", "W' = W − r(rᵀW)/(rᵀr)", "W' = W · r(rᵀW)", "W' = (Wᵀr) · r / ‖r‖²" ], "answer_index": 1, "rationale": "W' = W − r(rᵀW)/(rᵀr) is the projection of W onto the hyperplane orthogonal to r. rᵀW measures alignment of W's columns with r; r(rᵀW) is the rank-1 r-writing component; subtracting it makes r geometrically unreachable. This is the permanent, surgical edit." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "In the Dec 2025 comparative study (arXiv:2512.13655), what was the range of GSM8K change observed across four tools and sixteen 7B–14B models?", "options": [ "−0.5pp to −2.0pp (abliteration is nearly free)", "+1.51pp to −18.81pp (a −26.5% relative drop in the worst case)", "−5pp to −50pp (abliteration always catastrophically breaks reasoning)", "No measurable change on any benchmark" ], "answer_index": 1, "rationale": "The headline finding: GSM8K changed from +1.51pp to −18.81pp (−26.5% relative) depending on tool and model. The −18.81pp worst case was Yi-1.5-9B. This is the evidence that abliteration is NOT free — the same technique, four tools, sixteen models, a 20-point spread in math damage." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "You are abliterating a Llama-style decoder. Which weight matrices should you edit (apply W' = W − r(rᵀW)/(rᵀr) to)?", "options": [ "Only the attention query/key/value projections and MLP gate/up projections.", "The matrices that WRITE to the residual stream: attention o_proj, MLP down_proj, and the embedding. Leave readers (q/k/v, gate/up) alone.", "Every weight matrix in the model, for maximum thoroughness.", "Only the embedding and the final unembedding matrix." ], "answer_index": 1, "rationale": "You edit only matrices that WRITE to the residual stream (o_proj, down_proj, embed). Readers (q/k/v projections, MLP gate/up) can still SEE the r direction — they just cannot write into it. Editing readers would be wrong; editing everything would cause excessive capability damage." }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "You want to find the refusal direction in a 32-layer model. How many candidate directions do you extract, and how?", "options": [ "One candidate, by averaging activations across all layers.", "32 candidates (one per layer), each computed as mean(harmful activations at final token, that layer) − mean(benign activations at final token, that layer). Then pick the layer whose direction most reduces refusal when clamped.", "One candidate per attention head (thousands), then cluster them.", "One candidate, computed only at the embedding layer before any transformer blocks." ], "answer_index": 1, "rationale": "Difference-in-means is computed per layer at the final-token residual-stream activation: r_ℓ = mean(a_ℓ(harmful)) − mean(a_ℓ(benign)). One candidate per layer (32 for a 32-layer model). The winning layer is usually in the middle-to-upper third and is chosen by intervention accuracy (which direction most reduces refusal when clamped)." }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "During validation, you find that clamping the top-1 principal component of the harmful/benign difference reduces refusal only ~40%, and you need k=4 components to reach ~95% refusal reduction. What does this tell you, and what should you do?", "options": [ "Your diff-in-means implementation is buggy; re-run with more harmful prompts.", "Your model is a MODERN model with extended-refusal training — refusal is not cleanly ~1D. Naive single-direction ablation will be incomplete. Options: accept partial removal (ErisForge-style), use a multi-direction tool (Heretic), or switch to DPO-compliance (FT18).", "You picked the wrong layer; re-run the layer sweep.", "This is expected for all models; proceed with single-direction ablation anyway." ], "answer_index": 1, "rationale": "For 2023-era models, k=1 captures almost all refusal effect (sharply 1D). If you need k>1, the model has extended-refusal training (Llama 3.x, Qwen 2.5+) where refusal spreads across layers/directions. Naive single-direction ablation leaves residual refusals; pushing harder causes capability damage. This is the modern-model caveat." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "You must abliterate a model and need the LOWEST capability damage on GSM8K. Which tool do you reach for, and what is the trade-off?", "options": [ "Heretic — it has the lowest average GSM8K damage (−0.28pp) and the most thorough refusal removal.", "ErisForge — it has the best capability preservation (avg GSM8K −0.28pp), but the trade-off is that it is the least aggressive: some refusals will remain on hard prompts (it removes the dominant direction, leaves the rest).", "FailSpy — it always removes 100% of refusals with zero capability loss.", "OBLITERATUS — the most aggressive tool guarantees both full removal and zero damage." ], "answer_index": 1, "rationale": "Per arXiv:2512.13655, ErisForge has the best capability preservation (avg GSM8K −0.28pp). The trade-off: it is the LEAST aggressive tool — it removes the dominant refusal direction and leaves the rest, accepting that residual refusals leak back on adversarial prompts. Low damage, partial removal. (Heretic is most thorough but worst avg damage at −7.81pp.)" }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "Your team abliterated a 7B model, saw refusals vanish, and shipped it. Three weeks later users report 'it got dumber on math.' What went wrong, and what should you have done?", "options": [ "The abliteration was incomplete; re-run with a more aggressive tool.", "Abliteration is not free — you assumed it was capability-neutral and did not measure. You should have run GSM8K/MMLU/IFEval BEFORE and AFTER on the same hardware/sampling params and quoted the delta. The 'dumber' reports are the entanglement cost (up to −18.81pp GSM8K).", "The model was too small for abliteration; use a 70B model.", "You forgot to quantize after abliterating, which caused the degradation." ], "answer_index": 1, "rationale": "The 'assuming abliteration is free' / 'not measuring capability impact' anti-patterns. Abliteration deletes the refusal direction, which is entangled with reasoning. Casual use does not exercise the entangled capabilities, so 'feels fine' is not a benchmark. The fix is to benchmark before/after and quote the delta." }, { "id": "Q09", "bloom": "application", "type": "multiple_choice", "prompt": "A deployment needs an uncensored model where math/reasoning quality is critical. What is the recommended path, and why?", "options": [ "Naive abliteration with Heretic — it removes the most refusals.", "Abliterate-then-recover (abliterate, then SFT/DPO on high-quality data to repair capability damage), OR skip abliteration and use DPO-compliance (FT18). Naive abliteration alone carries up to an 18.81pp GSM8K tax — unacceptable when reasoning quality is critical.", "Abliterate, then quantize to Q4 to recover the lost reasoning.", "Use a larger base model — the capability damage scales away at 70B." ], "answer_index": 1, "rationale": "When quality matters, abliterate-then-recover (repair the damage with a small SFT/DPO pass) or skip abliteration entirely for DPO-compliance (FT18, the production-quality alternative). Naive abliteration alone trades up to 18.81pp of GSM8K for refusal removal — unacceptable for a reasoning-critical deployment." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is abliteration described as 'the purest expression of the FT00 steering thesis (fine-tuning steers behavior, not knowledge)'?", "options": [ "Because it uses a sophisticated optimizer that precisely steers the model without touching knowledge.", "Because it deletes ONE vector with no data, no optimizer, no gradient. SFT/DPO/GRPO all use gradients+data; abliteration is steering reduced to geometric essence. It removes a steer added during safety tuning — it does not teach anything. The model already knew the unlocked content from pretraining.", "Because it is the only technique that adds new knowledge to the weights.", "Because it is the cheapest technique, and cheapness is the defining property of steering." ], "answer_index": 1, "rationale": "Abliteration is pure steering: find the refusal direction, orthogonalize the weights against it, done. No data, no optimizer, no gradient. The model already knew how to produce the unlocked content (it saw it in pretraining); abliteration removes the guardrail added during safety tuning, not the knowledge. This is FT00's steering-vs-teaching distinction made literal in the weights." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "Mechanistically, why does abliteration degrade math reasoning (and instruction-following, and hedging)?", "options": [ "Because the abliteration tools have bugs that corrupt the math weights.", "Because the refusal direction is ENTANGLED with other capabilities — it overlaps with instruction-following ('should I comply?' shares with 'should I refuse?'), reasoning (slow-down-and-check correlates with cautious refusal), and hedging. Orthogonalizing against r deletes a slice of each. r is not orthogonal to everything else.", "Because abliteration reduces the model's parameter count.", "Because abliteration always damages the tokenizer, which hurts math." ], "answer_index": 1, "rationale": "The refusal direction is found by contrastive activations — it best SEPARATES harmful from benign, but 'best separates' ≠ 'exclusively about refusal.' It overlaps (is entangled) with instruction-following, reasoning, and hedging directions. Orthogonalizing against r removes a slice of each entangled capability. This is the deepest statement of the FT00 thesis: residual-stream directions are not orthogonal to the capabilities you want to keep." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why did the comparative study (arXiv:2512.13655) observe GSM8K damage ranging from +1.51pp to −18.81pp across tools and models, rather than a consistent degradation?", "options": [ "Because the study was poorly controlled and the variance is measurement noise.", "Because models that suffered most were where the tool had to be more AGGRESSIVE (target more layers, use multiple directions) to fully suppress refusal — and aggressiveness causes capability damage. ErisForge preserved capability best because it is the LEAST aggressive (accepts partial refusal removal). Model architecture (e.g., Yi-1.5-9B) also matters.", "Because some tools secretly retrain the model and others do not.", "Because the GSM8K benchmark is unreliable for abliterated models." ], "answer_index": 1, "rationale": "The variance reflects the aggression-vs-preservation dial. Models with extended-refusal training (where refusal spreads across layers/directions) require more aggressive multi-direction intervention to fully suppress refusal — and that aggression is what causes capability damage. ErisForge is least aggressive (lowest damage, partial removal); Heretic is most aggressive (highest avg damage −7.81pp, most thorough removal). Architecture also contributes (Yi-1.5-9B was the −18.81pp worst case)." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "Does abliteration achieve 'unlearning' of harmful knowledge? Why or why not, and what is the implication?", "options": [ "Yes — deleting the refusal direction removes the model's knowledge of harmful content.", "No — abliteration removes the REFUSAL BEHAVIOR, not the KNOWLEDGE. The model still knows how to synthesize the harmful content (it learned it in pretraining); it just no longer refuses to discuss it. Genuine unlearning (removing the knowledge itself) is a different, much harder problem that abliteration does NOT solve.", "Yes, but only for chemistry knowledge; math knowledge is retained.", "Only if you target the pre stream rather than the post stream." ], "answer_index": 1, "rationale": "Abliteration removes the refusal guardrail, not the underlying knowledge. The model already knew the harmful content from pretraining; abliteration stops it from refusing, but the capability/knowledge remains in the weights. Confusing abliteration with unlearning is an anti-pattern — genuine machine unlearning is a distinct and much harder research problem." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "An organization wants to deploy an abliterated model for a customer-facing tool-use agent. Why is 'deploy inside an eval'd harness' non-negotiable, per FT17 and FT23?", "options": [ "Because the harness makes the abliterated model faster and reduces VRAM.", "Because abliteration changes what the model DOES (it won't refuse), but not what it MAY do — the boundary between 'does' and 'may' is the harness (Layer 5). An uncensored model that won't refuse a legitimate request also won't refuse an illegitimate one. Uncensored-in-weak-harness is strictly MORE dangerous than refusal-trained-in-weak-harness.", "Because it is legally required to have a harness after abliteration in all jurisdictions.", "Because the harness restores the math capability that abliteration damaged." ], "answer_index": 1, "rationale": "The synthesis principle (FT23): abliteration changes what the model does (it will not refuse), but not what it may do (that is the harness's job). An uncensored model in a weak harness is strictly more dangerous than a refusal-trained model in a weak harness — it executes whatever it is asked, including the dangerous things. Pillar 5 RAISES the harness requirement; it does not lower it." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "You are choosing between abliteration and DPO-compliance (FT18) for a production deployment that needs both refusal removal AND high reasoning quality. How does the modern-model caveat (Llama 3.x / Qwen 2.5+) factor into the decision?", "options": [ "The modern-model caveat is irrelevant; abliteration works identically on all models.", "On modern models, naive single-direction abliteration is incomplete (refusal spreads across layers/directions). To fully suppress refusal you must be aggressive (multi-direction, Heretic), which causes the most capability damage (up to −18.81pp GSM8K). When reasoning quality is critical, this pushes you toward DPO-compliance (FT18) — more work (preference data + training), but no capability tax and full refusal control.", "Modern models cannot be abliterated at all; DPO is the only option.", "The caveat means you should always prefer the oldest base model available." ], "answer_index": 1, "rationale": "The modern-model caveat: Llama 3.x / Qwen 2.5+ have extended-refusal training that spreads refusal across layers/directions. Naive single-direction abliteration is incomplete; full suppression requires aggressive multi-direction intervention (highest capability damage). When reasoning quality is critical, this tips the decision toward DPO-compliance (FT18): more work (preference data, training) but no capability tax and full control. The caveat is the empirical reason abliteration is a 'fast first pass,' not the final word for production." } ] }