Abliteration: Refusal-Direction Orthogonalization
The headline technique of Pillar 5. The Arditi et al. finding that refusal is mediated by a single residual-stream direction — and the abliteration pipeline that finds, validates, and permanently projects it out via weight orthogonalization. Full technical depth on the mechanics; honest about the capability-degradation trade-off (GSM8K from +1.51pp to −18.81pp).
Refusal in chat-tuned LLMs is mediated by an approximately single direction in the residual stream (Arditi et al., NeurIPS 2024; authors include Nanda and Gurnee). Validated across 13 open chat models up to 72B. This is the mechanistic license for abliteration.
Abliteration is the purest Layer 3 steering technique. Find the refusal direction by difference-in-means on contrastive activations, then permanently orthogonalize every write-to-residual weight against it: W' = W − r(rᵀW)/(rᵀr). No data, no optimizer, no retraining. It removes the refusal guardrail — not the knowledge underneath.
Abliteration is not free. The Dec 2025 comparative study (arXiv:2512.13655) found GSM8K changed from +1.51pp to −18.81pp (−26.5% relative) across four tools and sixteen 7B–14B models. The refusal direction is entangled with instruction-following, reasoning, and hedging; orthogonalizing against it deletes a slice of each. Always benchmark before and after.
Modern models (Llama 3.x, Qwen 2.5+) use extended-refusal training that spreads refusal across layers/directions, breaking the naive single-direction assumption. This is why the comparative study saw such variance. On modern models, use multi-direction tools (Heretic) or skip to DPO-compliance (FT18) when quality matters.
Never deploy an abliterated model without an eval'd harness (FT23). Abliteration changes what the model DOES, not what it MAY do. An uncensored model in a weak harness is strictly more dangerous than a refusal-trained model in a weak harness. Pillar 5 raises the harness requirement; it does not lower it.