Compliance via DPO and SFT
The higher-fidelity alternatives to abliteration: DPO-toward-compliance and continued SFT on uncensored instruction data. Abliteration is the prototype, DPO is the refinement, SFT is the product — and the production uncensored models (Dolphin, Hermes 3) are trained, not abliterated.
There are exactly three ways to remove refusals, and they form a spectrum: abliteration (surgical, cheapest, capability-degrading), DPO-toward-compliance (targeted, moderate cost, needs preference pairs), and continued SFT on uncensored data (most natural, highest data requirement, best capability preservation when data is diverse).
DPO-toward-compliance is higher-fidelity than abliteration because it adjusts the model's policy via gradient descent — the optimization can route around the capability entanglement that abliteration's blunt direction-deletion hits.
SFT on uncensored data produces the most natural-feeling compliance because the model has genuinely learned the compliant distribution as its default behavior, not grafted compliance onto a model with the refusal direction deleted. The cost is data: a diverse, large instruction mix is mandatory, or the model mode-collapses.
The production uncensored models are trained, not abliterated. Dolphin3.0-R1-Mistral-24B (the only uncensored model on DeepSeek-R1 reasoning traces) and Nous Hermes 3 (arXiv:2408.11857, full-param SFT+DPO on Llama 3.1) prove the SFT+DPO path works at scale.
Abliteration is the prototype, DPO is the refinement, SFT is the product. Match the method to the actual requirement — cheap-and-fast, targeted-fix, or production-grade — not to the default of 'cheapest.'
Hartford's 'compliance over judgment' raises the harness requirement, it does not lower it. A compliance-by-design model is strictly more dangerous in a weak harness than a judging model in a weak harness. Pillar 5 demands a stronger Layer 5 (FT23), not a weaker one.