Dolphin / Hermes — Uncensored Lineages as Engineering Case Studies
Module FTDD-06 · Course 3 — LLM Fine-Tuning Masterclass
45 minutes · 4 sub-sections: Framing · Hermes 3 · Dolphin · Trade-offs & Synthesis
Studied as engineering, not advocacy. Production examples of the FT16–FT18 techniques.
Deep-Dives
The framing: case studies, not advocacy
The course stance is unchanged. The model steers; the harness bounds. An uncensored model is only responsible inside an eval'd harness. Uncensoring raises the harness requirement — it does not lower it.
Three engineering reasons to study them:
- Best-documented large-scale examples of the FT16–FT18 alignment-control techniques.
- They make the dataset-as-steering-wheel thesis concrete (OpenHermes 2.5).
- They expose the capability cost of uncensoring as measurable data (FT17: up to −18.8pp GSM8K).
Hermes 3 — full-param SFT+DPO at scale
Nous Research · arXiv:2408.11857 · Llama 3.1 8B/70B/405B
"Neutrally-aligned generalist instruct and tool-use model... unlocked, uncensored, highly steerable." Trained on primarily synthetic responses.
| Stage | What it steers | FT module |
|---|
| 1. Full-param SFT | format, instruction-following, character | FT12 |
| 2. DPO | sharpen toward compliance + steerability | FT13 |
Full-param (not LoRA) at all three scales — finds a higher-rank solution, justified for substantially shifting behavior. 405B full FT is a serious cluster job.
OpenHermes 2.5 — the steering wheel
Curated by Teknium
open-source + custom synthetic
One dataset steered a family: OpenHermes 2.5 → Nous Hermes 2 → Hermes 3. The course thesis made concrete — data matters more than algorithm. Flaws propagate too; audit your data like code.
Dolphin — compliance over judgment
Eric Hartford · Cognitive Computations
The philosophy: the model complies with instructions and harness policy rather than imposing judgmental refusals. Executed primarily via dataset curation (train toward compliance), not primarily abliteration.
Dolphin3.0-R1-Mistral-24B is technically distinctive:
- Base: Mistral Small 24B (Instruct-2501)
- ~800K reasoning traces from DeepSeek-R1, over 3 rounds
- The only uncensored model trained on R1 reasoning traces
Sits at the intersection of FT14 (GRPO/reasoning), FT15 (CoT distillation), FT16–18 (alignment control).
The capability cost is real and measurable
Removing refusal is not free. The refusal direction is entangled with other capabilities — steering away nudges them.
| FT17 finding (Dec 2025 study) | Effect |
|---|
| GSM8K math, best case | +1.5pp (rare gain) |
| GSM8K math, worst case | −18.8pp |
An uncensored model is NOT "the same model, minus the refusals." It is a different point in capability-compliance space. Read the full eval table, including regressions, and decide on the numbers for your task.
The synthesis (FT00 / FT23)
Uncensor the model so it executes;
harness the model so it executes only what it should.
| Model | Steered to... | Therefore needs... |
|---|
| Dolphin3.0-R1 | execute (compliance + R1 reasoning) | a harness that bounds what it MAY do |
| Hermes 3 | be "highly steerable" | a strong harness — steerability means a weak harness does damage |
| OpenHermes 2.5 | be the steering wheel | audit like code — flaws propagate to every spoke |
Anti-patterns
Studying as advocacy or condemnation. Both miss the engineering. Dolphin/Hermes are the best-documented large-scale alignment-control recipes — and they carry a measurable capability cost + raised harness requirement. Study the recipe and trade-offs.
Deploying uncensored without an eval'd harness. A compliance-oriented model will not self-refuse; it executes. Without a harness's policy gates, it executes the dangerous things too. The cardinal deployment error for this module.
Confusing data-driven compliance steering with abliteration. Hartford's approach trains toward compliance (data); abliteration deletes a refusal direction (weight edit). Different Layer 3 operations, different cost profiles.
What you can now do
- Describe the Hermes 3 recipe (full-param SFT then DPO on Llama 3.1) and place each stage on the Steering Stack.
- Describe the Dolphin lineage/philosophy and why Dolphin3.0-R1-Mistral-24B is technically distinctive.
- Explain OpenHermes 2.5's role as the dataset backbone — data matters more than algorithm.
- State the capability cost and the uncensored-in-harness synthesis as engineering, not ideology.
Next: FTDD-07 — DeepSeek-R1 · The reasoning lineage that fed Dolphin3.0-R1