Module FT12 — SFT: The Baseline

SFT: The Baseline

The opener of Pillar 3. SFT steers behavior — format, style, instruction-following, tool-call formatting, refusal calibration — without injecting knowledge. Build a high-quality SFT mixture (general + domain + tool + safety), diagnose the three failure modes (catastrophic forgetting, mode collapse, format leakage), and know when to stop at SFT versus escalate to preference methods (FT13).

minutes

artifacts

sub-sections

Supervised fine-tuning is the baseline of alignment — stage 1 of the modern post-training stack (SFT → DPO → GRPO) that every subsequent method builds on. This module nails what SFT can and cannot do: it steers format, style, and instruction-following by training on (input, target) pairs; it does not inject knowledge (the FT00 thesis). You build a defensible SFT mixture at the right ratios (40–50% general, 30–40% domain, 10–20% tool/safety), diagnose the three failure modes (catastrophic forgetting from too-narrow data, mode collapse from low diversity, format leakage from template bugs), and learn the rule for when to stop at SFT versus escalate to DPO: SFT for a single correct response; DPO for better-versus-worse preferences. A de-emphasized sidebar covers continued pretraining — the one exception that shifts knowledge distribution.

Key Claims

Load-Bearing Claims

SFT steers behavior; it does not inject knowledge. It redirects an already-capable base model's probability mass toward a target behavior — format, style, instruction-following, tool-call formatting, refusal calibration — by training on (input, target) pairs. If the base could already produce the behavior unreliably, that's SFT (~90% of real fine-tuning). If it couldn't, no SFT will fix it — you need a different base, CPT (rarely), or (usually) RAG. The cardinal error is treating SFT as knowledge injection.

SFT is stage 1 of the post-training stack — the foundation everything else builds on. The modern pipeline is SFT → DPO → (GRPO for reasoning). DPO assumes the model already produces plausible responses in the right format (it just ranks them); GRPO assumes it already produces candidate solutions (it just rewards correct ones). Skip SFT and the preference/reasoning stages have no substrate to refine.

The SFT mixture encodes what you want the model to be, and imbalance has a predictable signature. A defensible mix: 40–50% general instruction-following (Magpie/teacher from FT05), 30–40% domain examples, 5–10% tool-use formatting, 5–10% safety calibration. Too much general → no domain lift; too much domain → catastrophic forgetting; too much tool-use → format leakage; too much safety → refusal-happy. Curation beats volume (LIMA, Magpie).

Three SFT failure modes account for almost every bad run, and all three are steering failures the loss curve can't catch. Catastrophic forgetting (too-narrow data — fix: mix general data, prefer LoRA). Mode collapse (low-diversity data — fix: diversity filtering FT06). Format leakage (template bugs FT07 — fix: tokenizer chat template, round-trip test). Only the three-axis eval — general benchmarks, domain lift, format compliance — detects them.

After This Module

State what SFT can and cannot do — steers format/style/behavior/instruction-following; does NOT inject knowledge — and defend the distinction with the FT00 thesis and the LoRA-vs-full-FT evidence.

Build a high-quality SFT mixture — general instruction-following (Magpie/teacher distillation from FT05), domain examples, tool-use formatting, and safety calibration — with defensible ratios (40–50% general, 30–40% domain, 10–20% tool/safety) and explain how too much of one source skews the model.

Diagnose the three SFT failure modes — catastrophic forgetting (too-narrow data), mode collapse (low-diversity data), and format leakage (template bugs from FT07) — and name the mitigation for each (mix general data, diversity filtering, template hygiene).

Decide when to stop at SFT versus escalate to preference methods (FT13): SFT suffices when there is a single correct response; escalate to DPO when you must express better versus worse.

Explain the modern post-training stack — SFT → DPO → (GRPO for reasoning) — and place SFT as stage 1, the foundation every subsequent alignment method builds on.

Artifacts

Teaching Document

~3,800 words; 6 sub-sections — what SFT is/is-not (the thesis), building a high-quality SFT mix (ratios, sourcing, curation-beats-volume), the three failure modes (forgetting, mode collapse, format leakage), when to stop vs escalate (SFT vs DPO vs GRPO), the three-axis eval, and a de-emphasized CPT sidebar (plasticity-stability, replay training, NeurIPS 2025 perplexity filtering)

READ

Diagrams

5 Mermaid diagrams — the modern post-training stack (SFT→DPO→GRPO), the SFT mix ratios (with imbalance signatures), the three SFT failure modes (cause/signature/mitigation), the CPT-vs-SFT decision tree, and the SFT-to-preference escalation

READ

Slide Deck

12 reveal.js slides — title, what SFT is/is-not, SFT as stage 1, the SFT mix (with proportional bar), mixture imbalance signatures, the three failure modes, stop-vs-escalate, the three-axis eval, the CPT sidebar, anti-patterns, the lab, what you can now do

READ

Teaching Script

~2,050 words spoken; ~40 min at 140 wpm; 16 [SLIDE N] cues matching the deck