# Teaching Script — Module FT12: SFT: The Baseline

**Course**: Course 3 — LLM Fine-Tuning Masterclass
**Module**: FT12 — SFT: The Baseline
**Duration**: ~40 minutes (spoken at ~140 wpm)
**Format**: Verbatim transcript with `[SLIDE N]` cues. Read aloud or use as speaker notes.

---

[SLIDE 1 — Title]

Welcome to module FT twelve, SFT, the Baseline. This is the opener of pillar three, alignment. In FT eleven you learned to run the training loop. Now we zoom into the method that is the starting point for almost every real fine-tuning project — supervised fine-tuning, done right. SFT is the baseline of alignment. Everything else in this pillar — DPO, GRPO, abliteration — builds on top of it.

If you have not done FT eleven, go back. This module assumes you can run an SFTTrainer job, read a loss curve, and choose the TrainingArguments levers. It does not re-teach those.

[SLIDE 2 — What SFT is, what it is not]

The sentence. Supervised fine-tuning steers an already-capable base model toward a target behavior — format, style, instruction-following, tool-call formatting, refusal calibration — by training on input, target-output pairs. It does not inject knowledge.

This is the FT zero-zero thesis restated for the steering layer. The base model already saw trillions of tokens during pretraining. It already knows how to answer questions, write code, structure prose. What it does not reliably do is answer in your format, in your tone, following your instructions, calling your tools the way you want them called. SFT closes that gap. It redirects probability mass from a plausible continuation to the continuation you want. That redirection is steering.

What SFT does well. Format — respond in strict JSON. Style and tone — concise, formal, safety-cautious. Instruction following — obey multi-step instructions. Tool-call formatting — emit the exact blocks your harness parses. Refusal calibration — refuse what you want refused, comply with the rest. All behaviors the base is capable of. SFT makes them reliable.

What SFT does not do. It does not pour facts into the weights. The cardinal error — the most expensive mistake in the field — is "we'll fine-tune the model on our medical records and it will learn medicine." It will not. It will appear to, by memorizing surface patterns, and then fail catastrophically on the first case that differs from the training distribution. If you want the model to use your knowledge base in a particular way — always cite, always flag uncertainty — that is steering, and SFT is the right tool. If you want the model to know something it does not, retrieve. RAG is not this course, but for knowledge it is the correct answer.

[SLIDE 3 — The thesis test]

The test that tells you whether your problem is SFT. Before you fine-tune, prompt the base with your ideal input and a great system prompt. If the base could already produce the target behavior unreliably or in the wrong format — that is an SFT problem. About ninety percent of real fine-tuning. If it could not produce the behavior at all, no amount of SFT will reliably get you there. You need a different base, continued pretraining — the sidebar at the end — or, usually, RAG.

[SLIDE 4 — Why SFT is the foundation: the post-training stack]

SFT is stage one of the modern post-training stack. The pipeline is three stages. One, SFT — teach the model the format, the style, the basic behavior. This module. Two, DPO — refine the model's preferences now that it produces the right format. Module FT thirteen. Three, GRPO — sharpen reasoning on problems with checkable answers. Module FT fourteen.

SFT is first because it gives the later stages something to work with. DPO assumes the model already produces plausible responses in the right format — it just ranks them. GRPO assumes the model already produces candidate solutions — it just rewards the correct ones. Skip SFT and the preference and reasoning stages have no substrate to refine. This is why every real alignment pipeline starts here, and why SFT the baseline is the most accurate one-line description of this module.

[SLIDE 5 — Building a high-quality SFT mix]

Now the heart of the module — the steering wheel for the steering layer. A production SFT dataset is not a single source. It is a blend of complementary sources, each covering a behavior the model must reliably exhibit. The typical defensible mix: general instruction-following at forty to fifty percent, domain examples at thirty to forty percent, tool-use formatting at five to ten percent, safety calibration at five to ten percent.

Why those numbers. General is the substrate — Magpie-synthesized instructions from FT zero-five, or teacher-distilled pairs. It keeps the model a general assistant. Domain is your actual use cases — medical, legal, code, security. This is where the domain lift comes from. Tool-use is the exact tool-call schema your harness parses. Safety is refusals and clarifications, balanced.

The principle behind the numbers is the part to internalize: the mix encodes what you want the model to be. Too much of one source and the model skews toward it.

[SLIDE 6 — Why mixture ratios matter]

Too much general data and the model is a fine general assistant but your domain lift evaporates. You trained the model to be Qwen-two-point-five Instruct, which it already was. Too much domain data and the model becomes a specialist that forgot how to be a general assistant — it answers medical questions well and refuses to write a Python function. That is catastrophic forgetting, and it is the failure the general portion exists to prevent. Too much tool-use data and every response tries to emit a tool call, even on a plain question — format leaks into everything. Too much safety data and the model becomes refusal-happy.

The right ratio is found empirically, by eval. But the starting ratio — roughly half general, a third domain, a sixth tool and safety — is the one that consistently produces a model that is both capable and domain-lifted, without forgetting.

[SLIDE 7 — The cardinal data rule]

Curation beats volume. A two-thousand-example set you actually read and curated will steer better than fifty thousand raw outputs you did not look at.

This is the lesson of LIMA, of Magpie, of the entire pillar-one arc, restated for SFT because it is still the most common mistake. People believe scale of data is the lever. It is not — scale of curation is. Read your data, dedup it, filter it, decontaminate it, balance the mix. Then train. The gains are in filtering, not generation.

[SLIDE 8 — SFT failure mode one: catastrophic forgetting]

Three ways an SFT job goes wrong. Each with a signature and a mitigation.

Failure one: catastrophic forgetting. The model trains on a narrow distribution — pure domain, pure tool-use, pure safety — and loses general capability. You fine-tuned on medical Q and A and now the model writes worse Python than the base did. The knowledge did not leave the weights — FT zero-zero, steering not teaching — but the behavioral disposition shifted. The model's probability mass concentrated on the narrow distribution and away from general assistance.

The signature: domain metrics go up, general benchmarks go down. The gap between them is the forgetting. The mitigation: mix general data into the SFT set — the forty to fifty percent general. This is the same mechanism as continued pretraining's mixture-of-source training, which we cover in the sidebar. LoRA also mitigates forgetting implicitly — the low-rank adapter cannot move enough parameters to forget as much as full fine-tuning can. That is one reason LoRA is the default for SFT.

[SLIDE 9 — SFT failure mode two: mode collapse]

Failure two: mode collapse. The training data is low-diversity — every response looks the same, same opening, same structure, same length — and the model collapses onto that mode. Every output starts with "great question." Every summary is three bullets. Every code answer is wrapped in the same preamble. The model is not broken. It is boring. And boring is a quality failure.

The signature: every generation looks the same, regardless of input. Diversity metrics collapse. The mitigation: diversity filtering, from FT zero-six. Before training, measure the diversity of your set and drop near-duplicates, or actively sample to maximize embedding-space spread. Mix sources — a blend of Magpie plus teacher plus domain has more diversity than any single source. And do not over-train. Mode collapse worsens with epochs past convergence.

[SLIDE 10 — SFT failure mode three: format leakage]

Failure three: format leakage. The chat template applied during training does not match the template the inference harness applies, or the training examples use a different format than the harness expects. The model learns a format that works in training and breaks in production. This is the FT zero-seven failure, now manifest at the alignment layer.

The signature: the model produces fluent, confident output that the harness cannot parse — tool calls with the wrong field names, JSON that does not validate, responses that cut off because the EOS token is wrong. The loss curve looks fine, because the model learned a format. Production behavior is broken. The mitigation: template hygiene. Use the model's own tokenizer chat template — TRL does this for you. Test the round-trip: apply the template, tokenize, decode, and confirm the output matches what your inference path produces. One template bug poisons the whole run.

[SLIDE 11 — When to stop at SFT, when to escalate]

Now the decision that saves you from building a preference pipeline you did not need, or skipping one you did. The rule: SFT is enough when there is a single correct response. Escalate to DPO when you need to express better versus worse between multiple plausible responses.

SFT trains on input, target pairs — one right answer per input. It is the right tool when the target is unambiguous: the correct tool call, the right JSON, the desired summary. There is no preference to express. DPO trains on input, preferred, rejected triples — two answers per input, with a label saying which is better. It is the right tool when multiple responses are plausible but you have a preference: more helpful, less verbose, more cautious.

How to tell which side you are on. Ask, for your task: is there one right answer, or several acceptable answers with a quality ordering? One right answer — SFT. Several acceptable with preference — SFT to establish the format, then DPO. Reasoning with checkable answers — SFT then GRPO. Most real pipelines do SFT first, then escalate only if the SFT model still has a quality gap that preferences could close.

[SLIDE 12 — The three-axis eval]

If you did not eval, you did not fine-tune. You hoped. The eval obligation from FT eleven applies in full. For SFT specifically, three evals matter.

One, general benchmarks — MMLU, MT-Bench, AlpacaEval. Did the model forget general capability? Compare to the base. If these dropped, you have catastrophic forgetting. Add general data and re-run. Two, domain eval — a held-out set of domain input, ideal-output pairs, scored by exact match or an LLM judge. This is the domain lift — the whole point of the fine-tune. If it did not go up, the SFT did not work for its purpose. Three, format compliance — does the model emit the right format at the rate you need? Measure the parse-success rate on a sample. A model that produces great prose in the wrong format is a production failure.

The deliverable of an SFT project is the report: domain lift was X, general capability changed by Y, format compliance is Z percent. Without that triangle, you are shipping on faith.

[SLIDE 13 — Sidebar: continued pretraining]

Now the sidebar — the one exception to steering not teaching. Continued pretraining, CPT. This actually shifts the model's knowledge distribution. It is the one technique in Layer three that genuinely teaches rather than steers. It is de-emphasized because almost nobody reading this needs it, and RAG is usually better.

What CPT is. You take the base and keep pretraining it — next-token prediction on a large corpus — to shift its knowledge distribution toward a domain. Train a general base on a few billion tokens of medical literature and the model's internal representation of medicine shifts. It now knows the domain differently than the base did.

The fundamental tension: the plasticity-stability dilemma. To learn the new domain — plasticity — the model must change its weights. But changing weights risks destroying what it already knew — stability. The symptom is catastrophic forgetting.

The mitigations, in order of effectiveness. One, mixture-of-source or replay training. Do not train on pure domain data. Mix domain tokens with general tokens at roughly one to one to two to one. The general tokens replay what the model knew, anchoring it while the domain tokens shift it. This is the single most effective anti-forgetting technique — and it is the same mechanism as the SFT general-data mix, just at pretraining scale. Two, low-perplexity token filtering. Drop tokens the model finds high-perplexity — likely garbage — and tokens it finds trivially low-perplexity — already mastered, no signal. Train on the informative middle. The NeurIPS twenty-twenty-five finding formalized this: token-perplexity reduction mitigates catastrophic forgetting while improving domain adaptation. Three, small learning rate with warmup and cooldown. Large LRs move weights too fast and forget.

When CPT versus SFT. Do you need the model to know a substantial new domain? CPT, then SFT on top. Do you need the model to behave differently — format, style, tools? SFT. Almost always the right answer. Could RAG give the model the knowledge at inference instead? Use RAG. For most practitioners, RAG is better than CPT for knowledge. Reach for CPT only when you have a large domain corpus, a clear knowledge gap SFT cannot close, and the budget to do it right.

[SLIDE 14 — Anti-patterns]

Four anti-patterns to leave with.

First, SFT only on narrow data — forgetting. Training on pure domain or pure tool-use. The model becomes a specialist that forgot how to be a general assistant. Mitigation: mix general data. Prefer LoRA.

Second, ignoring mixture ratios. Dumping all available data in without balancing sources. The dominant source skews the model. Track the mix percentages explicitly. Eval general capability alongside domain lift.

Third, skipping eval. Shipping an SFT model without held-out eval, general benchmarks, or format-compliance checks. You cannot detect forgetting, mode collapse, or format leakage without eval. The three-axis triangle, every time.

Fourth, treating SFT as knowledge injection. Fine-tuning on documents expecting the model to learn the content. It memorizes surface patterns and fails out-of-distribution. SFT for behavior, RAG for knowledge, CPT rarely for genuine adaptation. The cardinal error of the field.

[SLIDE 15 — The lab: build an SFT mix]

The lab is the opener of pillar three. You build a two-thousand-sample SFT dataset by blending Magpie-synthesized general instructions with a domain subset — medical, legal, or security, your choice. You train it with TRL's SFTTrainer, reusing the FT eleven loop. You evaluate on the three axes — general capability, domain lift, format compliance — and you report the domain lift versus the base.

You will feel the mixture-ratio tradeoff with your own hands. And in the stretch goal, you provoke catastrophic forgetting on purpose — re-build with seventy percent domain and watch general capability degrade while domain lift rises. The gap between them is the forgetting, made visible. That imbalance is the lesson.

[SLIDE 16 — What you can now do]

You can now state what SFT can and cannot do — steers behavior, does not inject knowledge — and defend it with the thesis. You can build a high-quality SFT mixture at defensible ratios. You can diagnose the three failure modes — catastrophic forgetting, mode collapse, format leakage — and name the mitigation for each. You can decide when to stop at SFT versus escalate to DPO. And you can explain the modern post-training stack and place SFT as stage one, the foundation everything else builds on.

That is SFT, the baseline. Next, module FT thirteen: the DPO family and preferences. Now that you have the baseline, we move to the methods that refine it — when there is no single correct response, but a preference between plausible ones.

---

*End of module FT twelve. Duration: approximately forty minutes at one-hundred-forty words per minute.*