# Diagrams — Module FT12: SFT: The Baseline **Module**: FT12 — SFT: The Baseline **Diagram count**: 5 **Tool**: Mermaid (primary). Each diagram validated in [Mermaid Live Editor](https://mermaid.live). --- ## Diagram 1 — The Modern Post-Training Stack **Type**: Linear pipeline **Purpose**: The single diagram that places SFT in context — stage 1 of the three-stage modern alignment pipeline. Everything builds on top of it. **Reading the diagram**: Left to right. SFT establishes format and behavior. DPO refines preferences on top. GRPO sharpens reasoning on top of that. Each stage assumes the previous one is done. ```mermaid flowchart LR SFT["1. SFT\nformat · style · instruction-following\nthe baseline"] --> DPO["2. DPO\npreferences: better vs worse\n(Module FT13)"] DPO --> GRPO["3. GRPO\nreasoning on verifiable rewards\n(Module FT14, optional)"] BASE(["the base model\n(Framework layer 1+2)"]) -.->|"steer"| SFT style SFT fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4 style DPO fill:#14141f,stroke:#5eead4,color:#e4e4e8 style GRPO fill:#14141f,stroke:#5eead4,color:#e4e4e8 style BASE fill:#08080c,stroke:rgba(94,234,212,0.3),color:#9494a0 ``` --- ## Diagram 2 — The SFT Mix Ratios **Type**: Proportional / comparison **Purpose**: The single diagram that shows what a defensible SFT dataset blend looks like. The numbers are a starting point; the principle — the mix encodes what you want the model to be — is the part to internalize. **Reading the diagram**: The stacked bar is a 100% SFT dataset. Each band is a source and its share. Too much of any one band skews the model in a predictable direction (annotated at right). ```mermaid flowchart TD subgraph Mix["A defensible SFT mix (by share)"] direction TB GEN["General instruction-following — 40-50%\nMagpie (FT05) / teacher distillation\nthe substrate"] DOM["Domain examples — 30-40%\nyour actual use cases\nwhere domain lift comes from"] TOOL["Tool-use formatting — 5-10%\nexact tool-call schema\n(FT07 template hygiene)"] SAFE["Safety calibration — 5-10%\nrefusals + compliance\nbalance matters"] end Mix -.->|"too much general"| SKEW1["domain lift evaporates\nmodel stays a generic assistant"] Mix -.->|"too much domain"| SKEW2["catastrophic forgetting\nloses general capability"] Mix -.->|"too much tool-use"| SKEW3["format leaks into everything\ntool calls on plain questions"] Mix -.->|"too much safety"| SKEW4["refusal-happy\ndeclines reasonable requests"] style GEN fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style DOM fill:#14141f,stroke:#5eead4,color:#e4e4e8 style TOOL fill:#14141f,stroke:#5eead4,color:#e4e4e8 style SAFE fill:#14141f,stroke:#5eead4,color:#e4e4e8 style Mix fill:#08080c,stroke:rgba(94,234,212,0.2),color:#9494a0 style SKEW1 fill:#14141f,stroke:#f0a868,color:#f0a868 style SKEW2 fill:#14141f,stroke:#f08080,color:#f08080 style SKEW3 fill:#14141f,stroke:#f0a868,color:#f0a868 style SKEW4 fill:#14141f,stroke:#f0a868,color:#f0a868 ``` --- ## Diagram 3 — The Three SFT Failure Modes **Type**: Diagnostic chart **Purpose**: The three ways an SFT job goes wrong, each with a signature (what you observe) and a mitigation (what you do about it). Learn to name them on sight. **Reading the diagram**: Each row is a failure mode. The left column is the cause, the middle is the signature in eval, the right is the fix. The FT00 thesis runs underneath: all three are steering failures, not knowledge failures. ```mermaid flowchart TD F1["FAILURE 1 — Catastrophic forgetting\ncause: too-narrow data (pure domain/tool/safety)"] F1 --> S1["signature: domain metrics up\ngeneral benchmarks down"] S1 --> M1["mitigation: mix general data (40-50%)\nprefer LoRA (low-rank forgets less)"] F2["FAILURE 2 — Mode collapse\ncause: low-diversity data"] F2 --> S2["signature: every output looks the same\nregardless of input"] S2 --> M2["mitigation: diversity filtering (FT06)\nmix sources · don't over-train"] F3["FAILURE 3 — Format leakage\ncause: template bugs (FT07)"] F3 --> S3["signature: fluent output the harness\ncan't parse (wrong JSON/tool fields)"] S3 --> M3["mitigation: template hygiene\nuse the tokenizer chat template\ntest the round-trip"] style F1 fill:#14141f,stroke:#f08080,color:#f08080 style F2 fill:#14141f,stroke:#f0a868,color:#f0a868 style F3 fill:#14141f,stroke:#f0a868,color:#f0a868 style S1 fill:#08080c,stroke:rgba(240,128,128,0.3),color:#9494a0 style S2 fill:#08080c,stroke:rgba(240,168,104,0.3),color:#9494a0 style S3 fill:#08080c,stroke:rgba(240,168,104,0.3),color:#9494a0 style M1 fill:#14141f,stroke:#82e0aa,color:#82e0aa style M2 fill:#14141f,stroke:#82e0aa,color:#82e0aa style M3 fill:#14141f,stroke:#82e0aa,color:#82e0aa ``` --- ## Diagram 4 — CPT vs SFT Decision **Type**: Decision tree **Purpose**: The judgment call between SFT (behavior) and continued pretraining (knowledge). CPT is the de-emphasized sidebar — reach for it rarely, and only when the answer to the knowledge question is genuinely "yes." **Reading the diagram**: Start at the top. The first question is the thesis test: behavior or knowledge? Most paths end at SFT or RAG. CPT is the rare branch, and even then it is followed by SFT. ```mermaid flowchart TD Start["You want to change the model"] --> Q1{behavior or knowledge?} Q1 -->|"behavior: format, style, tools, refusal"| SFT["SFT — the baseline\n(this module)"] Q1 -->|"knowledge: new domain the model doesn't know"| Q2{can RAG provide it at inference?} Q2 -->|"yes — usually the answer"| RAG["RAG\nretrieve at inference, don't train"] Q2 -->|"no — must be in the weights"| Q3{substantial new knowledge?} Q3 -->|"no — surface patterns only"| SFT Q3 -->|"yes — shift the knowledge distribution"| CPT["CPT — then SFT on top\nthe sidebar (12.6)"] CPT --> REPLAY["mixture-of-source training\ndomain : general 1:1 to 2:1\nlow-perplexity token filter\nsmall LR + warmup/cooldown"] style Start fill:#14141f,stroke:rgba(255,255,255,0.12),color:#e4e4e8 style SFT fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#5eead4 style RAG fill:#14141f,stroke:#5eead4,color:#e4e4e8 style CPT fill:#14141f,stroke:#f0a868,color:#f0a868 style REPLAY fill:#08080c,stroke:rgba(240,168,104,0.3),color:#9494a0 style Q1 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8 style Q2 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8 style Q3 fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8 ``` --- ## Diagram 5 — The SFT-to-Preference Escalation **Type**: Branching decision **Purpose**: When to stop at SFT versus escalate to DPO (FT13) or GRPO (FT14). The rule: SFT for a single correct response; DPO for better/worse preferences; GRPO for verifiable-reward reasoning. **Reading the diagram**: After SFT (stage 1), evaluate the remaining quality gap. The shape of the gap determines whether you stop, escalate to DPO, or escalate to GRPO. ```mermaid flowchart TD SFT["SFT done — format + behavior established"] --> Q{is the quality gap closed?} Q -->|"yes — single correct response achieved"| SHIP["ship. SFT was enough."] Q -->|"no — gap is about better/worse preferences"| DPO["escalate to DPO (FT13)\npreferred vs rejected triples\nrefine tone, verbosity, helpfulness"] Q -->|"no — gap is about reasoning with checkable answers"| GRPO["escalate to GRPO (FT14)\nRL on verifiable rewards\nsharpen math, code, logic"] DPO --> RE["re-eval the gap\niterate or ship"] GRPO --> RE RE -->|"closed"| SHIP style SFT fill:#14141f,stroke:#5eead4,stroke-width:1.5px,color:#e4e4e8 style SHIP fill:#14141f,stroke:#82e0aa,color:#82e0aa style DPO fill:#14141f,stroke:#5eead4,color:#5eead4 style GRPO fill:#14141f,stroke:#5eead4,color:#5eead4 style RE fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8 style Q fill:#08080c,stroke:rgba(94,234,212,0.3),color:#e4e4e8 ``` --- ## Validation notes - All five diagrams use the course design system colors: `#14141f` panel fill, `#5eead4` accent for primary, `#82e0aa` (ok) / `#f08080` (danger) / `#f0a868` (warn) for semantic emphasis, `rgba(255,255,255,0.12)` and `rgba(94,234,212,0.3)` for secondary borders, `#e4e4e8` / `#9494a0` for text. - Paste each into [Mermaid Live Editor](https://mermaid.live) to render. All use stable Mermaid syntax (`flowchart` with `TD`/`LR`, `subgraph`, dotted `-.->` links) supported in current Mermaid (v10.4+). - Diagram 1 uses a dotted link for the base-model input to emphasize that SFT is the steer applied to the base. Diagram 2 uses dotted links to annotate the failure modes of imbalance — these are annotations, not flow. Diagram 5's re-eval loop (`RE`) closes back to the ship decision to show iteration. - For the slide deck (artifact 03), these are rendered as static captures from Mermaid Live, inlined into reveal.js.