{
  "module": "FTDD-07 — DeepSeek-R1",
  "course": "3 — LLM Fine-Tuning Masterclass",
  "version": "1.0.0",
  "duration_minutes": 45,
  "total_questions": 10,
  "bloom_distribution": {
    "target": "40% recall / 30% application / 30% analysis",
    "actual": { "recall": 4, "application": 3, "analysis": 3 }
  },
  "passing_score_percent": 70,
  "questions": [
    {
      "id": "Q01", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What did R1-Zero use as its reward signal during RL training?",
      "options": [
        "A learned reward model trained on human preference pairs.",
        "A large language model acting as a judge to score response quality.",
        "Verifiable rewards only: math correctness, code execution pass/fail, LeetCode verdicts.",
        "Human raters scoring each response on a Likert scale."
      ],
      "answer_index": 2,
      "rationale": "R1-Zero used verifiable rewards only — math correctness, code execution, LeetCode verdicts. No reward model, no judge, no human preferences. This became a field-wide lesson: if you can verify, do not approximate with a learned judge."
    },
    {
      "id": "Q02", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What were the four stages of the R1 pipeline, in order?",
      "options": [
        "Pretraining → SFT → DPO → GRPO",
        "Cold-start SFT → reasoning RL → rejection-sampling SFT → final RL",
        "Rejection sampling → cold-start SFT → final RL → reasoning RL",
        "Reasoning RL → cold-start SFT → final RL → rejection sampling"
      ],
      "answer_index": 1,
      "rationale": "(1) Cold-start SFT anchors format. (2) Reasoning RL (GRPO) scales capability. (3) Rejection-sampling SFT cleans the data into ~800K traces. (4) Final RL aligns the whole distribution. Each stage fixes a failure of the one before."
    },
    {
      "id": "Q03", "bloom": "recall", "type": "multiple_choice",
      "prompt": "How many dense models were distilled from R1, and by what training method?",
      "options": [
        "Two MoE models, trained with GRPO on verifiable rewards.",
        "Six dense models, trained SFT-only on ~800K curated traces (no RL on the students).",
        "One model, trained with DPO on preference pairs.",
        "Six dense models, each trained with a full GRPO reasoning RL loop."
      ],
      "answer_index": 1,
      "rationale": "Six dense models (Qwen 1.5B/7B/14B/32B, Llama 8B/70B), trained by SFT-only on the ~800K curated traces from Stage 3. No GRPO, no reward model, no RL loop on the students. This is the result that made CoT distillation standard."
    },
    {
      "id": "Q04", "bloom": "recall", "type": "multiple_choice",
      "prompt": "R1-Zero was 'a proof, not a product.' What were its main shortcomings?",
      "options": [
        "It could not reason at all; it only copied the prompt.",
        "It required enormous supervised datasets that were expensive to produce.",
        "Messy output: language mixing mid-chain, run-on chains, correct-but-ugly output, sometimes no clean final answer.",
        "It only worked on math, not code or logic."
      ],
      "answer_index": 2,
      "rationale": "R1-Zero was benchmark-competitive but messy: mixed Chinese/English mid-chain, run-on chains, correct intermediate work that did not converge to a clean final answer. R1's cold-start SFT stage exists specifically to fix this — it builds on the emergence, not contradicts it."
    },
    {
      "id": "Q05", "bloom": "application", "type": "multiple_choice",
      "prompt": "A team has a strong reasoning teacher model and wants to build a reasoning student. Compute is limited. Per R1's evidence, what is the correct first step?",
      "options": [
        "Run a full GRPO reasoning RL loop on the student with verifiable rewards.",
        "Train a reward model from the teacher's outputs, then run DPO on the student.",
        "SFT-only distillation from the teacher's curated traces. RL on the student is marginal per R1's ablation.",
        "Continued pretraining of the student on the teacher's reasoning corpus."
      ],
      "answer_index": 2,
      "rationale": "R1's own ablation shows SFT-only distillation captures most of the gain and RL on the student gives marginal returns. The teacher's traces are already a near-optimal steering signal. Distill first; reach for GRPO only if there is a verifiable-reward domain the traces under-cover and the compute to spend."
    },
    {
      "id": "Q06", "bloom": "application", "type": "multiple_choice",
      "prompt": "You are designing an RL training run for reasoning on a code-generation task. Following R1's lesson, what reward signal should you prefer?",
      "options": [
        "A large language model judging the code quality on a rubric.",
        "Execution-based verification: run the code against test cases and reward pass/fail.",
        "Human raters scoring code readability.",
        "A reward model trained on preference pairs of good and bad code."
      ],
      "answer_index": 1,
      "rationale": "R1's lesson: verifiable rewards beat reward models for reasoning RL. Code execution against test cases is a ground-truth check. If you can verify, do not approximate with a learned judge. Subjective signals (readability) are a complement, not the primary reward."
    },
    {
      "id": "Q07", "bloom": "application", "type": "multiple_choice",
      "prompt": "Stage 3 of the R1 pipeline (rejection-sampling SFT) is described as 'load-bearing for the rest of the field.' Why?",
      "options": [
        "It is where the model learns to reason for the first time.",
        "It is where the cold-start format is anchored.",
        "It produces the ~800K curated clean traces that become the raw material for distillation into the six dense students.",
        "It is where the final RL alignment happens."
      ],
      "answer_index": 2,
      "rationale": "Stage 3 takes the Stage-2 RL'd model, generates many candidates, filters for correct and readable traces. This curated set is the source of the 800K traces used to distill R1-Distill-Qwen-32B and the other students. Without Stage 3, there is no clean distillation dataset."
    },
    {
      "id": "Q08", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Why is R1-Zero's emergence of chain-of-thought the strongest evidence for the course thesis that reasoning is steering, not teaching?",
      "options": [
        "Because it shows that GRPO is more efficient than SFT.",
        "Because if reasoning were a knowledge problem, GRPO on a base with zero reasoning traces could not produce it. The base already had the capability (saw it in pretraining); RL only redirected probability mass to make reasoning pathways the chosen behavior.",
        "Because it proves that large models are always better than small ones.",
        "Because it shows that reward models are unnecessary for all tasks."
      ],
      "answer_index": 1,
      "rationale": "The logic: if reasoning required knowledge injection, you would need supervised demonstrations or continued pretraining. R1-Zero had neither — only RL on a base. The fact that reasoning emerged means the capability was latent; RL steered it to the surface. This is the steering thesis at scale."
    },
    {
      "id": "Q09", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "R1 reports that applying RL on top of the distilled students gave only marginal gains. What is the correct interpretation, and what does it imply for practitioners?",
      "options": [
        "RL is fundamentally broken and should never be used on students.",
        "The teacher's curated traces were already a near-optimal steering signal, showing the student the high-reward trajectories directly. RL on top could only marginally refine what traces had installed. Implication: distill first; RL the student only with a verifiable domain and compute to spare.",
        "The students were too small to benefit from RL.",
        "The distillation method was wrong and should have used DPO instead."
      ],
      "answer_index": 1,
      "rationale": "The traces carried the reasoning behavior. RL on the student could only refine the edges, at high compute cost. This is counterintuitive if you assume RL is always necessary for reasoning. The practitioner takeaway: SFT distillation is the default; RL on the student is the exception, reserved for verifiable-reward gaps."
    },
    {
      "id": "Q10", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "How does the distillation result (small models inheriting reasoning via SFT-only on teacher traces) support the steering thesis from the opposite direction of R1-Zero?",
      "options": [
        "It doesn't — distillation and RL are unrelated evidence.",
        "R1-Zero shows reasoning emerges from RL on a base (capability was latent). Distillation shows reasoning transfers via SFT on traces (small bases already had latent capability; traces steered them to express it). Both directions show the student/teacher inherits behavior, not new ability — the hallmark of steering.",
        "It shows that small models cannot reason and need full RL.",
        "It disproves the thesis by showing reasoning can be copied."
      ],
      "answer_index": 1,
      "rationale": "The two results bracket the thesis. R1-Zero: RL activates latent reasoning in a base (no traces needed). Distillation: SFT on traces transfers reasoning to small bases (the capability was already there; traces steer expression). If reasoning were knowledge, neither would work — emergence would need teaching, and transfer would be impossible. Both work, so reasoning is steering."
    }
  ]
}