{
  "module": "FTDD-10 — distilabel",
  "course": "3 — LLM Fine-Tuning Masterclass",
  "version": "1.0.0",
  "duration_minutes": 45,
  "total_questions": 10,
  "bloom_distribution": {
    "target": "40% recall / 30% application / 30% analysis",
    "actual": { "recall": 4, "application": 3, "analysis": 3 }
  },
  "passing_score_percent": 70,
  "questions": [
    {
      "id": "Q01", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What is distilabel's role in the Argilla ecosystem?",
      "options": [
        "It is the model-training framework (the complement to TRL's data side).",
        "It is the synthetic data pipeline framework — the data-construction complement to TRL's training side. TRL trains the model; distilabel builds the data the model trains on.",
        "It is the human-labeling platform for annotation.",
        "It is a serving engine for production inference."
      ],
      "answer_index": 1,
      "rationale": "distilabel is Argilla's synthetic data pipeline framework: generation, evolution, filtering, and formatting of synthetic SFT and preference datasets. It is the data-construction complement to TRL's training side. The division of labor: Argilla Datasets for human labeling, distilabel for synthetic generation, both feed TRL."
    },
    {
      "id": "Q02", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What is Magpie-style generation?",
      "options": [
        "Hand-authoring thousands of diverse seed instructions.",
        "An aligned (instruct-tuned) model, given only a pre-query template (system prompt + start of user turn, NO instruction), generates a plausible instruction as its first output. The model prompts itself.",
        "Using a reward model to rank responses.",
        "A deduplication technique based on sentence embeddings."
      ],
      "answer_index": 1,
      "rationale": "Magpie exploits the fact that instruct-tuning shaped the model's output distribution to complete user turns realistically. Given the start of a user turn, it generates a plausible instruction. Sampling many times with temperature yields a diverse, self-prompted instruction set without hand-authored seeds. (arXiv:2406.08464.)"
    },
    {
      "id": "Q03", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What is the LLM-as-judge step in a distilabel pipeline?",
      "options": [
        "A human annotator scoring each response by hand.",
        "An LLM scoring each generated response on dimensions like correctness, helpfulness, relevance, conciseness. The scores enable threshold filtering.",
        "A training algorithm that penalizes wrong responses.",
        "A deduplication method using embeddings."
      ],
      "answer_index": 1,
      "rationale": "The judge step uses an LLM (often stronger than the generator, or the same model in a different role) to score each response. Threshold filtering then keeps high-scoring examples and drops low-scoring ones. This is the quality gate that converts a noisy candidate pool into a curated dataset."
    },
    {
      "id": "Q04", "bloom": "recall", "type": "multiple_choice",
      "prompt": "What does a preference dataset for DPO contain per example?",
      "options": [
        "One prompt and one ideal response.",
        "One prompt and two responses labeled chosen and rejected.",
        "A prompt, a reward score, and a confidence interval.",
        "A system prompt and a single completion."
      ],
      "answer_index": 1,
      "rationale": "Preference data is paired: one prompt, two responses labeled chosen and rejected. The model learns to steer toward the chosen distribution and away from the rejected. distilabel builds these by generating multiple responses per prompt and ranking them via a judge or reward model, emitting prompt/chosen/rejected columns that TRL's DPOTrainer consumes."
    },
    {
      "id": "Q05", "bloom": "application", "type": "multiple_choice",
      "prompt": "A team needs a DPO dataset for a custom domain but has no human-labeled preference data. What is the correct end-to-end approach with distilabel?",
      "options": [
        "Hand-label thousands of preference pairs, which is the only reliable method.",
        "Use distilabel to generate domain prompts, generate K responses each, rank via judge/reward model, emit chosen/rejected pairs, then train with TRL's DPOTrainer. No human-labeled preference data required.",
        "Skip the judge and train DPO directly on raw generated pairs.",
        "Use only Magpie seeds without ranking, since DPO does not need preference ordering."
      ],
      "answer_index": 1,
      "rationale": "distilabel builds synthetic preference datasets end-to-end: generate/curate prompts, generate K responses per prompt, rank via judge or reward model, emit chosen/rejected pairs in DPO format, train with TRL's DPOTrainer. distilabel handles construction; TRL handles training; they meet at the dataset contract. The judge/ranking step is essential — DPO needs genuine preference ordering."
    },
    {
      "id": "Q06", "bloom": "application", "type": "multiple_choice",
      "prompt": "You generated a large candidate pool via Magpie + Evol-Instruct and are about to train SFT on all of it. What step are you missing, and why does it matter?",
      "options": [
        "Nothing — training on all generated data maximizes coverage.",
        "The judge-filter + dedup quality gate. Without it, noise dominates and the model steers into the average of the garbage. The judge threshold sets the dataset ceiling.",
        "You need to switch from SFT to DPO.",
        "You need to re-run Magpie with lower temperature."
      ],
      "answer_index": 1,
      "rationale": "Generation produces volume, not quality. The judge step scores each example; threshold filtering keeps the good and drops the bad. Dedup removes near-duplicates. Without these, the candidate pool's noise dominates training. Per the thesis (data matters more than algorithm), the judge threshold is the single highest-leverage knob in a synthetic pipeline."
    },
    {
      "id": "Q07", "bloom": "application", "type": "multiple_choice",
      "prompt": "Your distilabel preference pipeline is emitting pairs, but your DPO model is not improving much. Inspecting the pairs, you find the chosen and rejected responses are nearly identical in quality. What is wrong, and how do you fix it?",
      "options": [
        "The judge model is too strong; use a weaker one.",
        "The pairs are ambiguous (score gap too small). Filter to keep only pairs with a meaningful quality difference between chosen and rejected — they carry the most preference signal.",
        "You need to generate fewer responses per prompt.",
        "Switch from DPO to SFT."
      ],
      "answer_index": 1,
      "rationale": "Too-small a score gap means the chosen and rejected are nearly equal quality — the pair teaches the model little about preference. The useful pairs have a meaningful (but not overwhelming) quality difference. distilabel's pipeline includes a step to filter out both ambiguous (small gap) and trivial (huge gap) pairs, keeping those that carry the most learning signal."
    },
    {
      "id": "Q08", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Why is the judge threshold described as 'the single highest-leverage knob in a synthetic pipeline,' and how does this connect to the course thesis?",
      "options": [
        "Because it determines the generation backend (vLLM vs transformers).",
        "Because per the thesis ('data matters more than algorithm'), the dataset quality sets the ceiling on the trained model, and the judge threshold is what converts a noisy candidate pool into a curated dataset. A lenient judge admits noise; a well-calibrated judge is the difference between steering well and steering into a wall.",
        "Because it controls the learning rate during training.",
        "Because it is the only step that requires a GPU."
      ],
      "answer_index": 1,
      "rationale": "The thesis says data matters more than algorithm. The judge threshold directly determines dataset quality: it is the gate that separates curated training data from raw generation noise. Since dataset quality sets the trained-model ceiling, the judge threshold is the highest-leverage decision in the pipeline. A great dataset trains well with a simple algorithm; a noisy dataset fails regardless of algorithm."
    },
    {
      "id": "Q09", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Why must an LLM judge be calibrated and validated rather than trusted blindly?",
      "options": [
        "Because LLM judges are always wrong.",
        "Because an LLM judge has its own biases (verbosity, sycophancy, domain miscalibration) that can systematically distort scores. Like any evaluation, it must be spot-checked against human judgment and calibrated on a held-out sample before scaling.",
        "Because judges only work with proprietary models.",
        "Because calibration requires retraining the judge model."
      ],
      "answer_index": 1,
      "rationale": "An LLM judge is a scalable approximation of human curation, not ground truth. It has biases: verbosity (prefers longer), sycophancy (agrees with generator), domain miscalibration (wrong on specialized correctness). The discipline is the same as any evaluation: spot-check decisions, calibrate the threshold on a held-out sample, watch for systematic bias. A miscalibrated judge admits systematic noise no downstream algorithm will fix."
    },
    {
      "id": "Q10", "bloom": "analysis", "type": "multiple_choice",
      "prompt": "Why does combining Magpie (for diversity) and Evol-Instruct (for complexity) still require the judge-filter step before training?",
      "options": [
        "Because Magpie and Evol-Instruct are incompatible and cannot run together.",
        "Because Magpie and Evol-Instruct produce volume and diversity/complexity, but NOT quality guarantees. Evol-Instruct increases the variance of quality (harder instructions may be incoherent). The judge filter is what converts the large, diverse, multi-difficulty candidate pool into a curated dataset where each example is correct and useful.",
        "Because the judge is required to run Magpie.",
        "Because Evol-Instruct cannot run without a judge model."
      ],
      "answer_index": 1,
      "rationale": "Magpie gives diversity (self-prompted seeds); Evol-Instruct gives complexity (stretched difficulty). Neither guarantees quality. Evol-Instruct specifically increases quality variance — an evolved harder instruction may be nonsensical. The judge filter is the independent quality gate that must run on ALL generated/evolved examples before training. Volume + diversity + complexity still needs curation to become a good dataset."
    }
  ]
}