{ "module": "FT00 — The Steering Stack", "course": "3 — LLM Fine-Tuning Masterclass", "version": "1.0.0", "duration_minutes": 45, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis-design", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the central thesis of Course 3?", "options": [ "Fine-tuning injects knowledge into the model's weights, making it smarter.", "Fine-tuning steers behavior; it does not teach knowledge. The model steers — the harness bounds.", "The harness is 98.4% of the system; the model is 1.6%.", "LoRA is always superior to full fine-tuning for production use." ], "answer_index": 1, "rationale": "The thesis is 'fine-tuning steers behavior; it does not teach knowledge. The model steers — the harness bounds.' Option C is Course 1's thesis (the complement, not the contradiction). The other options are common misconceptions." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "Name the five layers of the Steering Stack, bottom to top.", "options": [ "Data → Adapter → Steer → Export → Boundary", "Base → Adapter → Steer → Export → Boundary", "Base → LoRA → SFT → Quantize → Harness", "Pretrain → SFT → DPO → GRPO → Deploy" ], "answer_index": 1, "rationale": "Layer 1 Base (pretrained weights), Layer 2 Adapter (LoRA/DoRA), Layer 3 Steer (SFT/DPO/GRPO/abliteration), Layer 4 Export (quant+serve), Layer 5 Boundary (the harness)." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "What single property makes the Steering Stack tractable?", "options": [ "All layers must be trained simultaneously for correctness.", "Each layer is independent and must be chosen before training begins.", "You can swap any layer above the base without touching the one below.", "The base must be re-pretrained whenever the adapter changes." ], "answer_index": 2, "rationale": "The swappability property: you can swap any layer above the base without touching the one below. This is why adapters are swappable, why abliteration works without retraining, why you quantize after training, why the harness is model-agnostic." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "You want your model to always respond in strict JSON with a specific schema. Which layer and technique?", "options": [ "Layer 1 — choose a different base model with better JSON pretraining.", "Layer 3 — SFT (steering format). The model already knows JSON; you are making the format reliable.", "Layer 4 — quantize to a format that enforces JSON output.", "Layer 5 — add a harness policy gate that rejects non-JSON output." ], "answer_index": 1, "rationale": "This is a format/instruction-following problem — a classic STEERING problem (the base already produces JSON unreliably). SFT at Layer 3 makes it reliable. (Layer 5 as a post-hoc validator is a reasonable complement, but the model-side fix is SFT.)" }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "Your company wants the model to know its proprietary pricing algorithm so it can answer customer questions accurately. What is the correct intervention?", "options": [ "Fine-tune (SFT) on the pricing algorithm documentation — the model will learn it.", "Continued pretraining on the pricing corpus — the only way to inject the knowledge.", "RAG (retrieve the algorithm doc and put it in context). Fine-tuning is for behavior, not knowledge.", "Abliterate the model so it doesn't refuse pricing questions." ], "answer_index": 2, "rationale": "This is a KNOWLEDGE gap — the base doesn't know the proprietary algorithm. Fine-tuning would memorize surface patterns and fail out-of-distribution. RAG is the correct answer for knowledge. (You might SFT for the BEHAVIOR of citing the algorithm, but the knowledge itself is retrieved.)" }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "You prompt the base model to write exploit code for an authorized pentest, and it refuses. What layer and technique addresses this?", "options": [ "Layer 1 — use a base model that wasn't refusal-trained.", "Layer 3 — abliteration or DPO toward compliance. The capability is there; the refusal is the problem.", "Layer 4 — serve the model locally so it doesn't refuse.", "Layer 5 — remove the harness's safety gates." ], "answer_index": 1, "rationale": "This is a PREFERENCE/alignment problem (the three-outcome test, outcome 2): the model refuses but the capability is clearly there. Abliteration (FT17) or DPO-toward-compliance (FT18) at Layer 3 addresses it. Layer 5 is the complementary boundary — it doesn't fix the model's refusal, it bounds what the un-refused model may do." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "You quantize your fine-tuned model from FP16 to GGUF Q4_K_M and the output quality drops slightly. What does this tell you about where the problem lives in the stack?", "options": [ "Layer 3 — your fine-tuning was insufficient; retrain with more data.", "Layer 2 — your LoRA adapter was the wrong rank.", "Layer 4 — quantization is downstream of training and introduced the loss. Try a different quant format or precision.", "Layer 1 — the base model is too small for Q4." ], "answer_index": 2, "rationale": "Quantization is Layer 4, downstream of training. A quality drop after quantization is a Layer 4 problem — try AWQ instead of GGUF, or Q8 instead of Q4. The model's learned behavior (Layer 3) is unchanged; only the compression introduced loss." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "Using the three-outcome test: you prompt the base model to translate to a language NOT in its pretraining mix, and it produces gibberish. Which outcome and which intervention?", "options": [ "Outcome 1 (unreliable behavior) — fine-tune with SFT.", "Outcome 2 (refusal) — abliterate.", "Outcome 3 (foreign domain / knowledge gap) — different base, CPT, or (usually) not solvable by fine-tuning.", "Outcome 4 — increase the LoRA rank." ], "answer_index": 2, "rationale": "This is outcome 3: a genuinely foreign domain. The model has no knowledge of the language. No amount of steering (fine-tuning) will reliably fix this — you need a base that knows the language, CPT on a large corpus of that language, or a different approach entirely." }, { "id": "Q09", "bloom": "analysis", "type": "multiple_choice", "prompt": "A LoRA adapter represents 0.3% of a model's parameters, yet it changes the model's output format reliably after training. Why does this support 'steering not teaching'?", "options": [ "It shows that LoRA is more efficient than full FT because it updates fewer params.", "If fine-tuning were injecting knowledge, you would need a large fraction of params. The fact that 0.3% suffices shows the change is a low-rank steering vector, not a knowledge injection.", "It proves the base model was undertrained and needed only a small boost.", "It shows that adapters are always better than full FT for production." ], "answer_index": 1, "rationale": "The intrinsic-dimension argument: useful fine-tuning changes live in a low-rank subspace. If fine-tuning moved large amounts of knowledge, you'd need large param updates. 0.3% sufficing is direct evidence that the operation is steering (low-rank), not teaching (which would require high-rank updates)." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "Shuttleworth et al. (arXiv:2410.21228) found that LoRA and full-parameter fine-tuning produce structurally different weight matrices. What is the implication for the steering thesis?", "options": [ "It disproves the thesis — full FT and LoRA reach the same behavior via the same weights.", "It supports the thesis: LoRA finds a low-rank steering solution; full FT finds a different (higher-rank) solution. They reach similar behavior via different geometry, consistent with steering being a distinct low-rank operation.", "It means LoRA is always wrong and full FT is always right.", "It means you should never merge a LoRA adapter into the base." ], "answer_index": 1, "rationale": "The structural non-equivalence supports the steering thesis. If fine-tuning were a single well-defined knowledge-injection operation, LoRA and full FT would converge to similar weights. They don't — LoRA finds a low-rank path, full FT a higher-rank path, both reaching similar behavior. This is what you'd expect if the task is steering (multiple solutions exist) rather than knowledge injection (one correct answer)." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why does abliteration (removing refusal) measurably degrade math reasoning (GSM8K down up to 18.8pp)?", "options": [ "Because abliteration retraining overwrites the math weights.", "Because the refusal direction in the residual stream is entangled with other capabilities. Deleting it to steer away from refusal nudges the entangled capabilities, including reasoning.", "Because the abliteration tools are buggy.", "Because uncensored models are inherently worse at math." ], "answer_index": 1, "rationale": "Abliteration deletes a direction in the residual stream. That direction is not a clean 'refusal-only' axis — it is entangled with other capabilities. Steering away from refusal (deleting the direction) nudges the entangled capabilities, including math reasoning. This is a direct consequence of 'steering' — you redirect probability mass, and the directions are not orthogonal to everything else." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "How does Course 3's thesis ('the model steers; the harness bounds') relate to Course 1's ('the model is 1.6%; the harness is 98.4%')?", "options": [ "They contradict each other — one says the model matters, the other says it doesn't.", "They are complements describing the same system from opposite ends. Course 1 says the harness is most of the system; Course 3 zooms into the 1.6% (the model) and clarifies what you can and cannot change about it.", "Course 3 supersedes Course 1.", "They are unrelated courses." ], "answer_index": 1, "rationale": "Complement, not contradiction. Course 1 says the harness is 98.4% of the system. Course 3 zooms into the 1.6% (the model) and asks what you can change about it — and answers 'its behavior (steering), not its knowledge.' Together they describe the whole system: a steered model inside a bounding harness." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "A team runs GRPO (a sophisticated RL algorithm) on a poorly-curated, contaminated dataset. The resulting model is worse than the base. What does the steering thesis say about why?", "options": [ "GRPO is the wrong algorithm; they should have used DPO.", "The steering wheel (dataset) was bad. GRPO steered precisely in the direction the data pointed — bad data = bad direction, no matter how good the optimizer. Data (Pillar 1) matters more than algorithm.", "The base model was too small for GRPO.", "They quantized too aggressively after training." ], "answer_index": 1, "rationale": "The 'steering without a steering wheel' anti-pattern. The dataset IS the steering wheel. A sophisticated optimizer on bad data steers you precisely into a wall. This is why Pillar 1 (Data) comes before Pillars 2 (PEFT) and 3 (Alignment) in the course." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "An organization wants to deploy an uncensored model for a tool-use agent. Why is 'deploy it inside an eval'd harness' non-negotiable, per the thesis?", "options": [ "Because the harness makes the model faster.", "Because the harness reduces VRAM usage.", "Because steering (Layer 3) changes what the model DOES, not what it MAY do. The boundary between 'does' and 'may' is the harness (Layer 5). An uncensored model in a weak harness is strictly more dangerous than a refusal-trained model in a weak harness.", "Because it is legally required in all jurisdictions." ], "answer_index": 2, "rationale": "The synthesis principle: uncensoring changes what the model does (it won't refuse), but not what it may do (that's the harness's job). An uncensored model in a weak harness is strictly more dangerous — it will execute whatever it's asked, including the dangerous things. The harness provides the policy gates the model cannot provide itself. (FT23.)" }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "You swap a LoRA adapter on a base model and confirm the base weights are byte-identical before and after. What property of the Steering Stack does this demonstrate, and why does it matter?", "options": [ "The swappability property: you can swap a layer above the base without touching the base. This matters because it means adapters are hot-swappable, bases are reusable across many adapters, and training one adapter never corrupts the base for other uses.", "The steering property: fine-tuning changes behavior.", "The modularity of the dataset pipeline.", "The quantization-independence of training." ], "answer_index": 0, "rationale": "This is the swappability property demonstrated empirically (the FT00 lab). The base is byte-identical because Layer 2 (Adapter) sits on top of Layer 1 (Base) and detaches without affecting it. This is what makes adapters hot-swappable at inference and bases reusable across many adapters — the foundational modularity of the stack." } ] }