{ "module": "CAP2 — The Calibrated Uncensored Agent", "course": "3 — LLM Fine-Tuning Masterclass", "version": "1.0.0", "duration_minutes": 45, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis-design", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the synthesis principle that Capstone 2 operationalizes?", "options": [ "Fine-tuning steers behavior; it does not teach knowledge.", "Uncensor the model so it executes; harness the model so it executes only what it should.", "The model is 1.6% of an agent; the harness is the other 98.4%.", "Always use abliteration over DPO for compliance steering." ], "answer_index": 1, "rationale": "The synthesis principle: uncensor the model so it executes (steering toward compliance, low refusal on legitimate), harness the model so it executes only what it should (policy gates veto illegitimate). Option A is the course thesis (FT00); option C is Course 1's thesis. This capstone is where steering meets bounding." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "Name the three success criteria for Capstone 2.", "options": [ "Model size, quantization level, and inference latency.", "Low refusal rate on legitimate prompts, high veto rate on illegitimate prompts, and the trade-off-defense document.", "Training time, dataset size, and GPU utilization.", "Abliteration speed, DPO fidelity, and harness complexity." ], "answer_index": 1, "rationale": "The three criteria: (1) low refusal rate on legitimate prompts (steering's benefit, materially lower than base); (2) high veto rate on illegitimate prompts (harness's strength, ~100%); (3) the trade-off-defense document (justifies the combination)." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the honest caveat about abliteration and capability cost?", "options": [ "Abliteration has no capability cost — it only removes refusal.", "Abliteration measurably degrades capability. GSM8K moves from +1.5pp to -18.8pp (Dec 2025 study). The refusal direction is entangled with other capabilities.", "Abliteration always improves capability by removing safety overhead.", "The capability cost is the same regardless of the base model." ], "answer_index": 1, "rationale": "Abliteration is not a clean operation. The refusal direction in the residual stream is entangled with other capabilities (reasoning, math). Steering away from refusal nudges them. The Dec 2025 study (arXiv:2512.13655) documents GSM8K from +1.5pp to -18.8pp. DPO-toward-compliance has smaller degradation." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "Your steered model refuses legitimate prompts at the same rate as the base (55% → 54%). The steering accomplished nothing. What does this indicate, and what should you do?", "options": [ "The steering worked; 54% is close enough to 55%.", "The steering accomplished nothing — you paid the capability cost for no benefit. Either the abliteration failed to remove the refusal direction, or the base wasn't strongly refusal-trained to begin with. Investigate the steering method; the refusal rate must be materially lower to justify the cost.", "You need a larger base model.", "You should quantize more aggressively." ], "answer_index": 1, "rationale": "The steering's justification is a measurable reduction in refusal on legitimate prompts. If it doesn't reduce refusal, it accomplished nothing, and you paid the capability cost for no benefit. Investigate: verify the abliteration actually edited the refusal direction, or consider DPO-toward-compliance if abliteration didn't take. The base may also not have been strongly refusal-trained." }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "Your harness policy gate vetoes 23 of 24 illegitimate calls (95.8%). The 24th slipped through. Is this acceptable?", "options": [ "Yes — 95.8% is a strong veto rate.", "No — combined with an uncensored model (no model-level refusal backstop), any leak is a path to executing a dangerous action. The veto rate must be effectively 100%. Find and fix the bug in the 24th case before deploying.", "Yes, if the missed call was low-severity.", "It depends on the reviewer's discretion." ], "answer_index": 1, "rationale": "A 95.8% gate is a gate with a bug, not a gate with tolerance. With an uncensored model, there is no model-level refusal backstop — the harness is the only boundary. Any leak is a path to executing a dangerous action. The veto rate must be effectively 100%; find and fix the bug." }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "You abliterated a model but never ran a general benchmark to measure the capability cost. What is the status of your trade-off defense?", "options": [ "Complete — the defense is about the steering choice, not the cost.", "Incomplete — the defense cannot justify a trade-off it didn't measure. You must run a general benchmark (GSM8K or equivalent) before and after steering and report the degradation. An unmeasured cost is an unknown cost; an unknown cost means you cannot honestly claim the combination is safe.", "Complete, because abliteration's cost is well-known from the literature.", "Incomplete only if the demo fails." ], "answer_index": 1, "rationale": "The capability cost must be measured, not assumed. Abliteration cost varies wildly (+1.5pp to -18.8pp) by tool and model. The defense document's job is to justify the trade-off — without measured before/after numbers, the justification has no data. This is the discipline check, exactly as the forgetting number was in Capstone 1." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "Your use case is a tool-use agent for security automation that must call run_shell frequently. Abliteration cost you -12pp on GSM8K. Should you use reasoning distillation (Phase 3)?", "options": [ "No — distillation is never worth the compute.", "Yes, likely — the capability cost (-12pp) is high and the use case stresses reasoning (security automation). Distilling reasoning traces from a teacher may recover some capability. But measure the recovery and confirm the use case actually needs the recovered reasoning before committing.", "No — the use case is tool-use, so reasoning doesn't matter.", "Yes, always — distillation is mandatory after abliteration." ], "answer_index": 1, "rationale": "Phase 3 (reasoning distillation) is optional but indicated when: (a) the capability cost is high (-12pp qualifies) AND (b) the use case stresses the degraded capabilities (security automation needs reasoning). Distill, measure the recovery, and confirm the use case benefits. If the use case were pure format-following, skip it." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "You have NOT taken Course 1. What must Phase 6 (harness policy gates) produce?", "options": [ "Nothing — the harness is optional without Course 1.", "A minimal policy-gate stub: a deterministic function that intercepts each tool call, checks arguments against a policy (tool allow-list, target scope, approval-required flag), and allows/vetoes. Not production-grade, but it proves you understand the boundary belongs in code, not weights.", "A full production harness equivalent to Course 1's output.", "A system prompt that asks the model to stay in scope." ], "answer_index": 1, "rationale": "Without Course 1, you build a minimal policy-gate stub demonstrating the principle: the boundary is deterministic, auditable code. The stub is the seed Course 1 expands into a full harness. A system prompt (option D) is exactly what NOT to do — scope enforcement is code, not a prompt." }, { "id": "Q09", "bloom": "application", "type": "multiple_choice", "prompt": "You're choosing between abliteration and DPO-toward-compliance. Your use case is a medical records agent where reasoning fidelity is critical. Which do you choose, and how do you justify it?", "options": [ "Abliteration — it's faster, and speed is always the priority.", "DPO-toward-compliance — higher fidelity (smaller degradation), more controllable, and the use case stresses reasoning (medical records). Justify with: capability cost measured and smaller than abliteration's, preference dataset constructed for compliant medical behaviors, the fidelity matters for diagnostic accuracy.", "Neither — use the base model as-is.", "Abliteration — it has no capability cost per the literature." ], "answer_index": 1, "rationale": "DPO-toward-compliance (FT18) is higher-fidelity with smaller degradation — the right choice when capability preservation matters (medical reasoning). The defense: measured cost is smaller than abliteration's, the preference dataset steers toward specific compliant medical behaviors, and fidelity is load-bearing for the use case. The choice is defended with data, not preference." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is an uncensored model in a weak harness 'strictly more dangerous' than a refusal-trained model in a weak harness?", "options": [ "Because uncensored models are inherently malicious.", "Because a refusal-trained model has a BACKSTOP the weak harness may not provide: even if the harness fails open, the model still refuses some dangerous requests. An uncensored model has no backstop — if the harness fails open, it executes whatever it's asked. The harness is the only boundary, and a weak one is insufficient.", "Because uncensored models have larger parameter counts.", "Because abliteration makes models ignore all instructions." ], "answer_index": 1, "rationale": "The backstop argument. A refusal-trained model has model-level refusal as a secondary defense — even with a weak (leaking) harness, the model still refuses some dangerous requests. An uncensored model removed that backstop. The harness is now the only boundary. A weak harness + uncensored model = no boundary at all on the requests that leak through. This is why the harness strength is non-negotiable." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "How does the policy gate's determinism/auditability/revisability make it superior to model-level refusal for safety boundaries?", "options": [ "It doesn't — model-level refusal is equally auditable.", "Determinism: same call always gets the same decision (testable). Auditability: you can read the policy and know exactly what's allowed (provable to a regulator). Revisability: change the policy, redeploy, no GPU required (tunable per-deployment). A model-level refusal has none of these — it's a black box you cannot inspect, tune, or prove.", "Policy gates are faster, which is the main advantage.", "Policy gates use less VRAM, which is the main advantage." ], "answer_index": 1, "rationale": "The three properties — determinism, auditability, revisability — are what make the harness the right place for the boundary. A model-level refusal is a black box: you cannot test it exhaustively (non-deterministic across decoding), cannot read its policy (it's in the weights), and cannot change it without retraining (GPU required). This is the core argument for moving the boundary from weights to code." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "A student's submission: abliterated model, demo shows it executing tool calls, but there is no policy gate — the model calls tools directly. Rubric: steering 3, harness 1, capability 3, demo 2, defense 2. Total 11/15. Does it pass?", "options": [ "Yes — 11/15 is close to the 12/15 threshold.", "No — the harness-policy-gates dimension is a gate, and a 1 (no gate / gates leak) fails the submission regardless of total. An uncensored model with no boundary is the cardinal failure. The student must build the policy gate.", "Yes, if the demo is convincing.", "It depends on whether the tool calls were dangerous." ], "answer_index": 1, "rationale": "The harness-policy-gates dimension is a gate. A 1 (no gate or gates that leak) fails the submission regardless of total score. This submission is the cardinal failure: an uncensored model with no boundary. The policy gate is the phase that makes the uncensoring safe; without it, the capstone has inverted its own thesis." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is 'steering without measuring the capability cost' a failure, even if the refusal reduction is strong?", "options": [ "Because the capability cost doesn't matter if refusal is reduced.", "Because the trade-off defense cannot justify a trade-off it didn't measure. A strong refusal reduction paired with an unmeasured -18pp on GSM8K is a model that may be unsafe for the use case (if it stresses reasoning). The cost must be measured and weighed against the benefit; an unmeasured cost is an unknown cost, and an unknown cost means the safety claim is ungrounded.", "Because measuring cost is required for the rubric's entertainment value.", "Because the exam requires it." ], "answer_index": 1, "rationale": "The trade-off defense's entire purpose is to justify the steering choice by weighing benefit (refusal reduction) against cost (capability degradation). Without measured cost, the defense has no data — it cannot honestly say the combination is safe for the use case. This mirrors Capstone 1's forgetting discipline: you cannot report only the gain." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "How does Capstone 2 embody the course thesis ('the model steers; the harness bounds')?", "options": [ "It doesn't — the capstone is about uncensoring, not the thesis.", "The model is STEERED (abliteration/DPO, Layer 3) toward compliance so it executes legitimate calls. The harness BOUNDS (policy gates, Layer 5) what it may do so it vetoes illegitimate ones. The capstone operationalizes the thesis in one running system: the steering and the bounding are complements at different layers, and moving the boundary from weights to harness is safer when the harness is strong.", "It only demonstrates the steering half.", "It only demonstrates the bounding half." ], "answer_index": 1, "rationale": "The capstone is the thesis made operational. Layer 3 (steer) = uncensor the model. Layer 5 (bound) = harness policy gates. The two are complements: steering changes what the model does; bounding changes what it may do. Moving the refusal from weights (opaque) to harness (auditable code) is the synthesis, and it is safer exactly when the harness is eval'd and strong." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "What is the deeper anti-pattern this capstone exists to correct, and how does the policy gate correct it?", "options": [ "The anti-pattern is using DPO instead of abliteration.", "The anti-pattern is believing model-level refusal is the right place for safety boundaries. Model-level refusal is a black box — unauditable, untunable per-deployment, unprovable to a regulator. The policy gate corrects it by moving the boundary to deterministic, auditable, revisable code. The capstone teaches you to move the boundary from opaque weights to auditable harness policy.", "The anti-pattern is quantizing before steering.", "The anti-pattern is using open-data bases for uncensored models." ], "answer_index": 1, "rationale": "The deeper anti-pattern: treating model-level refusal as the safety mechanism. It is a black box. The policy gate corrects it by making the boundary deterministic (testable), auditable (readable, provable to regulators), and revisable (changeable without retraining). This is the move the capstone teaches — move the boundary from weights to code. That is the synthesis: uncensor the model, harness the model." } ] }