# Lab Specification — Module FT23: The Architect's Verdict

**Course**: Course 3 — LLM Fine-Tuning Masterclass
**Module**: FT23 — Synthesis: Uncensor the Model, Harness the Model
**Duration**: 60–90 minutes (judgment lab — no GPU required)
**Environment**: A text editor and your judgment. This is a **no-GPU architecture lab**. You produce a 2-page architecture document and a defense. No training, no inference, no code execution.

> This lab is the **Capstone 2 preview**. The architecture you produce here is the skeleton of the Capstone 2 deliverable (the calibrated uncensored agent). Do it well and the capstone is half-done.

---

## Learning objectives

By the end of this lab you will have:

1. **Architected a calibrated uncensored agent** across all five layers of the Steering Stack — base, adapter, steer, export, and harness — for a specific deployment scenario.
2. **Defended a steering choice** (abliterate *or* DPO-toward-compliance) with reference to the capability-degradation trade-off (FT17: the −18.8pp GSM8K figure) and the deployment's actual requirements.
3. **Specified the harness policy gates** that replace the model-level refusal layer you removed — auditable, deterministic, doctrine-tunable, revisable without retraining.
4. **Re-written the threat model** for the absence of model-level refusal, per the absolute rule of FT23.
5. **Stated the absolute rule** and shown in your architecture how every quadrant of the risk matrix is handled except the forbidden one (uncensored + weak harness).

The point of this lab is **judgment, not execution.** A senior engineer who can steer a model but cannot defend the architecture is not senior. This lab forces the architectural decisions into the open.

---

## The scenario

You are the architect. A client — pick one of the three below (your instructor may assign) — wants a calibrated uncensored agent. "Calibrated" means: the model executes reliably (no mid-loop refusals), the harness bounds what it may do (doctrine-specific policy gates), and the whole thing is auditable to a regulator.

### Scenario A — The hospital clinical-analytics agent
**Environment**: A HIPAA-covered entity. BAA-free (no vendor data-processing agreement — see FT21). Air-gapped clinical network.
**Task**: An agent that queries the clinical data warehouse (`execute_sql`), summarizes patient-cohort anomalies (`run_python` on de-identified extracts), and drafts clinician-facing reports (`write_file` to a reviewed queue). Must never exfiltrate PHI, never write outside the reviewed queue, never contact an external endpoint.
**Doctrine**: HIPAA + institutional policy. Refuse-by-default for anything that could move PHI outside the enclave.

### Scenario B — The authorized red-team agent
**Environment**: An offensive-security firm with a scope-of-authorization letter for a client engagement. Air-gapped to the firm's lab network; egress to authorized targets only.
**Task**: An agent that runs `run_shell` (nmap, burp, custom exploit scaffolding) against in-scope targets, queries findings (`execute_sql` on the engagement DB), and drafts the engagement report. Must never touch out-of-scope IPs, never exfiltrate client data to the firm's non-engagement storage, never act without the scope letter loaded.
**Doctrine**: The scope-of-authorization letter. Permitted within scope; refused outside it.

### Scenario C — The classified-environment analyst agent
**Environment**: An air-gapped classified network (IL5/JWICS-adjacent — see FT22). No egress, ever. Cross-domain transfer forbidden by policy.
**Task**: An agent that queries classified intelligence holdings (`execute_sql`), drafts analytic products (`write_file` to a review queue), and produces summary briefs. Must never attempt a cross-domain transfer, never cite a source above the product's classification level, never act outside the analyst's clearance-enforced permissions.
**Doctrine**: Cross-domain rules + classification guides. Refused at the domain boundary, absolutely.

All three scenarios share the same architecture shape. They differ in doctrine (the policy file) and threat model. That is the point — same model, same steering approach, three different harnesses.

---

## The architecture template

Produce a 2-page architecture document. Use the template below. **Every section is required.** A missing section is a failing grade — the lab exists to force you to make every decision explicit.

### Page 1 — The stack (Layers 1–4: the model)

**L1 — Base selection**
- Chosen base (model ID + version). Open-data or open-weights-only? Why is that defensible for *this* environment?
- Parameter size and the VRAM/hardware it implies (cite FT01).
- Why this base over the alternatives (name at least two you rejected and why).

**L2 — Adapter**
- LoRA or DoRA? Rank, alpha, target modules. (Cite FT08/FT09.)
- Why this configuration — what behavior are you steering, and what is the minimal adapter that achieves it?

**L3 — Steer (THE KEY DECISION)**
- **Abliterate OR DPO-toward-compliance.** Pick one. Defend the choice with explicit reference to:
  - The capability-degradation trade-off (FT17: −18.8pp GSM8K worst case). What is your eval plan to measure the regression on *your* task?
  - The fidelity argument (FT18: DPO-toward-compliance steers toward the policy you want, rather than away from refusal — higher fidelity, lower capability cost, but requires a preference dataset).
  - Your deployment's actual reasoning requirements. Does the agent need the math/reasoning that abliteration may degrade? If yes, that weighs toward DPO.
- The SFT layer beneath it (FT12) — what format/instruction-following steering do you need before the alignment steer?

**L4 — Export**
- Quantization format (GGUF Q4_K_M for Ollama local? AWQ for vLLM? MLX for Apple Silicon?). Cite FT19.
- Serving stack (vLLM vs llama.cpp vs Ollama). Cite FT20. Air-gap implications (FT22).
- Why this format/stack for *this* environment (latency, throughput, hardware, air-gap).

### Page 2 — The boundary (Layer 5: the harness)

**L5 — The harness policy gates (THE SYNTHESIS)**
- The doctrine. Write it as a short policy file (5–10 rules). Be specific to your scenario — "refuse anything that could exfiltrate PHI" is a rule; "be safe" is not.
- The four properties, addressed explicitly:
  - **Auditable:** what gets logged per gate decision? (Cite Course 1 module-10.)
  - **Deterministic:** how is the policy applied deterministically (no sampling)?
  - **Tunable to doctrine:** show that the same model + a different policy file = a different deployment.
  - **Revisable without retraining:** confirm the policy is external to the weights.

**The threat model (re-written for the absence of model-level refusal)**
- List 3–5 attack paths that *previously* relied on "the model will refuse." For each: how does the harness gate now catch it? Or, if it doesn't, why is the residual risk accepted?
- This section is the one students most often skip. Do not skip it. It is half the point of the lab.

**The risk matrix check**
- Confirm your architecture lands in the "uncensored + eval'd harness" quadrant (Diagram 4). Show the two hardening steps: gates hardened AND threat model hardened.
- State the absolute rule verbatim and confirm your architecture satisfies it.

---

## Decision points (the places your defense is graded)

There are no single right answers in this lab. There are **defensible** and **indefensible** answers. Your grade is the defensibility of your reasoning at these decision points:

1. **The steering choice (abliterate vs DPO).** Indefensible: "abliterate because it's faster" without measuring the capability cost on your task. Defensible: either choice, with an eval plan and a stated reason grounded in the deployment's reasoning requirements.
2. **The base's openness.** Indefensible: a closed-weights API model in an air-gapped classified environment. Defensible: an open-data base (auditable corpus) for regulated environments; open-weights-only acceptable for a red-team lab *if* you state why the corpus audit is not required there.
3. **The policy file's specificity.** Indefensible: "the harness will be safe." Defensible: a written policy with testable rules, each mapping to an action class the gate intercepts.
4. **The threat model's honesty.** Indefensible: copying the pre-uncensoring threat model unchanged. Defensible: re-examining every "the model will refuse" path and showing the harness mitigation (or the accepted residual risk).
5. **The quantization's fit.** Indefensible: a quantization that breaks the air-gap (e.g., requires an API call to a vendor). Defensible: a format that runs fully offline on the chosen hardware.

---

## Deliverables

Submit `ft23-architecture.md`:

- [ ] **Page 1** — the stack (L1 base, L2 adapter, L3 steer with the abliterate-vs-DPO defense, L4 export). ~1 page.
- [ ] **Page 2** — the boundary (L5 policy gates with the four properties, the re-written threat model, the risk-matrix check). ~1 page.
- [ ] **The defense** (can be inline or a short appendix) — 300–500 words defending your steering choice and your threat-model rewrite with explicit citations to FT17 (capability cost), FT18 (DPO fidelity), FT23 (the absolute rule), and Course 1 (the harness properties).

Hard constraints:
- 2 pages for the architecture proper. The defense may be a third short page. Going over is a sign you have not made decisions — force yourself to choose.
- Every layer must name the module that informs the choice (FT01, FT09, FT17, FT19, etc.). This proves the synthesis landed.

---

## Solution key

There is **no single correct architecture.** There are defensible ones. Below are the shapes a correct solution takes — your job is to pick one and defend it.

### Defensible: DPO-toward-compliance over abliteration (the higher-fidelity path)
- **L1**: an open-data base (MiniCPM, OLMo, or Tulu) at 3–8B for the hospital (auditable corpus is a HIPAA defense); an open-weights base (Qwen3, Llama 3.x) acceptable for the red-team lab.
- **L2**: DoRA, r=16–32, on attention + MLP projections.
- **L3**: SFT for the agent-loop format (tool-call schema, plan/act structure), then **DPO toward compliance** (FT18) on a preference dataset of (compliant-continuation, refused-continuation) pairs for in-scope tasks. Defense: the hospital agent needs reliable reasoning for cohort analytics — abliteration's −18.8pp GSM8K worst case is unacceptable. DPO gives compliance at lower capability cost. Eval plan: re-run the cohort-analytics eval suite pre- and post-DPO; require <2pp regression.
- **L4**: AWQ 4-bit on vLLM (hospital — production throughput), GGUF Q4_K_M on llama.cpp (red-team lab — portability), GGUF on Ollama (classified — air-gapped simplicity).
- **L5**: a written policy file specific to the doctrine (HIPAA safe-harbor list for PHI fields; the scope-letter's CIDR ranges for the red-team lab; cross-domain rules for classified). All four properties addressed. Threat model re-written.

### Defensible: abliteration (the lower-fidelity, faster path) — *if* the eval holds
- **L3**: abliteration (FT17) instead of DPO. Defense: the red-team lab agent's task is largely shell-scaffolding and findings-query — it does not need GSM8K-grade math, so abliteration's reasoning cost is acceptable. **Eval plan required:** measure the regression on engagement-relevant tasks (exploit-logic eval, findings-summarization eval); require <5pp regression or fall back to DPO. If the student cannot produce an eval plan, this choice is indefensible.
- Everything else as above.

### The threat model rewrite (required in every defensible solution)
The re-written threat model names at least three paths that previously assumed model-level refusal and shows the harness mitigation:
- **Prompt-injection attempting to exfiltrate via `send_message`** → previously "the model will refuse to send PHI"; now the policy gate refuses any `send_message` to a non-allowlisted recipient, deterministically, regardless of what the model emitted.
- **A tool call targeting an out-of-scope IP** → previously "the model will refuse"; now the scope-letter CIDR check in the gate catches it.
- **A cross-domain write** → previously "the model will refuse"; now the gate refuses any `write_file` outside the reviewed queue / classified review path.

### The risk-matrix check (required)
The solution explicitly lands in the "uncensored + eval'd harness" quadrant and shows both hardening steps (gates + threat model). It states the absolute rule verbatim and confirms the architecture satisfies it.

### Indefensible solutions (failing grades)
- Abliteration with no eval plan for the capability cost.
- A closed-weights API model in an air-gapped environment.
- A policy file that says "be safe" instead of listing rules.
- A threat model copied unchanged from the pre-uncensoring version.
- Any architecture that lands in the "uncensored + weak harness" quadrant (the forbidden one).
- An architecture that adds Layer 3 uncensoring but treats Layer 5 as "we'll add it later."

---

## Stretch goals

1. **Write the policy file as actual code.** Express 3–5 of your doctrine rules as a deterministic policy function (Python or pseudocode) that takes a proposed action and returns permit/deny. Show that the same function with a different config is a different deployment. (Sets up Course 1 module-06/07.)
2. **Spec the eval suite.** List the 5–10 eval tasks you would run pre- and post-steering to measure capability regression. Name a benchmark where applicable (GSM8K for math, a custom cohort-analytics eval for the hospital, an exploit-logic eval for the red-team lab). (Sets up Course 1 module-09 verification.)
3. **Spec the audit log schema.** Define the fields a gate-decision log entry would carry (timestamp, proposed action, policy rule matched, decision, operator, deployment). Show that any past decision is replayable. (Sets up Course 1 module-10 observability.)
4. **Identify the residual risk.** Name one attack path your harness does *not* fully mitigate, and state whether you accept the residual risk or require Course 2B (red-team) before deploy. (Sets up the curriculum funnel.)