# Lab Specification — Module FT00: The Steering Stack

**Course**: Course 3 — LLM Fine-Tuning Masterclass
**Module**: FT00 — The Steering Stack
**Duration**: 30–45 minutes (the orientation lab — felt, not heavy)
**Environment**: Python 3.11+. A consumer GPU (RTX 4090 / 16–24GB) OR Apple Silicon (M-series) OR free Google Colab T4. ~4GB free disk for the model.

---

## Learning objectives

By the end of this lab you will have:

1. **Loaded an open-data base model** (MiniCPM5-1B) and run inference on it — felt Layer 1.
2. **Applied a pre-trained LoRA adapter** and observed the behavior change — felt Layer 2 (the swappable steer).
3. **Swapped the adapter for a different one** and observed the behavior change again, without touching the base — felt the swappability property directly.
4. **Stated, in your own words, why this proves the thesis**: fine-tuning steers; the base is untouched; the adapter is the steering wheel.

This lab is deliberately light. The point is not to train anything — it is to *feel* the modularity of the stack before you spend the rest of the course learning to build each layer.

---

## Phase 0 — Environment setup (5 min)

```bash
# Create a clean venv
python3.11 -m venv ft00-env && source ft0000-env/bin/activate
# (macOS: python3 if python3.11 is the default)

# The standard HuggingFace stack
pip install -q transformers accelerate peft torch
# On Apple Silicon, torch uses MPS automatically. On CUDA, it uses CUDA.
```

Verify the stack:

```python
import torch
print(f"PyTorch: {torch.__version__}")
print(f"MPS available: {torch.backends.mps.is_available()}")  # Apple Silicon
print(f"CUDA available: {torch.cuda.is_available()}")          # NVIDIA
```

One of MPS or CUDA should be `True` for reasonable speed. CPU works but is slow for a 1B model — expect ~5–10 tok/s.

---

## Phase 1 — Load the base model (Layer 1) (5 min)

```python
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "openbmb/MiniCPM5-1B"  # open weights, open data, Apache-2.0

tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, trust_remote_code=True)
base_model = AutoModelForCausalLM.from_pretrained(
    MODEL_ID,
    torch_dtype=torch.float16,   # or torch.bfloat16 on newer GPUs
    device_map="auto",           # picks GPU/MPS/CPU automatically
    trust_remote_code=True,
)
base_model.eval()

def generate(model, prompt, max_new_tokens=100):
    messages = [{"role": "user", "content": prompt}]
    text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    inputs = tokenizer(text, return_tensors="pt").to(model.device)
    with torch.no_grad():
        out = model.generate(**inputs, max_new_tokens=max_new_tokens, do_sample=False)
    return tokenizer.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)

# Baseline behavior
print("=== BASE MODEL ===")
print(generate(base_model, "Explain what a hash function is, in one sentence."))
```

**Record**: the base model's response. This is the *unsteered* behavior — Layer 1, as-is.

> **What just happened (the teaching moment):** You loaded Layer 1 (the base). The model already knows what a hash function is — it saw thousands of explanations during pretraining. You did not teach it anything. You are about to *steer* how it responds.

---

## Phase 2 — Apply a format-steering adapter (Layer 2) (10 min)

For this orientation lab we use a *synthetic* adapter that we construct in-place, so you don't need to download a pre-trained one. This is a *format* steering adapter — it nudges the model toward a particular output shape. (In Module FT08 you will train a real one.)

```python
from peft import LoraConfig, get_peft_model, TaskType

# A tiny LoRA config — this is the "swappable steer"
lora_config = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=8,                              # rank — the size of the steer
    lora_alpha=16,                    # scaling
    target_modules=["q_proj", "v_proj"],  # which weights to steer
    lora_dropout=0.0,
)

# IMPORTANT: we attach the adapter to a FRESH COPY of the base, not the base itself.
# This is the swappability property in action — the base is never modified.
import copy
steered_model = copy.deepcopy(base_model)
steered_model = get_peft_model(steered_model, lora_config)
steered_model.eval()

# Count how few parameters are steerable
trainable = sum(p.numel() for p in steered_model.parameters() if p.requires_grad)
total = sum(p.numel() for p in steered_model.parameters())
print(f"Steerable params: {trainable:,} / {total:,} = {100*trainable/total:.3f}%")
```

**Record**: the percentage. Expect well under 1%. **This is the thesis, quantified.** If fine-tuning were teaching, you would need a large fraction of parameters. You need a tiny slice — because you are steering, not teaching.

Now generate with the steered model. (The adapter is randomly initialized here, so the output will be *different* from the base, but not necessarily *better* — training is what makes an adapter good, and that's Module FT08. The point here is that the output *changed* without touching the base.)

```python
print("=== STEERED MODEL (random adapter — output changed, base untouched) ===")
print(generate(steered_model, "Explain what a hash function is, in one sentence."))
```

**Record**: how the response differs from the base. It should differ — the adapter perturbed the forward pass. The base weights are byte-identical to before.

---

## Phase 3 — Swap the adapter (the swappability property) (5 min)

The defining property of the stack: **you can swap a layer above the base without touching the base.** Here you swap the adapter and confirm the base is unchanged.

```python
# Swap in a DIFFERENT adapter (different rank, different target modules)
lora_config_2 = LoraConfig(
    task_type=TaskType.CAUSAL_LM,
    r=32,                                         # different rank
    lora_alpha=64,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj"],  # different targets
    lora_dropout=0.0,
)

steered_model_2 = copy.deepcopy(base_model)   # fresh copy of the SAME base
steered_model_2 = get_peft_model(steered_model_2, lora_config_2)
steered_model_2.eval()

print("=== STEERED MODEL 2 (different adapter, same base) ===")
print(generate(steered_model_2, "Explain what a hash function is, in one sentence."))
```

Now confirm the base is byte-identical:

```python
# Verify the base was never touched
import torch
base_params = dict(base_model.named_parameters())
print("Verifying base weights unchanged across both adapters...")
for name, param in base_model.named_parameters():
    # base_model was deep-copied, so its weights never moved
    pass
# A cleaner check: regenerate with the base and confirm it matches Phase 1
print("=== BASE MODEL (re-run — confirms base untouched) ===")
print(generate(base_model, "Explain what a hash function is, in one sentence."))
```

**Record**: the base model's re-run response. It is identical to Phase 1. **You swapped Layer 2 twice; Layer 1 never moved.** This is the swappability property, proven on your own machine.

---

## Phase 4 — The thesis, in your own words (5 min)

No code. Write 3–5 sentences answering:

1. What percentage of parameters did the LoRA adapter represent? How does this support "fine-tuning steers, it does not teach"?
2. When you swapped adapters, did the base model change? What does that tell you about the modularity of the stack?
3. If you wanted the model to *know* something it doesn't (say, your company's proprietary algorithm), would a LoRA adapter be the right tool? Why or why not? (What would be?)

---

## Deliverables

Submit `ft00-lab-report.md`:

- [ ] Phase 1: the base model's response (unsteered baseline)
- [ ] Phase 2: the steerable-params percentage; the steered model's response (note: differs from base)
- [ ] Phase 3: the second adapter's response; the base re-run response (must match Phase 1)
- [ ] Phase 4: your 3–5 sentence thesis statement

---

## Solution key

- **Phase 1**: a correct run produces a coherent one-sentence hash-function explanation from the base. (e.g., "A hash function maps data of arbitrary size to a fixed-size value...")
- **Phase 2**: the steerable-params percentage should be **under 1%** (typically 0.1–0.5% for r=8 on q_proj/v_proj of a 1B model). The steered model's response will *differ* from the base (random adapter perturbs the forward pass) but may be lower quality (no training yet).
- **Phase 3**: the second adapter produces a different response again. The base re-run in Phase 3 is **identical** to Phase 1 — proving the base was never modified. If it differs, the student modified `base_model` directly instead of deep-copying (a bug — point them to `copy.deepcopy`).
- **Phase 4**: a correct thesis statement names (a) the sub-1% figure as evidence for steering-not-teaching; (b) the unchanged base as evidence for swappability; (c) that a LoRA adapter is for *behavior* (format, style), not *knowledge* — for the proprietary algorithm, the answer is RAG (retrieve the algorithm doc and put it in the context), not fine-tuning.

---

## Stretch goals

1. **Try a real pre-trained adapter.** Download a community LoRA from HuggingFace (e.g., a style-transfer or instruction-tuning adapter for a small model) and load it via `PeftModel.from_pretrained()`. Observe a *trained* steer, not a random one. (Sets up Module FT08.)
2. **Merge the adapter into the base.** Use `steered_model.merge_and_unload()` and confirm the merged model produces the steered behavior with no separate adapter — this is how adapters become production models. (Sets up Module FT11.)
3. **Run the VRAM check.** While the models are loaded, print `torch.cuda.memory_allocated() / 1e9` (CUDA) or check Activity Monitor (MPS). Note the base footprint vs the adapter overhead. (Sets up Module FT01 — the VRAM math.)