The Steering Stack

Module FT00 · Course 3 — LLM Fine-Tuning Masterclass

60 minutes · 4 sub-sections: The Thesis · The Stack · Steering vs Knowledge · Course Map

The entry point. The one idea the entire course is built on.

Pillar 0 — Foundations

The thesis — one sentence

Fine-tuning steers behavior; it does not teach knowledge.
The model steers — the harness bounds.

Every technique in this course — SFT, DPO, GRPO, abliteration, quantization — is a steering technique.

It redirects an already-capable base model. It does not pour new knowledge into the weights.

The single exception is continued pretraining (de-emphasized — Module FT12).

Why this matters immediately

Once the thesis lands, four mysteries become obvious:

Mystery	Why the thesis explains it
Why QLoRA works at 1.5% of params	Steering is low-rank. A tiny adapter suffices.
Why uncensoring degrades math	Steering away from refusal nudges entangled capabilities.
Why uncensored is only safe in a harness	Steering changes what it DOES, not what it MAY do.
Why data > algorithm	The steering wheel is your dataset. Bad data = bad direction.

The complement to Course 1

Course 1

The model is ~1.6% of an agent.

The harness is the other 98.4%.

Course 3 (this course)

Zooms into the 1.6%.

What can you change about the model — and what should you leave to the harness?

Complement, not contradiction. The model steers; the harness bounds. Same system, opposite ends.

The Steering Stack

Five layers. Every module maps to one.

5. THE BOUNDARY · the harness · Courses 1 & 2A

↑

4. THE EXPORT · quant + serve · GGUF · vLLM · Ollama · MLX

↑

3. THE STEER · fine-tuning · SFT · DPO · GRPO · abliteration

↑

2. THE ADAPTER · LoRA / DoRA · <1% params, swappable

↑

1. THE BASE · pretrained weights · the world model + openness

The swappability property

You can swap any layer above the base without touching the one below.

This single property makes the whole field tractable:

Adapters are swappable (Layer 2 detaches from Layer 1)
Abliteration works without retraining (Layer 3 edits weights Layer 2 would train)
You quantize after training (Layer 4 compresses Layer 3's output)
The harness is model-agnostic (Layer 5 wraps Layer 4's serve)

The stack is modular by design.

Layer 1 — The Base

The pretrained weights. The "world model." You did not make it. You choose it (FT03).

Defining property for this course: openness.

Type	You get	Examples
Open-data	Weights + training corpus	MiniCPM · OLMo · Tülu · SmolLM3
Open-weights-only	Weights, no data	Llama 3.x

Open-data = auditable. Open-weights-only = trust the publisher's word. Load-bearing for HIPAA/government (FT21, FT22).

Layers 2 & 3 — Adapter and Steer

Layer 2 — The Adapter

LoRA / DoRA. Tiny trainable matrices — often <1% of params.

Base is frozen; only the adapter trains. Merges back in, or stays separate for hot-swapping.

This is what democratized fine-tuning. 7B on a $1,500 RTX 4090.

Layer 3 — The Steer

The full spectrum of behavior-shaping:

SFT — format, instruction-following
DPO family — preference (better/worse)
GRPO — reasoning (verifiable rewards)
Abliteration — steer away from refusal

Layers 4 & 5 — Export and Boundary

Layer 4 — The Export

Quantize then serve. Downstream of training.

GGUF — local/Ollama
AWQ — vLLM production
MLX — Apple Silicon

Mistakes cost quality or speed, not learned behavior.

Layer 5 — The Boundary (harness)

Not this course — Courses 1 & 2A.

What makes Layer 3's uncensoring safe. Policy gates, audit logs, threat model.

Independent of the model. Revisable without retraining. Safety lives here, not in the weights.

Steering vs Knowledge

The most important judgment in fine-tuning.

If the base model, with a perfect prompt, could already produce the target behavior — that's steering.
If it could not, no amount of fine-tuning will reliably get you there.

Steering (fine-tuning does this)	Knowledge (it does NOT)
Format & style	New facts the base never saw
Instruction following	A new language
Preference alignment	Deep domain vocabulary
Reasoning activation	Genuine new reasoning
Refusal removal	Adding safety the base lacks

The three-outcome test

Before you fine-tune, prompt the base with ideal input + a great system prompt.

Outcome	Diagnosis	Intervention
Produces it unreliably / wrong format	Steering	SFT (FT12)
Refuses or wrong kind, but capability there	Preference	DPO / abliteration (FT13, FT17)
Genuinely foreign — no idea	Knowledge gap	Different base, CPT, or RAG

The cardinal error: treating fine-tuning as knowledge injection. "Fine-tune on our medical records and it learns medicine." It will not. It memorizes surface patterns and fails out-of-distribution. For knowledge, retrieve.

The evidence (not just a slogan)

Intrinsic dimension (Aghajanyan 2020)

Useful fine-tuning changes live in a low-rank subspace.

If it were moving knowledge, you'd need large param updates. You need a tiny slice. That slice is the steering vector.

An Illusion of Equivalence (2024)

LoRA and full FT produce structurally different weight matrices.

Not approximations. Low-rank path vs higher-rank path, reaching similar behavior via different geometry.

Both findings are what you'd expect if fine-tuning is steering (multiple solutions) rather than knowledge injection (one correct answer).

Course map — pillars to layers

Pillar	Layer	Modules
P00 Foundations	Base + mental model	FT00–FT03
P01 Data	Substrate	FT04–FT07
P02 PEFT	Adapter + Steer	FT08–FT11
P03 Alignment	Steer	FT12–FT13
P04 Reasoning	Steer (frontier)	FT14–FT15
P05 Alignment Control	Steer	FT16–FT18
P06 Deploy	Export	FT19–FT20
P07 Sensitive Domains	Boundary	FT21–FT23

Read top to bottom: you are climbing the stack.

The synthesis (Module FT23)

Uncensor the model so it executes;
harness the model so it executes only what it should.

The destination of this course:

A model steered toward your intent (Layers 1–4)
Deployed inside a harness that bounds what it may do (Layer 5)

The honest caveat: abliteration degrades capability. An uncensored model in a weak harness is strictly more dangerous than a refusal-trained model in a weak harness. Never deploy uncensored without an eval'd harness.

Anti-patterns

"Teaching" via fine-tuning. Expecting the model to learn content from documents. It memorizes surface patterns, fails OOD. Use RAG for knowledge; fine-tune for behavior.

Intervening at the wrong layer. Format problem via CPT (overkill), knowledge problem via SFT (underkill), safety problem in weights (Layer 3 when you need Layer 5).

Steering without a steering wheel. Sophisticated algorithm (GRPO, DPO) on bad data. Bad data = bad direction, no matter the optimizer. This is why Pillar 1 (Data) comes first.

What you can now do

State the thesis and defend it with the intrinsic-dimension evidence.
Draw the five-layer stack and explain swappability.
Distinguish steering from knowledge injection.
Predict, for any goal, which layer the intervention belongs at.

Next: FT01 — VRAM Math · Can I Actually Run This?