# Teaching Script — Module FT01: VRAM Math

**Course**: Course 3 — LLM Fine-Tuning Masterclass
**Module**: FT01 — VRAM Math: Can I Actually Run This?
**Duration**: ~35 minutes (spoken at ~140 wpm)
**Format**: Verbatim transcript with `[SLIDE N]` cues. Read aloud or use as speaker notes.

---

[SLIDE 1 — Title]

Welcome back. This is module FT zero one — VRAM Math: Can I Actually Run This? FT zero zero gave you the steering thesis and the five-layer stack. This module is the gatekeeper question that comes before any of that matters. Because the best steering technique in the world is useless if you cannot afford to load the model.

Here is the single most important fact about fine-tuning large language models in twenty-twenty-six. You will run out of VRAM long before you run out of FLOPs. A modern GPU can do the math for a seven-billion-parameter fine-tune in reasonable time. Whether you can hold the working set in memory — that is what decides whether the job runs at all. Compute is a budget. Memory is a cliff. Step over the cliff and the job doesn't go slower. It OOMs and dies.

[SLIDE 2 — The cost framing]

Let me make the stakes concrete with money. Full fine-tuning a seven-billion-parameter model needs on the order of fifty thousand dollars of H-one-hundreds. Multiple eighty-gigabyte cards, often a whole node. QLoRA fine-tuning the same seven-billion-parameter model runs on a fifteen-hundred-dollar RTX forty-ninety — or free on a Colab T-four for smaller bases.

That is not a twenty-percent difference. That is "I need procurement and a budget cycle" versus "I start tonight." Most steering problems — format, preference, reasoning activation — do not need full fine-tuning. Most engineers who reach for full FT are paying thirty times for capability they will never observe in the output. The VRAM math is what tells you which camp you are in. So let's learn the math.

[SLIDE 3 — The three consumers]

When you train, VRAM is eaten by three things. Memorize them. Every OOM is one of these three.

Consumer one: model weights. The parameters themselves at whatever precision the base is held. FP16 is two bytes per param — a seven-billion-parameter model is about fourteen gigabytes. Four-bit is half a byte per param — the same seven-billion model is about three-point-five gigabytes. The precision choice for the base is the single biggest VRAM lever, which is exactly why QLoRA exists.

Consumer two: optimizer states and gradients. This is the one people forget, and it is where full fine-tuning explodes. AdamW — the default optimizer — keeps, for every trainable parameter, an FP sixteen weight, an FP sixteen gradient, an FP thirty-two master copy of the weight, the first moment, and the second moment. That is sixteen bytes per trainable parameter, before a single activation is stored. For a fully-trainable seven-billion model: sixteen times seven billion is about one hundred twelve gigabytes — just for weights and gradients and optimizer states. Add activations and you land in the one hundred to one hundred sixty gigabyte range. Multi-A-one-hundred territory.

Consumer three: activations. The intermediate values saved during the forward pass so the backward pass can compute gradients. These scale with sequence length, batch size, and number of layers — not just parameter count. This is the consumer you control with knobs, and we'll get to those.

[SLIDE 4 — The rules of thumb]

Here are the field rules of thumb. Anchor on the concrete gigabyte numbers — they are what you plan against.

QLoRA — four-bit frozen base — runs about one-point-five to two times the four-bit model size. For a seven-billion model, that's about ten to sixteen gigabytes. Fits an RTX forty-ninety twenty-four gigabytes. For a seventy-billion model, about forty-eight to sixty gigabytes. Fits a single A-one-hundred eighty gigabytes.

LoRA with a sixteen-bit base runs about two to three times the FP sixteen model size. Seven-billion LoRA is about eighteen to thirty gigabytes — an A-one-hundred, or a forty-ninety with aggressive checkpointing.

Full fine-tuning is about sixteen bytes per param plus activations. Seven-billion full FT: about one hundred to one hundred sixty gigabytes — multi-A-one-hundred, fifty thousand dollars of GPUs. Seventy-billion full FT: about one point zero to one point four terabytes. Eight to sixteen H-one-hundreds, multi-node. Nobody does this casually.

Read that QLoRA row and that full-FT row again. The spread between QLoRA at ten to sixteen gigabytes and full FT at one hundred to one hundred sixty gigabytes — for the same seven-billion model — is roughly ten times. That is the price of optimizer states and full gradients. Almost no steering task justifies paying it. The course default: start at QLoRA, escalate only with evidence.

[SLIDE 5 — Why PEFT is cheap (the optimizer argument)]

Let me state the economic argument for PEFT bluntly, because it is the whole game. Full fine-tuning's cost is dominated by optimizer states that PEFT never materializes.

In QLoRA and LoRA, only the adapter parameters are trainable — typically under one percent of the model. The optimizer states for an adapter are rounding error. The base weights stay frozen. They contribute no gradient and no AdamW states. That is why a seven-billion QLoRA fits in ten to sixteen gigabytes while a seven-billion full FT needs one hundred to one hundred sixty. The model is the same. The optimizer isn't.

This is worth repeating until it is reflex: full fine-tuning is expensive because of optimizer states over all parameters. PEFT is cheap because it makes optimizer states exist over almost nothing.

[SLIDE 6 — The three-question framework]

Before you open a model loader, answer three questions. They determine the GPU class. This is the planning tool you will use on every real job.

Question one: how big is the model? One billion, three billion, seven billion, seventy billion, four-oh-five billion.

Question two: full FT, or PEFT — LoRA or QLoRA? This is the ten-times cost lever. Default to QLoRA.

Question three: what context length do you need? This is the quadratic lever and the most underestimated. Be honest — your training data's longest sequence sets the floor.

Three numbers in, one GPU class out. One-billion QLoRA at four-K context: laptop, Mac, Colab T-four free tier. Seven-billion QLoRA at two to four-K: RTX forty-ninety, twenty-four gigs. Seven-billion LoRA at four-K: A-one-hundred, or a forty-ninety with checkpointing. Seven-billion full FT: multi-A-one-hundred, fifty thousand dollars. Seventy-billion QLoRA at four-K: one A-one-hundred eighty. Seventy-billion full FT: eight to sixteen H-one-hundreds, multi-node.

You will rarely need more precision than this for planning. The lab turns it into a function.

[SLIDE 7 — The knobs: context length]

Now the five variables that move the numbers. Learn what each buys you.

First, context length. This is the most underestimated cost in the field. Naive attention stores an N-by-N attention matrix per layer per head. Double the context and that matrix quadruples. At eight-K context on a seven-billion model you may spend more VRAM on attention activations than on the weights themselves.

The fix is FlashAttention two or three. It fuses the attention computation so the N-by-N materialization never happens. Memory goes from quadratic to linear, and you get a twenty to thirty percent speed bonus on top. In twenty-twenty-six it is effectively mandatory for anything beyond toy context. If your training config does not have attention-implementation equals flash-attention-two, treat that as a bug.

[SLIDE 8 — The knobs: batch size, checkpointing, optimizer, precision]

Second: batch size. Physical batch size multiplies activation memory linearly. The fix is gradient accumulation. Run micro-batches of size one or two and sum gradients over N steps before the optimizer step. You get the effective batch size of N — the gradient noise profile you wanted — at the physical memory cost of one. It is the single most common fix for a training OOM and it costs you nothing but wall-clock time.

Third: gradient checkpointing. Discard most activations during the forward pass, recompute them during the backward pass. The rule of thumb: about a sixty to seventy percent cut in activation memory for about thirty percent slower training. Turn it on by default for anything that OOMs. It is the cheapest gigabyte you will ever buy.

Fourth: optimizer choice. Full AdamW carries the sixteen-bytes-per-param bill. Paged AdamW eight-bit, from bitsandbytes, quantizes the optimizer states to eight-bit — roughly halving the optimizer footprint at negligible quality cost. For QLoRA it barely matters because the adapter optimizer is already tiny. It matters most for full FT and for LoRA with large adapter ranks. Make it your default.

Fifth: precision of the base. Going from FP sixteen — two bytes — to four-bit — half a byte — quarters the weight footprint. For QLoRA this is the whole point. For inference, four-bit is usually the right deployment default.

[SLIDE 9 — Inference VRAM]

Inference is the simpler half. No optimizer states, no full gradients. VRAM is dominated by the weight footprint, with a smaller chunk for the KV cache and activations that grows with context.

Weights are roughly params times bytes per param. FP sixteen seven-billion: about fourteen gigabytes. Four-bit seven-billion: about three-point-five to four gigabytes. A Q-four-K-M GGUF of seven-billion runs in about six gigabytes once you include runtime overhead. Four-bit seventy-billion: about thirty-five gigabytes of weights, runs in about forty with overhead — fits a single forty-eight or eighty-gig card.

Add about one to two gigabytes for KV cache and activations at short context, and watch it climb as context grows. A seventy-billion model at thirty-two-K context can spend ten or more gigabytes on KV cache alone. For capacity planning during fine-tuning, the inference footprint is your floor. Training always costs more than inference for the same model.

[SLIDE 10 — Apple Silicon]

Do not overlook Apple Silicon. An M-series Mac has unified memory — the GPU and CPU share the same pool, so a sixty-four-gigabyte Mac presents about forty-eight to fifty-eight gigabytes of usable GPU memory with no duplication. For small-model QLoRA — one-billion to seven-billion — this is a genuinely viable training path, especially via MLX or PyTorch's MPS backend. A seven-billion QLoRA at ten to fourteen gigabytes fits comfortably on a thirty-two-gigabyte Mac. A one-billion QLoRA fits on a sixteen-gigabyte Mac.

The catch: MPS throughput is lower than a dedicated NVIDIA card, and not every kernel is optimized. But for iteration, debugging, and small steering experiments, a laptop you already own beats a cloud GPU you have to rent. For the course's orientation work and many PEFT experiments, Apple Silicon is a first-class target, not a fallback.

[SLIDE 11 — Anti-patterns]

Three anti-patterns to leave with.

First: underestimating context length. The classic OOM. You plan for a seven-billion QLoRA at two-K context, compute about twelve gigabytes, provision a sixteen-gig card, then load your real data and discover the longest sequences are eight-K. Attention activations blow past the budget and the job dies at step three. Always size for your ninety-ninth-percentile sequence length, not the mean, and confirm FlashAttention is on.

Second: forgetting optimizer states in full FT. "We have a seven-billion model at fourteen gigabytes and an eighty-gig card, so full fine-tuning will fit comfortably." It will not. Full FT carries sixteen bytes per param — about one hundred twelve gigabytes before activations — because of the FP thirty-two master copy and AdamW's two moments. The fourteen-gigabyte inference footprint is irrelevant to the training budget.

Third: ignoring activation memory. Treating VRAM as just the weights. It is weights plus optimizer plus activations, and activations are the part that scales with your data. A seven-billion QLoRA is about three-point-five gigabytes of weights and six to ten gigabytes of overhead, most of it activations. The weights are the floor. The activations are what actually OOMs you.

[SLIDE 12 — What you can now do]

You can now explain why memory, not compute, is the binding constraint. You can name the three VRAM consumers and say which dominates for QLoRA versus full FT. You can apply the rules of thumb to place a job in the right GPU class. You can use the three-question framework — model size, method, context length — to pick a card before you touch a model loader. And you know the five knobs that give you memory back.

The lab builds a VRAM calculator that turns all of this into a Python function. Run it. Validate it against three real jobs. Make the numbers yours.

Next, module FT zero two: The Open Spectrum. Now that you know whether you can afford to load a model, the next foundations question is whether you can audit one. Can you prove what the base model saw during pretraining? That distinction — open-weights versus open-data — is load-bearing for everything in sensitive domains. Let's get to it.

---

*End of module FT01. Duration: approximately thirty-seven minutes at one-hundred-forty words per minute.*