# Teaching Script — Module FT04: Dataset Formats

**Course**: Course 3 — LLM Fine-Tuning Masterclass
**Module**: FT04 — Dataset Formats
**Duration**: ~35 minutes (spoken at ~140 wpm)
**Format**: Verbatim transcript with `[SLIDE N]` cues. Read aloud or use as speaker notes.

---

[SLIDE 1 — Title]

Welcome to Pillar One. This is module FT zero-four, Dataset Formats — the first module of the data pillar, and the substrate layer for everything that follows.

In module FT zero-zero I said the dataset is the steering wheel. Bad data means bad direction, no matter how good your optimizer. This module is about the wheel itself — the bytes your data becomes before the model ever sees them. Get the substrate wrong and every later module — PEFT, SFT, DPO, GRPO — builds on a crooked foundation. The model trains. The loss drops. The model is subtly wrong. And you will spend days blaming your data quality, your LoRA rank, your learning rate, before you ever find the one-character whitespace bug that actually caused it.

That is the entire module in one sentence. Let's make it concrete.

[SLIDE 2 — The three canonical formats]

There are exactly three shapes your data can arrive in. Learn to recognize each on sight.

Format one — raw messages. A list of role and content dicts. Role is one of system, user, assistant, or tool. This is the internal lingua franca. It is what the tokenizer actually consumes and what apply_chat_template expects. If your data is already in this shape, you are done with conversion.

Format two — instruction slash response. The classic Alpaca shape. Two columns: instruction and output. Sometimes a third, input. This is what most fine-tuning datasets on the Hub look like because it is trivial to produce from a CSV. You convert it to messages before training.

Format three — ShareGPT, the conversational shape. A top-level conversations key, each turn is from and value, with role names human and gpt — the OpenAI-era vocabulary, not the modern one. This is what most community-shared chat datasets use. You convert it too.

[SLIDE 3 — One internal representation]

Here is the pattern across all three. Whatever shape your data arrives in, you normalize it to a single messages column. From there, every downstream tool works identically. Apply_chat_template, TRL, the inspection loop — they all speak messages. One internal representation, one conversion, one bug surface.

The ShareGPT conversion is the one you will run most often. Map human to user, gpt to assistant, system stays system, tool stays tool. Rename from to role and value to content. The result is byte-for-byte identical to the raw-messages shape. That is the proof that all three formats are just different import conventions for the same underlying conversation.

[SLIDE 4 — apply_chat_template, the one right way]

Now the highest-leverage five minutes in the whole module. Most silent training failures trace back to skipping this.

The rule. Always apply the model's chat template. Never hand-concatenate strings.

A chat template is a Jinja file shipped with the tokenizer. It encodes the exact byte sequence the model was instruction-tuned against — the special tokens, their order, the whitespace, the newline placement, the system-prompt slot. For ChatML models like Qwen, it produces the im-start and im-end tokens. For Llama-3, it produces start-header-id and eot-id. For Qwen it is im-start again but with different end-of-sequence handling. Every family is different, and the differences are invisible to the eye but catastrophic to training.

The call is one line. tokenizer.apply_chat_template, pass the messages, set tokenize to False to get a string back, set add_generation_prompt to False for training or True for inference. That single call replaces every hand-rolled f-string you have ever written.

[SLIDE 5 — Why hand-concat is the number-one silent bug]

Why is hand-concatenation the number-one silent bug? Three failure modes, each invisible until you decode the tokenized output — which is why nobody catches them.

First, token-boundary errors. Tokenizers merge text greedily at whitespace. The string im-start with no space and im-start with a leading space tokenize to different token sequences, because the whitespace changes how the byte-pair merges happen. When you hand-concat, you almost always get the whitespace wrong by one character. The model sees a sequence it never saw during instruction tuning. Training still runs. Loss still drops. The model is just worse.

Second, special-token merging. Write im-start as a raw string and the tokenizer treats it as literal text — six or seven ordinary tokens — unless you remembered add_special_tokens equals True plus the added-tokens decoder is set up correctly, which it only is if you used the template. With the template, im-start becomes a single token ID. Without it, the model learns a completely different vocabulary boundary. This is the bug that produces "the model trained fine but talks like a different person."

Third, wrong role tokens entirely. A Llama-3 model has no im-start token. A ChatML model has no start-header-id. Copy a formatting snippet from a Qwen tutorial and apply it to a Llama-3 model, and the tokenizer falls back to treating those delimiters as unknown text. The model trains on garbage boundaries and you wonder why it cannot follow instructions.

Apply_chat_template solves all three at once. It reads the tokenizer's own Jinja file — the same file the publisher used during instruction tuning — and produces the exact bytes the model expects. There is no scenario in which hand-concatenation is safer, faster, or simpler. It is a bug waiting one training run to fire.

[SLIDE 6 — The inspection loop]

Now the habit that separates engineers who ship from engineers who debug. The inspection loop.

After apply_chat_template, the next call is tokenizer of the text. That produces input_ids, attention_mask, and if you ask, labels. Before you hand those to a trainer, decode the input_ids back to text and read the result.

The snippet is three lines. text equals apply_chat_template, messages, tokenize False. inputs equals tokenizer of text, return_tensors pt. Then print tokenizer dot decode of inputs input_ids zero. Read the output.

What you are looking at is what the model will actually see during training — token by token, special token by special token. The bugs are obvious once you look. A missing im-end or eot-id at the end means the model never learns to stop. A literal im-start rendered as the characters less-than, pipe, i, m, underscore means the special-token-merging bug. A trailing space before the closing token means stopping drift.

Ten seconds of reading decoded text saves ten hours of why-is-the-model-broken. Do this once per dataset. It is non-negotiable.

[SLIDE 7 — Checking the special tokens]

A second, sharper inspection. Print the token IDs of the special tokens specifically. For each token the template uses — im-start, im-end, endoftext — check whether it appears in tokenizer dot added_tokens_encoder. If it is a key, it maps to a single token ID and you are correct. If it is absent, that token will be split into ordinary text — the catastrophic mismatch bug. Your tokenizer and your model are mismatched. Stop. Reconcile before training.

This is the check that catches the worst bug of all: loading a tokenizer from a different model than the one you are training. The template may look fine in decoded text, but the special-token registry reveals the mismatch instantly.

[SLIDE 8 — SFT data versus preference data]

Now two distinct training objectives, two distinct data shapes. Mixing them is a bug.

SFT data — supervised fine-tuning. The shape from earlier: a list of messages ending in an assistant turn. There is exactly one right answer per example. The trainer computes cross-entropy loss on the assistant turns and ignores the rest via the loss mask. SFT steers format and instruction-following. It does not compare alternatives.

Preference data — two responses to the same prompt, one preferred. Used by DPO, KTO, IPO, the whole preference-optimization family. Two acceptable shapes. Shape A: an explicit prompt key, a chosen key, a rejected key, each holding message lists. Shape B: full conversations under chosen and rejected, each ending in the assistant turn. TRL's DPOTrainer accepts both.

[SLIDE 9 — The defining property of preference data]

The defining property of preference data: chosen and rejected share the same prompt and differ only in the final assistant turn. If they differ earlier in the conversation, your dataset is malformed and DPO will produce nonsense.

The mistake to avoid: feeding SFT data to a DPO trainer, or vice versa. SFT data has no chosen-rejected split, so DPO has nothing to contrast. Preference data has no single correct response, so SFT would train on whichever you happened to put first. The shape signals the objective. Match the shape to the trainer.

[SLIDE 10 — The bug taxonomy]

A checklist of the format bugs you will encounter, ranked by how silently they fail.

Missing EOS. Model never stops generating. Cause: last assistant turn not closed with im-end or eot-id.

Wrong role tokens. Model can't follow instructions. Cause: copied a ChatML template onto a Llama-3 model, or vice versa.

Specials as text. Model trains on literal im-start text. Cause: hand-concatenated instead of apply_chat_template.

Trailing spaces. Stopping behavior drifts. Cause: a stray space before the closing special token.

Loss on user turns. Model echoes the prompt. Cause: labels not masked to negative one hundred on non-assistant tokens.

Mixed formats. Half the dataset malformed. Cause: didn't normalize before map.

Every one of these is caught by the inspection loop. None is caught by loss curves. That asymmetry — silent in training, obvious in inspection — is why inspection is non-negotiable.

[SLIDE 11 — Anti-patterns]

Four anti-patterns to leave with.

First, hand-concatenating strings. It looks simpler than apply_chat_template. It is not. It is a bug deferred. Always use the template.

Second, not inspecting tokenized output. If you have never called decode on your input_ids, you are flying blind. The decoded text is the ground truth. The raw messages are not. Inspect once per dataset.

Third, mixing formats. Half instruction-response, half messages, fed to the same trainer. Normalize to one format up front.

Fourth, assuming the tokenizer handles roles. The tokenizer does not know what a role is. It knows tokens. The mapping from role to token is what the chat template encodes. Skip the template and you are guessing.

[SLIDE 12 — The connection to the thesis]

How does this connect to FT zero-zero? The thesis was that the dataset is the steering wheel — bad data means bad direction, no matter the optimizer. This module is about the wheel itself. Format errors make the wheel crooked at the byte level. The model trains on tokens that differ from its instruction-tuning distribution, and steers subtly off-course.

Apply_chat_template and the inspection loop are the practices that keep the substrate clean. Do them right, and every later module — PEFT, SFT, DPO, GRPO — can trust its inputs. Skip them, and you are building a sophisticated steering system on top of a wheel that is off by one character.

[SLIDE 13 — The lab]

The lab is called Format, Template, Inspect. You take a raw CSV of question-answer pairs, convert it to all three canonical formats, apply the model's chat template, decode the tokenized result, and catch a deliberate formatting bug I planted in the starter data. CPU only — we use the tokenizer, never the model. About thirty minutes. By the end you will have felt every concept in this module on your own machine.

[SLIDE 14 — What you can now do]

You can now recognize the three canonical formats and convert any of them to messages. You can apply the chat template and explain why hand-concatenation is a silent bug. You can structure SFT and preference data correctly. And you can run the inspection loop to catch format bugs before they cost you a training run.

That is the substrate. Next, module FT zero-five: Synthetic Data Generation. Because real data is scarce, expensive, and often wrong — and the way you generate synthetic data determines whether your steering wheel is true or crooked at the source.

---

*End of module FT04. Duration: approximately thirty-five minutes of spoken content at a deliberate pace including slide transitions and code reading.*