Module FT04 — Dataset Formats

Dataset Formats

The three canonical dataset formats, the one right way to apply a chat template, and the five-second inspection loop that catches every silent formatting bug before it costs you a training run.

minutes

artifacts

sub-sections

Fine-tuning steers behavior; the dataset is the steering wheel. This module is the wheel itself — the bytes your data becomes before the model ever sees them. Get the substrate wrong and the model trains, the loss drops, and the model is subtly wrong, while you spend days blaming data quality, LoRA rank, and learning rate instead of the one-character whitespace bug that actually caused it.

Key Claims

Load-Bearing Claims

There are exactly three canonical dataset formats — raw messages, instruction/response, and ShareGPT — and all three normalize to a single `messages` column. One internal representation, one conversion, one bug surface.

Always apply the model's chat template with `apply_chat_template`. Never hand-concatenate strings. Hand-concat is the #1 silent bug in fine-tuning: token-boundary errors, special-token merging, wrong role tokens. Loss still drops; the model is subtly worse.

Decode the tokenized `input_ids` and read the result before training. The decoded text is the ground truth of what the model sees. Every format bug — missing EOS, trailing spaces, specials-as-text — is visible here; none is visible in loss curves.

SFT data (one response) and preference data (chosen/rejected) are structurally distinct. The shape signals the objective; mismatching them silently trains the wrong objective.

After This Module

Recognize the three canonical dataset formats — raw messages, instruction/response, and ShareGPT — and load each correctly with HuggingFace datasets, normalizing all to a single messages column.

Apply the model's chat template with apply_chat_template and explain why hand-concatenating strings is the single most common silent bug in fine-tuning (token-boundary errors, special-token merging, wrong role tokens).

Structure SFT data (one correct response) versus preference data (chosen/rejected pairs) and place each behind the correct trainer (SFTTrainer vs DPOTrainer).

Run the end-to-end inspection — decode the tokenized input_ids back to text — to catch format bugs before they cost a training run, and classify each bug from the format-bug taxonomy.

Artifacts

Teaching Document

~3,000 words; 5 sub-sections — the three formats, apply_chat_template (the one right way), the inspection loop, SFT vs preference data, the bug taxonomy

READ

Diagrams

5 diagrams — the three formats side by side, the apply_chat_template flow (with anti-pattern branch), SFT vs preference structure, the inspection loop, the bug taxonomy matrix

READ

Slide Deck

reveal.js slide deck, dark theme, design-system teal

READ

Teaching Script

Verbatim teaching transcript, [SLIDE N] cues