Dataset Formats

Module FT04 · Course 3 — LLM Fine-Tuning Masterclass

45 minutes · The substrate layer of Pillar 1 (Data)

Three formats. One template. The inspection loop that catches every silent bug.

Pillar 1 — Data · FT04

Why this module is the substrate

The dataset is the steering wheel (FT00). Format errors make the wheel crooked — at the byte level.

The model trains. The loss drops. The model is subtly wrong.

You blame data quality, LoRA rank, learning rate — never the one-character whitespace bug.

This module is how you keep the wheel true before any later module depends on it.

The three canonical formats

Recognize each on sight. Normalize to one.

FormatShapeOrigin
Raw messages[{role, content}]The lingua franca
Instruction / responseinstruction + outputAlpaca / Vicuna
ShareGPT{conversations: [{from, value}]}Community chat
Whatever arrives, you convert it to a single messages column. One internal representation; one conversion; one bug surface.

One internal representation

Valid roles: system · user · assistant · tool. Anything else is a bug.

# ShareGPT -> messages (the conversion you'll run most)
ROLE_MAP = {"human": "user", "gpt": "assistant",
            "system": "system", "tool": "tool"}

def sharegpt_to_messages(conv):
    return [{"role": ROLE_MAP[t["from"]], "content": t["value"]}
            for t in conv["conversations"]]

Result is identical to the raw-messages shape — proof all three formats are import conventions for one underlying conversation.

apply_chat_template — the one right way

Always apply the model's chat template. Never hand-concatenate strings. 
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,             # string back, not token IDs
    add_generation_prompt=False # full conversation (training)
)

The template is a Jinja file shipped with the tokenizer — the exact bytes the model was instruction-tuned against.

FamilyDelimiters
ChatML (Qwen, Yi)<|im_start|> ... <|im_end|>
Llama-3<|start_header_id|> ... <|eot_id|>
MiniCPMChatML variant, different EOS

Why hand-concat is the #1 silent bug

Three failure modes — each invisible until you decode the tokenized output.

BugWhat happens
Token-boundary errorsOff-by-one whitespace → different BPE merges → model trains on a sequence it never saw
Special-token merging<|im_start|> as raw text → 6 ordinary tokens, not 1 special ID
Wrong role tokensChatML delimiters on a Llama-3 model → unknown text, garbage boundaries
Loss still drops. Trainer still runs. Model is subtly worse. You debug for days and never find the one-character bug.

The inspection loop

The five-second check that catches every format bug.

# 1. Render to string
text = tokenizer.apply_chat_template(messages, tokenize=False)

# 2. Tokenize
inputs = tokenizer(text, return_tensors="pt")

# 3. THE INSPECTION — read this
print(tokenizer.decode(inputs["input_ids"][0]))
The decoded text is the ground truth of what the model sees. The raw messages are not.

10 seconds of reading saves 10 hours of "why is the model broken." Once per dataset. Non-negotiable.

Checking the special tokens

for tok in ["<|im_start|>", "<|im_end|>", "<|eot_id|>"]:
    if tok in tokenizer.added_tokens_encoder:
        print(f"{tok!r:18} -> single token ID "
              f"{tokenizer.added_tokens_encoder[tok]}  (CORRECT)")
    else:
        print(f"{tok!r:18} -> NOT a special token  (BUG)")
If a token the template uses is absent from added_tokens_encoder, your tokenizer and model are mismatched. Stop. Reconcile before any training.

SFT vs preference data

Two objectives. Two shapes. Don't mix them.

SFT — one correct response

{"messages": [
  {"role": "user", "content": q},
  {"role": "assistant", "content": a}
]}

Cross-entropy on the assistant turn. → SFTTrainer (FT12)

Preference — chosen vs rejected

{"prompt":   [{user...}],
 "chosen":   [{assistant good}],
 "rejected": [{assistant bad}]}

Contrastive: chosen > rejected. → DPOTrainer (FT13)

Defining property of preference data: chosen and rejected share the same prompt and differ only in the final assistant turn.

The bug taxonomy

BugSymptomRoot cause
Missing EOSModel never stopsNo closing im_end / eot_id
Wrong role tokensCan't follow instructionsChatML template on Llama-3
Specials as textTrains on literal textHand-concat, not template
Trailing spacesStopping driftsSpace before closing token
Loss on user turnsModel echoes promptlabels not -100-masked
Mixed formatsHalf dataset malformedDidn't normalize before map()
All caught by: tokenizer.decode(input_ids[0])  ·  None caught by: loss curves.

Anti-patterns

Hand-concatenating strings. Looks simpler than the template. It is a bug deferred — token-boundary errors, specials-as-text, wrong role tokens.
Not inspecting tokenized output. If you've never called decode on your input_ids, you're flying blind. Inspect once per dataset.
Mixing formats. Half instruction/response, half messages. Normalize to one format up front.
Assuming the tokenizer handles roles. The tokenizer knows tokens, not roles. The mapping is what the template encodes.

The connection to the thesis

FT00: the dataset is the steering wheel. Bad data = bad direction, no matter the optimizer.

FT04 is the wheel itself. Format errors corrupt it at the byte level:

  • The model trains on tokens that differ from its instruction-tuning distribution
  • It steers subtly off-course
  • You blame data / rank / LR for days — never the whitespace
The fix: apply_chat_template + the inspection loop. Keep the substrate clean so PEFT, SFT, DPO can trust their inputs.

What you can now do

  1. Recognize the three canonical formats and convert any to messages.
  2. Apply the chat template — and explain why hand-concat is a silent bug.
  3. Structure SFT and preference data correctly behind the right trainer.
  4. Run the inspection loop to catch format bugs before a training run.
Lab: "Format, Template, Inspect" — convert raw Q/A to all three formats, apply the template, decode the tokenized result, catch the planted bug. CPU only, ~30 min.

Next: FT05 — Synthetic Data Generation

Course 3 — LLM Fine-Tuning Masterclass · FT04 · Pillar 1