{ "module": "FT04 — Dataset Formats", "course": "3 — LLM Fine-Tuning Masterclass", "version": "1.0.0", "duration_minutes": 35, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis-design", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "What are the three canonical dataset formats for LLM fine-tuning?", "options": [ "CSV, JSON, and Parquet", "Raw messages ({role, content}), instruction/response (instruction + output), and ShareGPT ({conversations: [{from, value}]})", "Training, validation, and test splits", "Supervised, preference, and RLHF formats" ], "answer_index": 1, "rationale": "The three canonical shapes are raw messages (the internal lingua franca), instruction/response (the Alpaca/Vicuna two-column shape), and ShareGPT (the community shape with from/value and human/gpt role names). CSV/JSON/Parquet are file formats, not dataset formats; splits and objectives are orthogonal concerns." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the internal representation that all three canonical formats normalize to?", "options": [ "A single 'messages' column: a list of {'role', 'content'} dicts", "A single 'text' column of pre-rendered strings", "A tensor of input_ids", "A JSON object with 'prompt' and 'completion' keys" ], "answer_index": 0, "rationale": "Whatever shape data arrives in, you convert it to a single 'messages' column (list of {role, content} dicts). From there every downstream tool — apply_chat_template, TRL, the inspection loop — works identically. One internal representation, one conversion, one bug surface." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "What does tokenizer.apply_chat_template() do?", "options": [ "Trains the tokenizer on your dataset.", "Renders messages into the model's native byte sequence by running the Jinja2 chat_template shipped with the tokenizer — the exact format the model was instruction-tuned against.", "Splits the conversation into training and validation sets.", "Counts the tokens in each message for cost estimation." ], "answer_index": 1, "rationale": "apply_chat_template runs the Jinja2 template shipped WITH the tokenizer, producing the exact bytes the model's publisher used during instruction tuning — special tokens, order, whitespace, system-prompt slot. It is the contract between the model and your data." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "You have a JSONL of {'instruction': ..., 'output': ...} rows. What must you do before handing it to a messages-expecting trainer like TRL SFTTrainer?", "options": [ "Nothing — SFTTrainer auto-detects the instruction/response shape.", "Convert each row to a {'messages': [{'role':'user','content':instruction},{'role':'assistant','content':output}]} via ds.map(), then remove the original columns.", "Rename 'instruction' to 'messages' and 'output' to 'response'.", "Concatenate instruction and output into a single 'text' field." ], "answer_index": 1, "rationale": "Normalize to the messages shape once via ds.map(): wrap instruction (+ optional input) as a user message and output as an assistant message, then remove_columns. Leaving data as instruction/response and relying on a formatting_func is where hand-concat bugs creep in. Convert once, sleep at night." }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "You download a ShareGPT-format dataset with {'conversations': [{'from': 'human', 'value': ...}, {'from': 'gpt', 'value': ...}]}. How do you normalize it to messages?", "options": [ "Pass it directly to apply_chat_template — ShareGPT is already the native format.", "Rename the top-level key 'conversations' to 'messages' and leave from/value as-is.", "Map roles via {'human':'user','gpt':'assistant','system':'system','tool':'tool'}, rename 'from'->'role' and 'value'->'content' for each turn.", "Delete the 'from' field; apply_chat_template will infer the role from order." ], "answer_index": 2, "rationale": "ShareGPT uses OpenAI-era role names (human/gpt) and from/value keys. Map each turn: role = ROLE_MAP[t['from']], content = t['value']. The result is identical to the raw-messages format — proving all three formats normalize to one shape. apply_chat_template expects role/content, not from/value." }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "You are building an inference prompt (not training data). What argument do you pass to apply_chat_template, and why?", "options": [ "add_generation_prompt=False — to render the full conversation including the closing special tokens.", "add_generation_prompt=True — to append the assistant-role header and leave the turn open for the model to complete.", "tokenize=True — to get token IDs directly instead of a string.", "add_special_tokens=False — to skip the BOS token." ], "answer_index": 1, "rationale": "For inference you want the prompt to END with the assistant header so the model starts generating the response. add_generation_prompt=True appends the assistant-role opening and leaves the turn open. For training you use False so the full conversation (with EOS) is rendered. Getting this backwards truncates training data or yields an incomplete inference prompt." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "Which snippet is the correct inspection loop to verify your dataset's formatting before training?", "options": [ "print(dataset['train'][0])", "text = tokenizer.apply_chat_template(messages, tokenize=False); inputs = tokenizer(text, return_tensors='pt'); print(tokenizer.decode(inputs['input_ids'][0]))", "print(tokenizer.vocab_size)", "loss = model(**inputs).loss; print(loss.item())" ], "answer_index": 1, "rationale": "The inspection loop: render with apply_chat_template (tokenize=False), tokenize the result, then DECODE the input_ids back to text and read it. The decoded text is the ground truth of what the model sees. Raw messages are not. Loss values will not reveal format bugs. Do this once per dataset." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "You want to confirm that '<|im_start|>' is a single special token (not ordinary text) in your tokenizer. What do you check?", "options": [ "tokenizer.vocab_size", "tokenizer.added_tokens_encoder — if '<|im_start|>' is a key, it maps to a single token ID", "tokenizer.model_max_length", "tokenizer.pad_token_id" ], "answer_index": 1, "rationale": "added_tokens_encoder maps each special token string to its single token ID. If '<|im_start|>' is present, the tokenizer treats it as one token. If absent, the template's delimiters will be split into ordinary text tokens — the catastrophic 'specials-as-text' bug. Print this dict for every special token the template uses before training." }, { "id": "Q09", "bloom": "application", "type": "multiple_choice", "prompt": "You need preference data for DPO. Which structure is correct?", "options": [ "{'messages': [{'role':'user','content':q},{'role':'assistant','content':a}]}", "{'prompt': [{'role':'user','content':q}], 'chosen': [{'role':'assistant','content':good}], 'rejected': [{'role':'assistant','content':bad}]}", "{'instruction': q, 'output': good, 'negative': bad}", "{'text': q + good + bad}" ], "answer_index": 1, "rationale": "Preference data has a shared prompt plus two divergent completions. Shape A is {prompt, chosen, rejected} where chosen/rejected are assistant messages; Shape B is {chosen: [full conv], rejected: [full conv]}. The defining property: chosen and rejected share the same prompt and differ only in the final assistant turn. Option A is SFT data (one response); the others are not accepted by DPOTrainer." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is hand-concatenating chat strings (f'<|im_start|>user\\n{q}<|im_end|>') a silent bug rather than an obvious one?", "options": [ "Because Python raises no error — the string is valid, the tokenizer accepts it, the trainer runs, and the loss drops. The model is just subtly worse because it trained on a slightly out-of-distribution token sequence.", "Because hand-concatenation is faster than apply_chat_template.", "Because the tokenizer automatically fixes whitespace errors.", "Because modern models are robust to formatting differences." ], "answer_index": 0, "rationale": "The bug is silent because nothing errors: the string is valid Python, the tokenizer produces token IDs, training runs, loss decreases. But the token sequence differs from what the model saw during instruction tuning (wrong whitespace, specials-as-text, wrong role tokens), so the model is trained slightly out of distribution. You blame data/rank/optimizer for the subtle quality drop and never find the one-character whitespace bug." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "A colleague copies a formatting snippet from a Qwen (ChatML) tutorial and applies it to a Llama-3 model. What happens?", "options": [ "Nothing — all models use the same chat format.", "apply_chat_template automatically adapts the format to the model.", "The Llama-3 tokenizer has no <|im_start|>/<|im_end|> tokens, so they are treated as unknown text; the model trains on garbage delimiters and cannot follow instructions. This is the 'wrong role tokens' bug.", "The model trains fine but runs slower." ], "answer_index": 2, "rationale": "Every model family has its own special tokens and template. Llama-3 uses <|start_header_id|>/<|eot_id|>; it has no <|im_start|>. If you hand-concat ChatML delimiters onto a Llama-3 model, those strings tokenize as ordinary text (or [UNK]), and the model learns a completely different vocabulary boundary. apply_chat_template reads the tokenizer's OWN template, preventing this mismatch." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "After training, your model generates text and never stops until it hits max_tokens. Inspecting the decoded input_ids of your training data, what do you expect to find?", "options": [ "The training data is too short.", "The learning rate was too high.", "The final assistant turn is not closed with the model's end token (<|im_end|> / <|eot_id|>) — the 'missing EOS' bug. The model never learned the stop signal.", "The LoRA rank was too low." ], "answer_index": 2, "rationale": "The 'won't stop generating' symptom maps to the 'missing EOS' bug: the last assistant turn was not closed with the end-of-turn special token. The model was never trained to emit that token, so at inference it runs until max_tokens. Caught by decoding input_ids and checking the final token — never caught by loss curves." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is it a bug to feed SFT data (one assistant response per prompt) to a DPO trainer, or vice versa?", "options": [ "It isn't — both trainers accept any messages-shaped data.", "SFT data has no chosen/rejected split, so DPO has nothing to contrast. Preference data has no single 'correct' response, so SFT would train on whichever response appeared first. The data shape signals the training objective; mismatching them silently trains the wrong objective.", "DPO requires more VRAM than SFTTrainer.", "SFT data must be in CSV format; DPO data must be in JSONL." ], "answer_index": 1, "rationale": "The shape of the data encodes the objective. SFT (one response) trains cross-entropy; DPO (chosen vs rejected) trains a contrastive ranking. Feeding SFT data to DPO leaves no contrast to learn; feeding preference data to SFT trains on an arbitrary one of the two responses. Normalize to the shape that matches your trainer's objective." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "You decode the tokenized training input_ids and notice '<|im_start|>' appears in the output as the characters '<', '|', 'i', 'm', '_' rather than as a single rendered special token. What is the bug, and what is the fix?", "options": [ "The model is too small; use a larger one.", "The 'specials-as-text' bug: the template's delimiters are being tokenized as ordinary text instead of single special-token IDs. Fix: ensure you used apply_chat_template (not hand-concat) AND that '<|im_start|>' is registered in tokenizer.added_tokens_encoder. If absent, your tokenizer and model are mismatched.", "The training data is corrupt; re-download it.", "The tokenizer vocabulary is too small; increase vocab_size." ], "answer_index": 1, "rationale": "If '<|im_start|>' renders as ordinary characters in the decoded output, it is being tokenized as multiple ordinary tokens, not one special ID. Two root causes: (1) you hand-concatenated instead of using apply_chat_template (which marks specials correctly), or (2) the token is absent from added_tokens_encoder, meaning your tokenizer and model are mismatched. Verify via added_tokens_encoder before any training." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "How does the format substrate (this module) connect to the FT00 thesis that 'the dataset is the steering wheel'?", "options": [ "Format is irrelevant — only data quality matters.", "FT00 says bad data = bad direction no matter the optimizer. Format errors (hand-concat, missing EOS, wrong role tokens) make the wheel crooked: the model trains on slightly-wrong tokens and steers subtly off-course. apply_chat_template + the inspection loop is how you keep the wheel true before any later module depends on it.", "Format only matters for inference, not training.", "The steering wheel is the optimizer, not the dataset." ], "answer_index": 1, "rationale": "FT00 established that the dataset IS the steering wheel and that a sophisticated algorithm on bad data steers precisely into a wall. FT04 is the first module of Pillar 1 (Data) because format errors corrupt the wheel at the byte level: the model trains on tokens that differ from its instruction-tuning distribution and steers subtly wrong. apply_chat_template and the inspection loop are the practices that keep the substrate clean so every later module (PEFT, SFT, DPO) can trust its inputs." } ] }