"What is TRL and what is its role in the HuggingFace stack?"	"TRL (Transformers Reinforcement Learning) is the canonical post-training library. v1.0 released March 31, 2026. 75+ training methods, ~3M downloads/month, ships a Stability Contract and production CLI. It is the library you reach for to steer a transformer model: SFT, DPO, KTO, RLOO, GRPO, reward modeling, async RL."	c3::ft11::recall
"Name the six stages of the SFTTrainer end-to-end pipeline, in order."	"(1) Load dataset (chat-format, train/eval split). (2) Load model (bf16, FlashAttention). (3) Attach PEFT config (LoraConfig or None). (4) Build SFTConfig (TrainingArguments levers). (5) trainer.train() (loss curve, eval every N steps). (6) Save / merge / load (adapter or full model)."	c3::ft11::recall
"What does SFTConfig extend, and what does it add?"	"SFTConfig extends transformers.TrainingArguments and adds SFT-specific fields: packing (pack short examples into one sequence for throughput), max_length, and completion-only loss masking (train on the assistant's response, not the prompt). It holds all the training levers: LR, batch size, scheduler, eval, logging."	c3::ft11::recall
"What are the three failure modes of a training loss curve, and the probable cause of each?"	"(1) Loss not decreasing: LR too low, data problem, or template bug (FT07). (2) Loss exploding (NaN/spike): LR too high, numerical instability/FP16 overflow (use BF16), or EOS/template bug producing degenerate targets. (3) Loss plateaus: converged (eval flat and low) OR stuck (eval rising = overfitting; try LR warmup/decay, more data)."	c3::ft11::application
"What is the typical learning rate range for full FT vs LoRA, and why the difference?"	"Full FT: 1e-5 to 5e-5. LoRA: 1e-4 to 5e-4 (an order of magnitude higher). Adapters are small and start random; they need faster learning to catch up to the frozen base. Using a full-FT LR on LoRA = loss barely moves; using a LoRA LR on full FT = loss explodes."	c3::ft11::recall
"What is the formula for effective batch size, and why does gradient accumulation matter?"	"effective_batch = per_device_batch_size x gradient_accumulation_steps x world_size. When VRAM-limited you can't fit a large per_device_batch, so you accumulate gradients across several forward/backward passes before stepping the optimizer. This simulates a large batch without the memory cost (at the cost of wall-clock time)."	c3::ft11::recall
"What is gradient checkpointing, and what is the tradeoff?"	"Recompute activations during backward instead of storing them. Cuts activation memory ~60-70% for ~30% slower wall-clock time. It is the FT01 VRAM-math lever — turn it on before reducing batch size when you can't fit the model."	c3::ft11::recall
"Why is BF16 preferred over FP16 for training, and what is the failure if you use FP16?"	"FP16 has a narrow dynamic range; gradients can overflow to inf during backward, producing NaN loss. BF16 has the same memory footprint but a wider range, so it doesn't overflow. On Ampere-or-later hardware, always use BF16. If you're on pre-Ampere (no BF16), you're stuck with FP16 + gradient clipping and should plan to upgrade."	c3::ft11::application
"What is FlashAttention 2/3 and why is it effectively mandatory for SFT?"	"FlashAttention computes the exact attention but in tiles that fit in SRAM, avoiding materializing the full attention matrix. For long sequences it's a 2-4x speedup and a large memory saving. Set attn_implementation='flash_attention_2' when loading the model. Skipping it on serious SFT is leaving 2-4x throughput on the table."	c3::ft11::application
"What is the grad norm, and what does a spike tell you?"	"The grad norm is the norm of the gradient vector. A healthy run has a stable, slowly-decreasing grad norm. A sudden 10x spike between two logging steps is the early-warning system for instability — the run is about to explode (NaN) even if the loss looks fine. Kill it, lower the LR, ensure gradient clipping (max_grad_norm, default 1.0)."	c3::ft11::analysis
"What are the four optimizer options in TRL, and when do you use each?"	"AdamW (default, robust, most memory — two momentum states per param). AdamW 8-bit / bitsandbytes (quantizes optimizer states to 8-bit, saves memory at small accuracy cost). Adafactor (factored second-moment, low memory, finicky). PagedAdamW (uses CUDA paging to offload states to CPU, avoids OOM — the QLoRA default, pairs with 4-bit quantized bases)."	c3::ft11::recall
"What does merge_and_unload do, and when do you use it vs keeping the adapter separate?"	"merge_and_unload folds a PEFT adapter into the base weights, producing a standalone transformers model with no adapter layer. Use it for DEPLOYMENT (a single model to quantize/serve). Keep the adapter separate for EXPERIMENTATION (swappable, small) or hot-swapping at inference (one base, many adapters via load_adapter)."	c3::ft11::application
"How do you load an adapter for hot-swapping at inference without merging?"	"from peft import PeftModel; model = PeftModel.from_pretrained(base, adapter_path). The base is untouched; the adapter sits on top. To swap to a different adapter: model.load_adapter(other_path). This is the FT00 swappability property in code — Layer 2 detaches from Layer 1 without affecting it."	c3::ft11::application
"What are the three LR scheduler options, and which is the SFT default?"	"cosine (smooth decay following a cosine curve to near-zero — the common default, gentle and reliable). linear (straight-line decay to zero, slightly more aggressive at the end). constant_with_warmup (warmup then flat — used when you train a fixed budget and don't want late-training decay, e.g., you'll pick the best checkpoint)."	c3::ft11::recall
"What is the typical warmup ratio range and why is it needed?"	"0.03 to 0.1. Warmup ramps the LR from zero to the target over the first fraction of steps, stabilizing early training when gradients are noisy and the model is far from any good solution. Skip it and you risk an early spike (especially with a high LR). For LoRA with its higher LRs, warmup is near-mandatory."	c3::ft11::recall
"What are the three logging backends, and what should you log at minimum?"	"Backends: Weights & Biases (report_to='wandb', rich dashboards), Trackio (report_to='trackio', HuggingFace's lighter Spaces-backed tracker), TensorBoard/JSONL (report_to='tensorboard' or 'none'). Log at minimum: train loss, learning rate, grad norm (every step); eval loss (every eval step); throughput periodically to spot slowdowns."	c3::ft11::application
"What is the single most useful plot, and what does the gap between its two lines tell you?"	"Train loss and eval loss on the same chart. The gap is the overfitting signal: if train keeps dropping while eval rises, you are memorizing, not steering. Stop at the eval-loss minimum and use that checkpoint (load_best_model_at_end=True, metric_for_best_model='eval_loss')."	c3::ft11::analysis
"Name the five anti-patterns from the module and why each ruins a run."	"(1) No eval (flying blind — can't tell convergence from overfitting). (2) Wrong LR for the method (full-FT LR on LoRA = no learning; LoRA LR on full FT = explosion). (3) No logging (a NaN'd run and a converged run look identical from the final checkpoint). (4) Ignoring grad norm spikes (the canary before the NaN). (5) FP16 instead of BF16 (overflow → NaN; same memory, strictly worse on supported hardware)."	c3::ft11::analysis
"A student runs a LoRA SFT job with learning_rate=2e-5 and the loss barely moves from step 1. What is the likely cause and the fix?"	"Wrong LR for the method. 2e-5 is the full-FT band; LoRA needs 1e-4 to 5e-4 because adapters are small and start random, needing faster learning against a frozen base. Fix: raise the LR to 2e-4. (Also worth checking: is the chat template being applied / is the completion mask correct? But the LR mismatch is the first suspect.)"	c3::ft11::analysis
"A training run is stable for 200 steps, then the loss jumps to NaN. The student is on FP16 on a Colab T4. Diagnose and fix."	"FP16 overflow (failure mode 2). FP16's narrow dynamic range causes gradients to overflow to inf during backward, producing NaN. The T4 lacks BF16 support. Fixes (in order): (1) reduce per_device_batch_size to 1 and raise gradient_accumulation_steps to keep the effective batch; (2) lower the LR; (3) ensure max_grad_norm=1.0 clipping. The real fix is BF16-capable hardware (Ampere+), but the run can be stabilized on a T4."	c3::ft11::analysis
"What does TRL's packing do, and what is the throughput tradeoff?"	"Packing concatenates multiple short examples into a single sequence (with proper attention masking) to fill the context window. For datasets with short sequences it roughly doubles throughput (samples/sec) because you stop wasting compute on padding tokens. The tradeoff: none significant — packing is a pure win for short-sequence data. Turn it off only for very long-sequence datasets where examples already fill the context."	c3::ft11::analysis
"Why does TRL train only on the completion (masking the prompt), and what happens if you get this wrong?"	"An instruct model should learn to RESPOND, not to parrot the user. TRL applies the model's chat template and masks the prompt tokens from the loss, so gradients flow only from the assistant's response. If completion masking is wrong (or you train on a raw 'text' field), the model learns to imitate the user too — producing a model that echoes prompts instead of answering them. This is the FT07 template bug; the loss curve looks like 'LR too low' but the cause is the data."	c3::ft11::analysis
"What is PagedAdamW and why is it the QLoRA default optimizer?"	"PagedAdamW uses CUDA paging to transparently offload optimizer states to CPU memory, avoiding OOM spikes during training. It pairs with QLoRA's 4-bit quantized bases (which already push memory to the limit) by ensuring the optimizer states — which for AdamW are 2x the trainable params — don't blow the VRAM budget. Use optim='paged_adamw_8bit' or 'paged_adamw_32bit' in SFTConfig."	c3::ft11::analysis
"In the lab, you set load_best_model_at_end=True with metric_for_best_model='eval_loss'. Why, and what would go wrong without it?"	"So the model in memory at the end of training is the checkpoint with the LOWEST eval loss — the one you ship. Without it, trainer.train() leaves you with the LAST checkpoint, which may be past the eval-loss minimum (overfitting). You'd ship a memorizing model. Combined with save_strategy='steps', this guarantees you can always recover the best checkpoint."	c3::ft11::application
"Why does the module call TRL's design principle 'fine-tuning steers behavior'?"	"Every trainer in TRL is a steering tool: SFTTrainer steers format/instruction-following, DPOTrainer steers preference, GRPOTrainer steers reasoning toward verifiable rewards. None inject knowledge. The library's job is to make steering reliable, observable, and cheap — applying chat templates, completion masking, PEFT attachment, and packing so you spend time on data and hyperparameters, not glue code. This is the FT00 thesis realized in software."	c3::ft11::analysis