distilabel
The synthetic data pipeline framework. Magpie-style generation, Evol-Instruct evolution, judge-based filtering, and preference dataset construction — the end-to-end standard for building synthetic SFT and preference data.
distilabel is the standard for synthetic data construction. It is Argilla's pipeline framework for generating, evolving, filtering, and formatting synthetic SFT and preference datasets. Where TRL handles the training side, distilabel handles the data-construction side — the steering wheel factory.
Magpie-style generation removes the prompt-engineering bottleneck. Instead of hand-authoring thousands of seed instructions, Magpie uses the instruct-tuned model's own output distribution to generate diverse, self-prompted instructions at scale. distilabel integrates this pattern for high-volume SFT data.
Judge-based filtering is the quality gate. distilabel pipelines use an LLM-as-judge step to score generated responses on correctness, helpfulness, or preference, then filter by threshold. The judge is what separates a curated dataset from a noisy one — and per the course thesis, the dataset quality is the ceiling on the trained model.
Preference datasets (for DPO) are a first-class output. distilabel builds preference pairs (chosen/rejected) by generating multiple responses per prompt and ranking them via a judge or reward model. This is the DPO-data pipeline, integrated with TRL on the training side.