# Deep-Dive FTDD-02 — OLMo 2/3 + Tülu 3 (Ai2) **Course**: Course 3 — LLM Fine-Tuning Masterclass **Deep-Dive**: FTDD-02 **Duration**: 45 minutes **Level**: Senior Engineer and above **Prerequisites**: FT00 (Steering Stack), FT02 (Open Spectrum), FTDD-01 (MiniCPM) > *The fully-open comparison. Allen Institute's OLMo and Tülu are what "open" means when a research lab commits to releasing everything — weights, data, code, and the complete post-training recipe. The counterpoint to MiniCPM's product-and-edge orientation.* --- ## Learning Objectives After this deep-dive, you will be able to: 1. Distinguish Ai2's OLMo (the open base) from Tülu 3 (the open post-training recipe) and explain why both are needed for full reproducibility. 2. Map the OLMo 2 → OLMo 3 progression and the base/instruct/think variants, citing arXiv:2501.00656 for OLMo 2 and arXiv:2411.15124 for Tülu 3. 3. Trace the Tülu 3 post-training pipeline (SFT → DPO → RLVR) and place each stage on the FT00 steering stack. 4. Defend why fully-open (weights + data + code + recipe) matters for reproducibility — and why "weights-only open" is not enough. 5. Contrast the OpenBMB/MiniCPM and Ai2/OLMo-Tülu philosophies: both open-data, but MiniCPM is product/edge-oriented while OLMo/Tülu is research-oriented. --- ## The Subject The **Allen Institute for AI (Ai2)** is the canonical fully-open lab. Where OpenBMB (FTDD-01) builds small models for product and edge deployment, Ai2 builds models whose *primary contribution is the openness itself* — the demonstration that a frontier-scale model can be released with every component a researcher needs to reproduce, audit, and extend it. Two releases carry this banner: **OLMo** (the open base) and **Tülu 3** (the open post-training recipe). | Metric | Value | | --- | --- | | Lab | Allen Institute for AI (Ai2) | | License | Apache-2.0 (weights, data, code) | | Open base | OLMo 2 (arXiv:2501.00656) · OLMo 3 (Nov 2025, 7B + 32B) | | Open post-train | Tülu 3 (arXiv:2411.15124), up to 405B | | Pipeline repo | `allenai/open-instruct` | | OLMo 2 nickname | "2 OLMo 2 Furious" | | OLMo 3 variants | base · instruct · think | | Tülu 3 stages | SFT → DPO → RLVR (verifiable rewards) | Ai2 matters for this course because it is the strict end of the open spectrum (FT02). Where MiniCPM is open-data and open-recipe, OLMo and Tülu push further: they release intermediate checkpoints, the eval harness, and the *exact data mixture*. A skilled person can rebuild the model from scratch; a security team can prove no training-time exfiltration occurred. This is the tier that IL5/IL6 and air-gapped environments increasingly require (FT22). --- ## OLMo — The Open Base ### OLMo 2 — "2 OLMo 2 Furious" OLMo 2 (arXiv:2501.00656) is the open base that established the modern fully-open standard. The nickname — "2 OLMo 2 Furious" — is the community's nod to the paper's density and the team's pace. What OLMo 2 contributes is not a benchmark win; it is the release of every component the FT02 open-recipe tier promises: - **Weights** (multiple sizes, base and instruct). - **Training data** (the actual corpus, not an aggregate description). - **Training code** (the exact scripts and configs). - **Intermediate checkpoints** (so you can study training dynamics). - **Evaluation suite** (so you can reproduce the benchmarks). The OLMo 2 paper is the canonical "fully open" citation this course returns to in FT02. When the course says "open-recipe means you can rebuild and prove it," OLMo 2 is the example. Every other open-recipe release (SmolLM3, MiniCPM) is measured against the OLMo standard. ### OLMo 3 — the three-variant release OLMo 3, released November 2025, comes in two sizes (7B and 32B) and — critically — three variants: - **base** — the pretrained weights, post-training-free. What you would fine-tune from. - **instruct** — instruction-tuned (SFT + preference). The general-purpose chat model. - **think** — a reasoning-tuned variant (the GRPO/RL-on-verifiable-rewards lineage, FT14). The three-variant release is significant because it makes the steering stack (FT00) concrete. The *base* is Layer 1. The *instruct* is base + Layer 3 (SFT + DPO). The *think* is instruct + Layer 3 (GRPO). A student can load all three from the same family and *see* what each steering stage changed — a pedagogical gift that a single-variant release cannot provide. This is why OLMo 3 is the model to reach for when teaching the difference between SFT, DPO, and GRPO: the same base, three steers, directly comparable. --- ## Tülu 3 — The Open Post-Training Recipe If OLMo is the open base, **Tülu 3** (arXiv:2411.15124) is the open post-training stack. Tülu 3's contribution is the full, reproducible recipe for turning a base model into an aligned, capable instruct model — released as data, code, and configs, at scales up to 405B parameters. ### The three-stage pipeline Tülu 3 post-training is a three-stage pipeline, and each stage maps cleanly onto the FT00 steering stack: ``` BASE MODEL (Layer 1) │ ▼ [1] SFT ────────────────── Layer 3 (format, instruction-following) │ (Supervised Fine-Tuning on curated instruction data) ▼ [2] DPO ────────────────── Layer 3 (preference alignment) │ (Direct Preference Optimization on preference pairs) ▼ [3] RLVR ────────────────── Layer 3 (reasoning, verifiable rewards) │ (Reinforcement Learning on Verifiable Rewards — math, code, etc.) ▼ TÜLU 3 INSTRUCT MODEL ``` **Stage 1 — SFT.** Supervised Fine-Tuning on a curated instruction dataset. This is the format-and-injection-following steer (FT12). Tülu 3 releases the SFT data mixture, so you can see exactly what instructions the model was steered toward. **Stage 2 — DPO.** Direct Preference Optimization on preference pairs (FT13). This is the preference-alignment steer — "this response is better than that one." Tülu 3 releases the preference dataset and the DPO hyperparameters. **Stage 3 — RLVR.** Reinforcement Learning on Verifiable Rewards (FT14). This is the reasoning steer — the model generates solutions to problems with verifiable answers (math, code, logic), and correct solutions are rewarded. RLVR is the technique behind the "reasoning model" wave, and Tülu 3 is the open implementation of it. The load-bearing property: every stage's data, code, and config is released. You can rebuild Tülu 3 from an OLMo base. You can ablate a single stage (what if you skip RLVR?). You can swap in your own data at any stage. This is what "open post-training recipe" means in practice — not a paper describing the recipe, but the recipe itself, executable. ### The open-instruct repo The practical handle for all of this is `allenai/open-instruct` — the repository that contains the Tülu 3 training code, data preparation scripts, and configuration files. If you want to reproduce Tülu 3, or build a Tülu-style pipeline on your own base, open-instruct is where you start. It is the post-training analogue of OLMo's pretraining code: the executable recipe, not the description. --- ## Why Fully-Open Matters for Reproducibility The argument for fully-open (weights + data + code + recipe) is not ideological. It is the FT02 argument, sharpened: reproducibility is the property that lets you *prove* things about a model, and reproducibility requires every component. Three things only fully-open releases give you: 1. **Reproducibility in the strict sense.** You can rebuild the model from a pinned commit and get the same weights (modulo nondeterminism, which is itself documented). This is the no-silent-drift argument (FT02) — you can prove the model you validated is the model you are running. 2. **Ablation.** You can remove or change a single component (a data source, a post-training stage) and measure the effect. This is how research advances — and it is impossible with a weights-only release, where the data and recipe are opaque. 3. **Trust in the supply chain.** You can audit the training pipeline end-to-end and rule out hidden training-time exfiltration. This is the property air-gapped and classified environments require (FT22). The Tülu 3 release is the clearest demonstration of (2) — ablation — in the open ecosystem. Because the full post-training recipe is released, a researcher can ask "what does RLVR actually contribute?" by training Tülu 3 with and without the RLVR stage, on the same base, with the same SFT and DPO. With a weights-only release, that question is unanswerable. --- ## OpenBMB vs Ai2 — Two Open Philosophies Both OpenBMB (MiniCPM) and Ai2 (OLMo/Tülu) are open-data and open-recipe. Both ship Apache-2.0. But they have different centers of gravity, and the difference matters when you choose a base. | Property | OpenBMB / MiniCPM | Ai2 / OLMo + Tülu | | --- | --- | --- | | **Orientation** | Product / edge | Research | | **Model sizes** | Small (1B–4B) + multimodal | Mid-to-large (7B–405B), text-focused | | **Post-training recipe** | Disclosed, productized | Fully released, ablation-ready (Tülu 3) | | **Multi-variant releases** | Modality axis (text/vision/omni) | Steer axis (base/instruct/think) | | **Default use case** | Ship a small model on edge hardware | Reproduce, audit, research the stack | | **Course role** | The on-ramp base (FT00 lab) | The fully-open reference (FT02, FT22) | The summary: **MiniCPM is what you ship; OLMo/Tülu is what you study.** If your goal is a small model on a phone or an edge device, MiniCPM's modality axis and small sizes win. If your goal is to understand, audit, or reproduce the full training-and-post-training stack — especially for a regulated deployment that demands proof — OLMo and Tülu are the reference. This is why the course uses both. FT00's lab loads MiniCPM5-1B because it is cheap to iterate on. FT02's audit references OLMo 2 because it is the fully-open standard. And when the course needs to teach *what post-training actually does* (FT12 SFT, FT13 DPO, FT14 GRPO), OLMo 3's base/instruct/think variants and Tülu 3's released pipeline are the clearest teaching tools — the same base, each steer visible. --- ## Anti-Patterns ### Treating "fully-open" as "production-ready" OLMo and Tülu are research artifacts first. They are fully open, auditable, and reproducible — but they are not always the most capable models at their size point. A closed or weights-only model may outperform them on benchmarks. Choose them for openness, auditability, and reproducibility, not because they top every leaderboard. Capability is FT03's concern. ### Assuming OLMo 3 "think" = GPT-class reasoning The think variant is a reasoning-tuned model (RLVR), but it is a 7B/32B model, not a frontier model. It demonstrates the RLVR technique (FT14) openly — which is its research contribution — but it will not match a closed frontier reasoning model on the hardest tasks. Use it to *understand* reasoning fine-tuning, not to deploy a frontier reasoner. ### Skipping the open-instruct repo Tülu 3's value is in the recipe, and the recipe lives in `allenai/open-instruct`. Reading the paper without reading the repo gives you the description; reading the repo gives you the executable pipeline. If you want to reproduce or adapt Tülu 3, the repo is the primary source. --- ## Key Terms | Term | Definition | | --- | --- | | **Ai2** | Allen Institute for AI; the canonical fully-open lab | | **OLMo 2** | The open base that established the modern fully-open standard (arXiv:2501.00656); nicknamed "2 OLMo 2 Furious" | | **OLMo 3** | The three-variant release (Nov 2025, 7B + 32B): base, instruct, think | | **Tülu 3** | The open post-training recipe (arXiv:2411.15124); SFT → DPO → RLVR, up to 405B | | **RLVR** | Reinforcement Learning on Verifiable Rewards — the reasoning steer (Tülu 3 stage 3, FT14) | | **open-instruct** | `allenai/open-instruct` — the repo containing the Tülu 3 training code, data prep, and configs | | **Fully-open** | Weights + data + code + recipe + checkpoints + eval — the strictest tier (FT02) | | **base / instruct / think** | OLMo 3's three variants: pretrained, instruction-tuned, reasoning-tuned | --- ## Lab Exercise See `07-lab-spec.md`. The lab compares OLMo 3's base, instruct, and think variants on a reasoning task — making the three steering stages (none, SFT+DPO, +RLVR) directly visible on the same base, no fine-tuning required. --- ## References 1. **OLMo 2 paper** — arXiv:2501.00656; the fully-open base standard. 2. **Tülu 3 paper** — arXiv:2411.15124; the open post-training recipe. 3. **open-instruct repo** — `allenai/open-instruct`; the executable Tülu 3 pipeline. 4. **OLMo project page** — `allenai.org/olmo`; the family overview. 5. **Course 3 FT02** — the Open Spectrum; OLMo/Tülu as the open-recipe reference. 6. **Course 3 FT14** — GRPO and verifiable rewards; RLVR is the open implementation. 7. **Course 3 FTDD-01** — MiniCPM; the product/edge counterpoint to OLMo/Tülu's research orientation.