# Deep-Dive FTDD-01 — MiniCPM Family (OpenBMB) **Course**: Course 3 — LLM Fine-Tuning Masterclass **Deep-Dive**: FTDD-01 **Duration**: 45 minutes **Level**: Senior Engineer and above **Prerequisites**: FT00 (Steering Stack), FT02 (Open Spectrum) > *The course's on-ramp hero. A family of small, genuinely-open models from OpenBMB that you can run, audit, and fine-tune on a single consumer GPU — the base every early module in this course loads by default.* --- ## Learning Objectives After this deep-dive, you will be able to: 1. Place the MiniCPM family on the open spectrum and explain why it is the course's default base (open-data + Apache-2.0 + consumer-hardware-sized). 2. Map the family — MiniCPM5-1B, MiniCPM3-4B, MiniCPM-V 4.6, MiniCPM-o 4.5 — to the modality gap each fills, and explain the SigLip-400M vision encoder's role. 3. Trace the Ultra\* datasets (UltraChat, UltraFeedback, Ultra-FineWeb) to the steering techniques they enable (SFT, DPO, continued pretraining) and the arXiv provenance (2310.01377). 4. Defend, with the Nature Communications paper, why an open-data family is the right teaching vehicle: cheap iteration, auditable provenance, no license friction. 5. Choose between SWIFT and LLaMA-Factory for fine-tuning a MiniCPM base and explain the tradeoff. --- ## The Subject **OpenBMB** — the "Open Lab for Big Model Base" — is a collaboration between Tsinghua University and the company ModelBest. Its defining product is the **MiniCPM** family: a roster of small dense and multimodal models released under Apache-2.0 with open training data. If this course has a default base — the model every early lab loads, every fine-tuning example points at, every VRAM calculation uses as its worked example — it is MiniCPM. | Metric | Value | | --- | --- | | Lab | OpenBMB — Tsinghua University + ModelBest | | License | Apache-2.0 | | Headline model | MiniCPM5-1B (~1.08B params, May 2026, Intelligence Index 17.9) | | Mid-size text | MiniCPM3-4B | | Vision-language | MiniCPM-V 4.6 (SigLip-400M encoder) | | Omni-modal | MiniCPM-o 4.5 (full-duplex streaming) | | Open datasets | UltraChat · UltraFeedback · Ultra-FineWeb | | Canonical paper | Nature Communications, s41467-025-61040-5 | | Community walkthrough | Sam Witteveen, youtu.be/ox1mW2N9Z_Y | MiniCPM matters for three reasons, and they are the three reasons a course on fine-tuning cares about a base model at all. First, it is **small enough to iterate on cheaply**. A 1B model loads on a laptop GPU, fine-tunes in minutes, and lets a student run a dozen experiments in an afternoon. That is what teaching fine-tuning requires — not a frontier model that costs $50 per run. Second, it is **genuinely open**. Apache-2.0 weights, named training data, and a published recipe. This is not Llama's "weights with an aggregate data description." You can point at every byte the model saw — which is the auditability predicate this course keeps returning to (FT02). Third, it is **auditable end-to-end**. The Ultra\* datasets are public. When the course says "fine-tune on UltraFeedback," you can go read the dataset, inspect its distribution, and understand what preference signal you are actually injecting. With a weights-only base, you are steering a model whose pretraining distribution is opaque; with MiniCPM, the whole stack is inspectable. --- ## The Family MiniCPM is not one model — it is a family that spans the modality axis. Each member exists because the previous one could not do something. ### MiniCPM5-1B — the on-ramp The course's default. A dense ~1.08B-parameter text model, released in May 2026, scoring an Intelligence Index of 17.9 — strong for a model that fits comfortably in 2GB of VRAM. This is the model FT00's "Load, Steer, Swap" lab loads, the one FT08's QLoRA walkthrough fine-tunes, and the one every VRAM calculation in FT01 uses as its worked example. It is not the smartest model in the family. It is the one you can afford to break, reset, and break again — which is what learning requires. Why 1B and not 7B? Because the course's thesis (FT00) is that fine-tuning steers behavior rather than injecting knowledge, and that thesis is *demonstrable* on a 1B model in minutes. You do not need a frontier model to learn that SFT changes format, that DPO shifts preference, or that abliteration deletes a direction. You need a model cheap enough to run the experiment ten times. ### MiniCPM3-4B — the mid-size text base When 1B is too weak and 7B is too expensive, MiniCPM3-4B is the middle ground. A denser text base that retains the family's open-data posture and Apache-2.0 license. This is the base you reach for when the lab needs genuine reasoning capability but you still want to fit on a single 12GB GPU. It is also the model where the gap between "steering a weak model" and "steering a capable model" becomes visible — the same LoRA that barely moves a 1B base produces clear, evaluable behavior shifts on 4B. ### MiniCPM-V 4.6 — vision-language The first multimodal member. MiniCPM-V 4.6 pairs a MiniCPM text backbone with the **SigLip-400M** vision encoder — a 400-million-parameter image encoder that produces the visual embeddings the language model conditions on. The architecture is the standard vision-language pattern: images pass through SigLip to become token-like embeddings, which are concatenated with the text tokens and processed by the shared transformer. This is the base the course reaches for whenever a lab needs image input — captioning, visual question answering, document understanding. SigLip matters specifically because it is itself an open, well-understood encoder. The vision-language stack is auditable down to the image encoder, not just the language backbone. ### MiniCPM-o 4.5 — omni-modal, full-duplex The most ambitious member. MiniCPM-o 4.5 is an omni-modal model — it accepts and produces text, image, and audio — and it does so in a **full-duplex** streaming mode, meaning it can listen and respond simultaneously rather than in alternating turns. This is the architecture pattern behind real-time voice assistants. For this course, MiniCPM-o is the "where this is going" reference: the same fine-tuning techniques you learn on MiniCPM5-1B scale up to omni-modal steers, but the omni-modal case is where the VRAM math (FT01) and the data pipeline (Pillar 1) become genuinely hard. --- ## The Ultra\* Datasets The reason MiniCPM is the course's data reference, not just its model reference, is the Ultra\* family of open datasets. These are the corpora that appear again and again in the data modules (Pillar 1) and the alignment modules (Pillar 3). | Dataset | Purpose | Steering layer | Course module | | --- | --- | --- | --- | | **UltraChat** | Large-scale multi-turn dialogue | Layer 3 (SFT) — instruction format | FT04, FT12 | | **UltraFeedback** | Preference / feedback pairs | Layer 3 (DPO) — preference signal | FT05, FT13 | | **Ultra-FineWeb** | Curated web pretraining mix | Layer 1 (base) / continued pretraining | FT06 | The canonical citation for the Ultra\* line is **arXiv:2310.01377** (Cui et al., 2023), which introduced UltraFeedback and established the open-preference-dataset pattern that the DPO modules lean on. The point is not that these are the best datasets — it is that they are *open, documented, and reproducible*, which means the course can point at a concrete preference pair and say "this is the signal DPO is optimizing." With a closed dataset, that pointer is impossible. This is why the data modules return to Ultra\* repeatedly. When FT13 teaches DPO, it teaches it on UltraFeedback because you can go read the dataset. When FT05 teaches synthetic data generation, it uses the Ultra\* pipeline as the reference because the pipeline is published. The datasets are the steering wheel (FT00) — and a steering wheel you can inspect is a teaching tool a closed dataset cannot match. --- ## Why MiniCPM Is the Teaching Vehicle The Nature Communications paper (s41467-025-61040-5) makes the case formally. MiniCPM's contribution is not a single benchmark win — it is the demonstration that a small, open, well-documented model can be a viable research and teaching platform for the entire fine-tuning stack. The paper walks through pretraining data curation, mid-training, and post-training, with each stage's data and recipe disclosed. Three properties make MiniCPM the ideal teaching vehicle, and they map directly onto the course's needs: 1. **Cheap iteration.** A 1B model fine-tunes in minutes on a consumer GPU. A student can run the full SFT → DPO → quantize → deploy loop in a single afternoon. This is non-negotiable for a course — you cannot teach fine-tuning on a model that takes six hours per run. 2. **Auditable provenance.** Open weights, open data, open recipe. Every claim about what the model saw can be checked against the published corpus. This is the FT02 auditability predicate, made concrete on the base the course actually uses. 3. **Apache-2.0, no friction.** No field-of-use restrictions, no MAU clauses, no named-product carve-outs (contrast Llama's Community License). A student can build on MiniCPM, ship a product, and never consult a lawyer about the model license. For a course that wants students to *ship* things, this matters. Sam Witteveen's walkthrough (youtu.be/ox1mW2N9Z_Y) is the recommended community companion to this deep-dive — it covers the family overview and the fine-tuning workflow in a format that complements the formal paper. --- ## Fine-Tuning MiniCPM: SWIFT vs LLaMA-Factory Two frameworks dominate MiniCPM fine-tuning, and the course references both. The choice is a real engineering decision, not a preference. **SWIFT** (Scalable lightWeight Infrastructure for Fine-Tuning) is ModelBest/OpenBMB's own framework — the first-party option. Its advantage is native, first-day support for every MiniCPM variant, including the vision and omni-modal models. If you are fine-tuning MiniCPM-V 4.6 or MiniCPM-o 4.5, SWIFT is the path of least resistance because the chat templates, modality handlers, and example scripts are maintained by the people who built the model. Its disadvantage is that it is MiniCPM-flavored — it supports other models, but its center of gravity is the OpenBMB ecosystem. **LLaMA-Factory** is the general-purpose framework — a unified interface for fine-tuning dozens of model families, MiniCPM included. Its advantage is breadth: if your team already uses LLaMA-Factory for Llama, Qwen, and Mistral, adding MiniCPM is a config change, not a new tool. Its disadvantage is that support for the newest MiniCPM variants (especially the omni-modal models) can lag SWIFT by weeks to months, and the example scripts are community-maintained rather than first-party. The decision rule: if you are working with MiniCPM-V or MiniCPM-o, or you want first-party assurance that the chat template and modality handling are correct, use SWIFT. If you are fine-tuning MiniCPM5-1B or MiniCPM3-4B as part of a multi-model pipeline, use LLaMA-Factory. Both produce equivalent results on the text models; the difference is ecosystem fit. Neither is the framework this course uses for its core training-loop deep-dive (FTDD-04 covers TRL; FTDD-05 covers Axolotl; FTDD-03 covers Unsloth). SWIFT and LLaMA-Factory are the MiniCPM-specific entry points. They are the right choice when the base is MiniCPM and you want the path of least resistance. --- ## Anti-Patterns ### Treating MiniCPM5-1B as a production frontier model It is not. It is a small, capable, open model — ideal for teaching, prototyping, and edge deployment. It will lose head-to-head capability comparisons against 70B+ models. The course uses it because it is cheap to iterate on, not because it is the strongest model available. Match the base to the deployment (FT03). ### Assuming the Ultra\* datasets are clean because they are open Open ≠ clean (FT02 anti-pattern). UltraChat, UltraFeedback, and Ultra-FineWeb are open and documented, which means you *can* audit them — not that the audit has already been done. Run the same PII sweep, deduplication, and decontamination (FT06) you would run on any dataset. Openness gives you the ability to vet; it does not do the vetting. ### Picking SWIFT or LLaMA-Factory without considering the modality The text models fine-tune equivalently in both. The vision and omni-modal models do not — SWIFT's first-party modality support is the deciding factor for MiniCPM-V and MiniCPM-o. Choosing LLaMA-Factory for an omni-modal fine-tune because "it's what we already use" is a reasonable instinct that will cost you days of template-debugging. --- ## Key Terms | Term | Definition | | --- | --- | | **OpenBMB** | "Open Lab for Big Model Base" — Tsinghua University + ModelBest; the lab behind MiniCPM | | **MiniCPM5-1B** | The ~1.08B dense text model; the course's default on-ramp base (May 2026, Intelligence Index 17.9) | | **MiniCPM3-4B** | The mid-size dense text base; the single-GPU reasoning-capable option | | **MiniCPM-V 4.6** | The vision-language variant; pairs MiniCPM with the SigLip-400M vision encoder | | **MiniCPM-o 4.5** | The omni-modal, full-duplex variant (text + image + audio) | | **SigLip-400M** | The 400M-parameter image encoder used in MiniCPM-V; open and auditable | | **Ultra\* datasets** | UltraChat (SFT), UltraFeedback (DPO, arXiv:2310.01377), Ultra-FineWeb (pretraining) — the open corpora the data modules return to | | **SWIFT** | ModelBest/OpenBMB's first-party fine-tuning framework; best for MiniCPM-V/o | | **LLaMA-Factory** | The general-purpose multi-model fine-tuning framework; best for text MiniCPM in a multi-model pipeline | --- ## Lab Exercise See `07-lab-spec.md`. The lab loads MiniCPM5-1B, applies a format-steering LoRA via SWIFT, and measures the before/after behavior shift — the steering thesis (FT00) felt on the course's default base, in under 30 minutes. --- ## References 1. **OpenBMB MiniCPM** — GitHub: `OpenBMB/MiniCPM`; the family repository and primary source. 2. **Nature Communications paper** — s41467-025-61040-5; the formal case for MiniCPM as an open research/teaching platform. 3. **Cui et al. (2023)** — *UltraFeedback* (arXiv:2310.01377); the open-preference-dataset citation. 4. **Sam Witteveen walkthrough** — youtu.be/ox1mW2N9Z_Y; the community companion video. 5. **Course 3 FT00** — the Steering Stack; the thesis MiniCPM5-1B demonstrates. 6. **Course 3 FT02** — the Open Spectrum; the tier classification MiniCPM occupies (open-recipe). 7. **SWIFT** — `modelscope/swift`; the first-party MiniCPM fine-tuning framework. 8. **LLaMA-Factory** — `hiyouga/LLaMA-Factory`; the general-purpose alternative.