MiniCPM Family (OpenBMB)
The course's on-ramp hero · Deep-Dive FTDD-01 · Course 3
45 minutes · Small, genuinely-open models you can run, audit, and fine-tune on one GPU
Why a 1B model from Tsinghua + ModelBest is the base every early module loads — and what the rest of the family adds.
Deep-Dives
The family — one modality axis
OpenBMB
Open Lab for Big Model Base — Tsinghua University + ModelBest. Apache-2.0.
The progression
- MiniCPM5-1B — ~1.08B text · the on-ramp
- MiniCPM3-4B — denser text · mid-size
- MiniCPM-V 4.6 — + SigLip-400M · vision
- MiniCPM-o 4.5 — omni · full-duplex
Each step adds a modality or capability the previous lacked. SigLip-400M is the component that turns a text base into a vision-language model.
MiniCPM5-1B — the on-ramp hero
Why 1B, not 7B? The FT00 thesis — fine-tuning steers behavior, not knowledge — is demonstrable on a 1B model in minutes. You need a model cheap enough to run the experiment ten times.
The model FT00's lab loads, FT08's QLoRA walks through, and every VRAM calc uses as its worked example.
The Ultra* datasets — the steering wheel
| Dataset | Purpose | Steering layer | Module |
|---|
| UltraChat | Multi-turn dialogue | SFT — instruction format | FT04, FT12 |
| UltraFeedback | Preference / feedback pairs | DPO — preference signal | FT05, FT13 |
| Ultra-FineWeb | Curated web pretraining | Layer 1 / CPT | FT06 |
arXiv:2310.01377 (Cui et al., 2023) — the open-preference-dataset citation. Open data means the course can point at a concrete preference pair and say "this is the signal DPO optimizes."
Open datasets = a steering wheel you can inspect. A closed dataset cannot match this as a teaching tool.
Why MiniCPM is the teaching vehicle
PROPERTY 1
Cheap iteration
1B fine-tunes in minutes on a consumer GPU. Run the SFT→DPO→quantize→deploy loop in an afternoon.
PROPERTY 2
Auditable provenance
Open weights + data + recipe. Point at every byte the model saw. The FT02 auditability predicate.
PROPERTY 3
Apache-2.0
No MAU / field-of-use clauses. Ship a product without consulting a lawyer about the model license.
Nature Communications paper (s41467-025-61040-5) makes the case formally — a small, open, documented model as a viable platform for the whole fine-tuning stack.
Fine-tuning: SWIFT vs LLaMA-Factory
SWIFT
ModelBest / OpenBMB — first-party
- Native support for ALL MiniCPM variants
- First-party chat templates & modality handlers
- Center of gravity = OpenBMB ecosystem
LLaMA-Factory
General-purpose
- Unified interface for dozens of families
- Add MiniCPM as a config change
- Vision/omni support lags SWIFT
Deciding factor: modality. Text models (5-1B, 3-4B) — toss-up. Vision/omni (V, o) — SWIFT, for first-party modality handling.
Anti-patterns
Treating MiniCPM5-1B as a production frontier model. It is a small, capable, open model — ideal for teaching and edge. It will lose head-to-head against 70B+. Match the base to the deployment (FT03).
"Ultra* datasets are clean because they're open." Open ≠ clean (FT02). Open means you can audit — not that it's done. Run the PII sweep, dedup, decontaminate (FT06) like any dataset.
Picking the framework without considering modality. Choosing LLaMA-Factory for an omni-modal fine-tune because "it's what we use" will cost you days of template-debugging.
What you can now do
- Place MiniCPM on the open spectrum and explain why it's the course's default base.
- Map the family (5-1B, 3-4B, V, o) to the modality gap each fills, and explain SigLip-400M's role.
- Trace the Ultra* datasets to the steering layers they enable and the arXiv provenance.
- Defend, with the Nature Comms paper, why an open-data family is the right teaching vehicle.
- Choose between SWIFT and LLaMA-Factory and defend the tradeoff.
The lab: load MiniCPM5-1B, apply a format-steering LoRA via SWIFT, measure the before/after behavior shift. The steering thesis, felt in under 30 minutes.
Next: FTDD-02 — OLMo 2/3 + Tülu 3 (Ai2)