Module FTDD-03 — Unsloth

Unsloth

The single-GPU speed and memory optimizer: hand-written Triton kernels, manual autograd, and 4-bit optimizers deliver ~2x speed and ~60% less VRAM — the tool that made 7B QLoRA viable on a $1,500 RTX 4090.

minutes

artifacts

sub-sections

Unsloth is the consumer-GPU enabler. By rewriting the performance-critical forward/backward paths in hand-written Triton kernels, computing gradients manually, and storing optimizer state at 4-bit, it achieves ~2x training speed and ~60% memory reduction with ZERO algorithmic changes — same fine-tuning, faster and cheaper. Plus Dynamic 4.0 GGUF quants for intelligent per-layer export. CUDA-focused (Apple Silicon uses MLX). When to choose Unsloth vs Axolotl vs TRL.

Key Claims

Load-Bearing Claims

Unsloth's gains are engineering wins, not algorithmic changes. ~2x speed and ~60% less VRAM come from three implementation-level decisions: hand-written Triton kernels (forward/backward), manual autograd (bypass PyTorch's graph overhead), and 4-bit optimizers (8x optimizer-state reduction). The fine-tuning ALGORITHM is unchanged — a QLoRA through Unsloth produces the same adapter as one through TRL, just faster and cheaper.

Unsloth is the consumer-GPU enabler — it made 7B QLoRA viable on a $1,500 RTX 4090. Before Unsloth, 7B fine-tuning on consumer hardware needed a $2K–$5K GPU or aggressive compromises. After Unsloth, the RTX 4090's 24GB is enough. This democratization is pedagogical, not just economic: a student who can run a dozen experiments/day learns steering-vs-knowledge (FT00) by feel, not from a textbook.

Unsloth is CUDA-focused; Apple Silicon users use MLX (FT20) instead. The hand-written Triton kernels target NVIDIA GPUs. On M-series Macs, the optimizations don't apply. This is a deliberate scope boundary — NVIDIA dominates the ecosystem and the kernels are where the performance lives. Route by platform: NVIDIA CUDA → Unsloth; Apple Metal → MLX.

Dynamic 4.0 GGUF quants extend Unsloth's optimization from training into export. Intelligent per-layer quantization (sensitive layers get more bits, robust layers get fewer) beats uniform quants at the same file size. Matters for GGUF/Ollama export (FT19/FT20) and constrained edge devices — more quality per byte. Choose Unsloth for single-GPU/speed/GGUF; Axolotl for multi-GPU; TRL for full control.

After This Module

Explain how Unsloth achieves ~2x speed and ~60% memory reduction (Triton kernels, manual autograd, 4-bit optimizers) and defend why these are engineering wins, not algorithmic changes.

Defend Unsloth's role as the consumer-GPU enabler — the tool that made 7B QLoRA viable on a $1,500 RTX 4090 — and why this democratization is pedagogical, not just economic.

Identify Unsloth's CUDA focus and route Apple Silicon users to MLX (FT20).

Describe Dynamic 4.0 GGUF quants — intelligent per-layer quantization that beats uniform quants — and when it matters (GGUF/Ollama export, edge devices).

Choose between Unsloth (single-GPU, speed/memory, GGUF export), Axolotl (multi-GPU, configs), and TRL (full control) for a given fine-tuning job.

Artifacts

Teaching Document

~2,600 words; how Unsloth achieves the win (Triton kernels, manual autograd, 4-bit optimizers), the consumer-GPU enabler claim, the CUDA focus + Apple Silicon caveat, Dynamic 4.0 GGUF quants, and the Unsloth-vs-Axolotl-vs-TRL decision

READ

Diagrams

4 diagrams — how Unsloth achieves the win (three engineering decisions), the consumer-GPU enabler (before/after), Dynamic 4.0 vs uniform quantization (per-layer), Unsloth vs Axolotl vs TRL decision

READ

Slide Deck

10 slides — title, what Unsloth is/isn't, the three engineering decisions, the consumer-GPU enabler, CUDA focus, Dynamic 4.0, framework comparison, anti-patterns, what you can now do

READ

Teaching Script