# Lab Specification — Deep-Dive DD-01: Re-Score Pi and Trace the Minimal Loop

**Course**: Master Course — Harness Engineering
**Deep-Dive**: DD-01 — Pi: The Minimal Baseline
**Duration**: 30–45 minutes
**Environment**: Python 3.10+. No GPU, no external network, no model calls. The lab is a deterministic, type-hinted simulation of Pi's execution trace — you re-score Pi against the 12-module rubric with the full Modules 1–12 lens, trace one tool call through the 7-step dispatch, and justify the three changes you would make if you forked Pi toward "minimal production."

---

## Learning objectives

By the end of this lab you will have:

1. **Re-scored Pi against the 12-module rubric** with the full Modules 1–12 lens — producing a scored card with evidence and a total that matches (or defends a deviation from) the deep-dive's 25/60.
2. **Traced one tool call** (`read_file({path: "auth.ts"})`) through Pi's 7-step dispatch, confirming each step and identifying the steps a hardened harness would add.
3. **Modeled the three "minimal production" changes** (token-budget stop, per-turn observability payload, basic compaction threshold) and confirmed each closes a scored gap WITHOUT compromising Pi's co-evolution or thinness — the defining property of a correct fork.
4. **Justified the fork boundary** — articulating why these three changes preserve Pi's value while a single-component security addition (e.g., "just add a sandbox") does not.

This lab is the empirical anchor for the deep-dive's central claim: Pi's low score is the expected result for a deliberate thin harness, and the correct fork adds infrastructure, not cleverness.

---

## Phase 0 — Set up (5 min)

Create the lab file. No dependencies beyond the standard library.

```bash
mkdir -p dd01-lab && cd dd01-lab
cat > rescore_pi.py << 'PYEOF'
"""DD-01 Lab: Re-score Pi against the 12-module rubric, trace a tool call,
and model the three 'minimal production' changes.

Pi is the reference thin harness: 4 tools, <1k prompt, ~1,200 LOC, trust-the-model.
This lab does NOT call a model — it simulates the dispatch and the scoring.
"""
from __future__ import annotations
from dataclasses import dataclass, field
from enum import Enum
from typing import Optional


class Module(str, Enum):
    M1_LOOP = "M1 Execution Loop"
    M2_TOOLS = "M2 Tool Design"
    M3_CONTEXT = "M3 Context Mgmt"
    M4_MEMORY = "M4 Memory"
    M5_SANDBOX = "M5 Sandboxing"
    M6_PERMISSION = "M6 Permission"
    M7_ERRORS = "M7 Error Handling"
    M8_STATE = "M8 State"
    M9_VERIFICATION = "M9 Verification"
    M10_SUBAGENTS = "M10 Subagents"
    M11_OBSERVABILITY = "M11 Observability"
    M12_PROMPT = "M12 Prompt Assembly"


@dataclass
class ScoreEntry:
    module: Module
    score: int           # 1-5, or 0 for n/a (subagents absent)
    pattern: str         # the design pattern Pi uses
    tradeoff: str        # the tradeoff accepted
    evidence: str        # file/code evidence
    rationale: str       # WHY this score for THIS use case


# The reference scoring from the teaching document.
REFERENCE_SCORES: dict[Module, ScoreEntry] = {
    Module.M1_LOOP: ScoreEntry(Module.M1_LOOP, 4, "ReAct dumb-loop",
        "error compounding; no lookahead", "the ~80-line while loop",
        "Correct for co-evolution; loses 1 point for error-compounding risk."),
    Module.M2_TOOLS: ScoreEntry(Module.M2_TOOLS, 5, "4-tool schema-first",
        "less extensibility", "4 tool defs", "Minimum capable set = lowest-noise set (Vercel finding)."),
    Module.M3_CONTEXT: ScoreEntry(Module.M3_CONTEXT, 2, "none",
        "context rot on long sessions", "absence of compaction",
        "Relies on minimal tool output; rots on long bash sessions."),
    Module.M4_MEMORY: ScoreEntry(Module.M4_MEMORY, 1, "ephemeral only",
        "no multi-session", "no persistent store", "In-context only; cannot resume across sessions."),
    Module.M5_SANDBOX: ScoreEntry(Module.M5_SANDBOX, 1, "none (OS process only)",
        "blast radius = host", "no container/VM", "Correct for trusted single-user; wrong for untrusted input."),
    Module.M6_PERMISSION: ScoreEntry(Module.M6_PERMISSION, 2, "trust-the-model",
        "no per-action approval", "absence of gates", "No gates; the model acts freely."),
    Module.M7_ERRORS: ScoreEntry(Module.M7_ERRORS, 2, "minimal try/catch",
        "no taxonomy; retries possible", "basic try/catch", "Lets model self-correct via tool-result errors."),
    Module.M8_STATE: ScoreEntry(Module.M8_STATE, 1, "none (stateless)",
        "every interruption = restart", "no checkpoint", "Cannot resume after interrupt."),
    Module.M9_VERIFICATION: ScoreEntry(Module.M9_VERIFICATION, 1, "none (ship and hope)",
        "no quality gate", "absence", "No output verification."),
    Module.M10_SUBAGENTS: ScoreEntry(Module.M10_SUBAGENTS, 0, "n/a (single-agent by design)",
        "—", "absence", "Scored 0/5 where absent."),
    Module.M11_OBSERVABILITY: ScoreEntry(Module.M11_OBSERVABILITY, 2, "console.log",
        "un-debuggable on long sessions", "minimal logging", "Below the structured-logging floor."),
    Module.M12_PROMPT: ScoreEntry(Module.M12_PROMPT, 5, "ultra-thin (<1k tokens)",
        "less explicit behavior control", "the system prompt", "Deliberate delegation; co-evolves."),
}
PYEOF
echo "Phase 0 scaffold written."
```

The reference scores encode the deep-dive's analysis. Your job is to reproduce them, defend any deviation, then model the fork.

---

## Phase 1 — Re-score Pi and defend the total (10 min)

Implement the scoring summary and the total computation.

```python
def render_scorecard(scores: dict[Module, ScoreEntry]) -> int:
    """Render the 12-module scorecard and return the total /60."""
    print("=" * 88)
    print("DD-01 PI — 12-MODULE RUBRIC SCORECARD")
    print("=" * 88)
    print(f"{'MODULE':<26}{'SCORE':<8}{'PATTERN':<28}{'TRADEOFF'}")
    print("-" * 88)
    total = 0
    for entry in scores.values():
        s = f"{entry.score}/5" if entry.score > 0 else "n/a"
        print(f"{entry.module.value:<26}{s:<8}{entry.pattern:<28}{entry.tradeoff}")
        total += entry.score
    print("-" * 88)
    print(f"{'TOTAL':<26}{total}/60")
    print("=" * 88)
    print("\nKey principle: the score reflects what Pi IS NOT, not a failure of what Pi IS.")
    print("Pi is deliberately thin; the rubric scores production-readiness across all dimensions.\n")
    return total


if __name__ == "__main__":
    total = render_scorecard(REFERENCE_SCORES)
    assert total == 25, f"Expected 25/60, got {total}/60"
    print(f"PASS: total is {total}/60 — matches the deep-dive reference.")
```

**Run it.** The expected total is **25/60**. If your total differs, identify which entry you weighted differently and write a one-sentence defense. The defense must cite the use case (trusted single-user personal assistant) and the deliberate-thinness principle.

**Record:** the total and any deviations with their defenses.

---

## Phase 2 — Trace a tool call through the 7-step dispatch (10 min)

Implement the dispatch trace for `read_file({path: "auth.ts"})`.

```python
@dataclass
class DispatchStep:
    step: int
    name: str
    action: str
    pi_implements: bool     # True = Pi does this; False = a hardened harness would add it
    note: str = ""


def trace_read_file_dispatch() -> list[DispatchStep]:
    """Trace read_file({path: 'auth.ts'}) through Pi's dispatch.

    Returns the 7 steps Pi implements, plus annotations for the steps
    a hardened harness would add (the gaps).
    """
    return [
        DispatchStep(1, "model emits tool_use",
            "model emits tool_use: read_file({path: 'auth.ts'})", True),
        DispatchStep(2, "validate name",
            "harness validates 'read_file' against the 4-tool registry", True),
        DispatchStep(2.5, "[HARDENED ONLY] permission gate",
            "a hardened harness would ask: is read_file permitted for this path principal?",
            False, "Pi omits — trust-the-model (M6: 2/5)"),
        DispatchStep(3, "schema-validate",
            "harness schema-validates {path: string}", True),
        DispatchStep(3.5, "[HARDENED ONLY] scope check",
            "a hardened harness would check: is 'auth.ts' inside the allowed scope?",
            False, "Pi omits — no filesystem scoping (M5: 1/5)"),
        DispatchStep(4, "execute",
            "harness executes readFileSync('auth.ts')", True),
        DispatchStep(5, "return content",
            "returns file content", True),
        DispatchStep(5.5, "[HARDENED ONLY] taint tag / sanitize",
            "a hardened harness would tag the content as untrusted or sanitize it",
            False, "Pi omits — no untrusted-content tagging (ASI01 surface)"),
        DispatchStep(6, "append to history",
            "appends result as {role: 'tool', content: ...}", True),
        DispatchStep(7, "next model call",
            "next model call sees the appended result", True),
    ]


def render_trace(steps: list[DispatchStep]) -> None:
    print("=" * 88)
    print("TOOL DISPATCH TRACE — read_file({path: 'auth.ts'})")
    print("=" * 88)
    for s in steps:
        marker = "[PI]      " if s.pi_implements else "[GAP]     "
        print(f"  Step {s.step:<4} {marker}{s.name}")
        print(f"           {s.action}")
        if s.note:
            print(f"           -> {s.note}")
    gaps = [s for s in steps if not s.pi_implements]
    print("-" * 88)
    print(f"Pi implements {sum(1 for s in steps if s.pi_implements)} steps; "
          f"omits {len(gaps)} steps a hardened harness would add.")
    print("The omitted steps are the M5/M6/M2.4 gaps — the Course 2B attack surface.")
    print("=" * 88)


# Add to __main__:
#   steps = trace_read_file_dispatch()
#   render_trace(steps)
```

**Run it.** The trace shows 7 implemented steps and 3 gap annotations (permission gate, scope check, taint tag). The 3 gaps are the exact surface Course 2B's SDD-B01 offensive expansion lands on: no permission gate (ASI03/ASI10 surface), no scope check (ASI05/M5 gap), no taint tag (ASI01 indirect-injection surface).

**Record:** confirm the 3 gap annotations and connect each to the module/ASI row it corresponds to.

---

## Phase 3 — Model the three "minimal production" changes (15 min)

Now the fork. Implement each of the three proposed changes as a function that confirms it closes a scored gap WITHOUT compromising co-evolution (adds infrastructure, not model-specific cleverness) or thinness (does not change the threat model).

```python
@dataclass
class ForkChange:
    name: str
    closes_module: Module
    score_delta: int        # how much the score improves (e.g., +3 on M1 means 4->... capped at 5)
    adds_cleverness: bool   # False = preserves the future-proof test
    changes_threat_model: bool  # False = preserves the thinness value


def model_minimal_production_fork() -> list[ForkChange]:
    """The three changes that move Pi from 'minimal baseline' to 'minimal production'.

    Each must: (1) close a scored gap; (2) add infrastructure, NOT model-specific
    cleverness (preserves co-evolution); (3) NOT change the threat model (preserves thinness).
    """
    return [
        ForkChange(
            name="token-budget stop condition",
            closes_module=Module.M1_LOOP,
            score_delta=1,           # M1: 4/5 -> 5/5
            adds_cleverness=False,   # it's a spending cap, not lookahead logic
            changes_threat_model=False,
        ),
        ForkChange(
            name="8-field per-turn observability payload",
            closes_module=Module.M11_OBSERVABILITY,
            score_delta=3,           # M11: 2/5 -> 5/5
            adds_cleverness=False,   # it's a log payload, not model logic
            changes_threat_model=False,
        ),
        ForkChange(
            name="threshold-triggered compaction summarizer",
            closes_module=Module.M3_CONTEXT,
            score_delta=2,           # M3: 2/5 -> 4/5 (a summarizer, not full JIT retrieval)
            adds_cleverness=False,   # it's a history-length check, not model logic
            changes_threat_model=False,
        ),
    ]


def evaluate_fork(changes: list[ForkChange], base_total: int) -> tuple[int, bool]:
    """Evaluate whether the fork preserves Pi's two core values.

    Returns (new_total, preserves_core_values).
    preserves_core_values is True only if NO change adds cleverness AND NO change
    alters the threat model.
    """
    new_total = base_total
    preserves = True
    for c in changes:
        # cap each module at 5/5
        delta = min(c.score_delta, 5 - REFERENCE_SCORES[c.closes_module].score)
        new_total += delta
        if c.adds_cleverness:
            preserves = False
        if c.changes_threat_model:
            preserves = False
    return new_total, preserves


def render_fork(changes: list[ForkChange], new_total: int, preserves: bool) -> None:
    print("=" * 88)
    print("MINIMAL PRODUCTION FORK — the three changes")
    print("=" * 88)
    for c in changes:
        clever = "adds cleverness (breaks co-evolution)" if c.adds_cleverness else "infrastructure only (preserves co-evolution)"
        threat = "changes threat model (breaks thinness)" if c.changes_threat_model else "threat model unchanged (preserves thinness)"
        print(f"\n  {c.name}")
        print(f"    Closes: {c.closes_module.value} (+{c.score_delta} score)")
        print(f"    {clever}")
        print(f"    {threat}")
    print("\n" + "-" * 88)
    print(f"Base total: 25/60  ->  Forked total: {new_total}/60")
    verdict = ("PRESERVES both core values (co-evolution + thinness). "
               "This is the correct 'minimal production' fork.") if preserves else \
              ("BREAKS a core value — re-examine the change.")
    print(f"Fork verdict: {verdict}")
    print("=" * 88)


# Add to __main__:
#   changes = model_minimal_production_fork()
#   new_total, preserves = evaluate_fork(changes, base_total=25)
#   render_fork(changes, new_total, preserves)
```

**Run it.** Expected output: the forked total is **31/60** (25 + 1 + 3 + 2), and `preserves_core_values` is `True` — no change adds cleverness or alters the threat model. This is the empirical proof that the three changes move Pi from "minimal baseline" to "minimal production" WITHOUT compromising its two core values.

**Record:** the forked total, the per-change score delta, and the `preserves_core_values` flag.

---

## Phase 4 — The fork-boundary analysis (5 min)

Add a function that articulates WHY the three infrastructure changes are correct while a single-component security addition is not.

```python
def articulate_fork_boundary() -> str:
    """Why the three infrastructure changes preserve Pi, and 'just add a sandbox' does not."""
    return (
        "FORK BOUNDARY ANALYSIS\n"
        "=======================\n\n"
        "The three 'minimal production' changes (token budget, observability, compaction)\n"
        "are correct because each:\n"
        "  1. Closes a SCORED operational gap (M1.2 stop, M10 log, M3 context)\n"
        "  2. Adds INFRASTRUCTURE (a cap, a payload, a summarizer) — not model-specific\n"
        "     cleverness, so the future-proof test still passes\n"
        "  3. Does NOT change the threat model (still trust-the-model, single-user)\n\n"
        "Contrast: 'just add a sandbox.' A sandbox:\n"
        "  1. Changes the THREAT MODEL (moves from single-user-trusted to multi-user)\n"
        "  2. Without the FULL security layer (scope + gates + tagging + observability),\n"
        "     gives the COST of the sandbox without the BENEFIT — indirect injection\n"
        "     (ASI01) still flows, in-container writes are still unrestricted, and\n"
        "     compromise is still invisible.\n\n"
        "CONCLUSION: infrastructure additions (the three changes) preserve Pi's core\n"
        "values. Security additions require the FULL layer (Course 2A), not one component.\n"
        "Fork Pi coherently, or not at all."
    )


# Add to __main__:
#   print(articulate_fork_boundary())
```

**Record:** the fork-boundary analysis. This is the principle that distinguishes a correct Pi fork from an incorrect one.

---

## Deliverables

Submit `dd01-rescore-pi.md` containing:

- [ ] The Phase 1 scorecard: 12 modules with scores, patterns, tradeoffs, and the 25/60 total (or a defended deviation).
- [ ] The Phase 2 dispatch trace: 7 implemented steps + 3 gap annotations, each gap connected to its module/ASI row.
- [ ] The Phase 3 fork evaluation: the three changes, their score deltas, the forked total (31/60), and the `preserves_core_values` flag.
- [ ] The Phase 4 fork-boundary analysis: why infrastructure changes preserve Pi and single-component security additions do not.
- [ ] A 3–4 sentence reflection: which of the three changes you would prioritize in your own Pi fork, and why the "add a sandbox" anti-pattern is tempting but wrong.

---

## Solution key

The expected outputs (assuming correct implementation):

- **Phase 1**: total = **25/60**. Matches the deep-dive reference. Deviations are acceptable if defended against the use case.
- **Phase 2**: 7 implemented steps (model emits tool_use → validate name → schema-validate → execute → return → append → next call) + 3 gaps (permission gate (M6), scope check (M5), taint tag (M2.4 / ASI01 surface)).
- **Phase 3**: forked total = **31/60** (25 + 1 + 3 + 2). `preserves_core_values` = `True`. No change adds cleverness; no change alters the threat model.
- **Phase 4**: infrastructure additions (cap, payload, summarizer) preserve co-evolution and thinness; security additions require the full layer (Course 2A), not one component.

The reflection should name the distinction between "infrastructure that closes an operational gap" (correct) and "a single security component that changes the threat model without the supporting layer" (the anti-pattern). The token-budget stop is the most common priority because it prevents the worst single outcome (runaway cost) for the fewest lines of code.

If the forked total is NOT 31/60, the most likely cause: a score delta was mis-capped (each module caps at 5/5) or a change was mis-attributed to the wrong module. Re-check the `closes_module` and `score_delta` for each `ForkChange`.

If `preserves_core_values` is `False`, a change was incorrectly marked as adding cleverness or changing the threat model. All three changes should be `False` for both flags.

---

## Stretch goals

1. **Model the incorrect fork.** Add a fourth `ForkChange` ("add a sandbox alone") with `changes_threat_model=True` and `closes_module=Module.M5_SANDBOX` with `score_delta=4`. Re-run `evaluate_fork`. Confirm: `preserves_core_values` flips to `False` — the sandbox alone changes the threat model without the supporting layer. This is the empirical version of the anti-pattern.
2. **Model the full security fork (Course 2A).** Replace the three infrastructure changes with the full security stack (sandbox + scope + gates + tagging + observability). Compute the new total and confirm it raises M5, M6, and M11 substantially — but note that this is a DIFFERENT fork for a DIFFERENT use case (multi-user/untrusted-input), not an upgrade to Pi's personal-assistant use case.
3. **Quantify the future-proof test.** Simulate a model upgrade by parameterizing the model's per-turn accuracy. Run Pi (dumb loop, thin prompt) vs a hypothetical thick harness (lookahead + reflection + thick prompt) across a sweep of model-accuracy values. Confirm Pi's performance improves monotonically with model accuracy (the harness adds nothing that degrades), while the thick harness may degrade if its lookahead logic conflicts with the upgraded model — the empirical future-proof test.