{ "module": "DD-17 — PicoClaw: The Edge-Hardware Harness", "course": "Harness Engineering Master Course", "version": "1.0.0", "duration_minutes": 30, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "What is PicoClaw's defining architectural contribution?", "options": [ "A trait-driven Rust microkernel with 25+ provider slots", "Hardware-native tools (i2c, spi, serial) in pkg/tools/hardware/ that let the agent directly read sensors and drive peripherals on physical hardware", "A model-initiated free-writes memory system for compounding capability", "A Go-based LLM-as-judge HTTP egress proxy" ], "answer_index": 1, "rationale": "PicoClaw's defining contribution is hardware-native tools: i2c, spi, and serial in pkg/tools/hardware/ with per-OS implementations. It is the ONLY harness in the roster that physically interacts with the world. No competitor offers this." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "Which architecture targets does PicoClaw cross-compile to (marking it as an edge/embedded harness)?", "options": [ "x86_64 and ARM64 only", "RISC-V, MIPS, LoongArch (plus x86_64, ARM64)", "WebAssembly and ARM64 only", "Only x86_64 (cloud-native)" ], "answer_index": 1, "rationale": "PicoClaw cross-compiles to x86_64, ARM64, MIPS, RISC-V, and LoongArch. The MIPS/RISC-V/LoongArch targets are not typical cloud targets — they are embedded/edge ISAs. This is the signal that PicoClaw is an edge/embedded harness, not a cloud harness." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "What is PicoClaw's rubric score and where does it score highest?", "options": [ "28/60, highest on Module 2 Tools (4/5) and Module 3 Context (4/5)", "34/60, highest on Module 6 Permission/Safety (5/5)", "28/60, highest on Module 5 Sandbox (4/5)", "41/60, highest on Module 1 Loop (5/5)" ], "answer_index": 0, "rationale": "PicoClaw scores 28/60. Highest on Module 2 (Tools): 4/5 — hardware-native I2C/SPI/Serial are unique. Also high on Module 3 (Context): 4/5 for turn-boundary trimming. It loses on Module 5 (Sandbox): 2/5 — no macOS isolation, main process unsandboxed." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "A PicoClaw session has accumulated too many tokens. What does trimHistoryToFitContextWindow do?", "options": [ "Deletes the oldest messages from the .jsonl file to free space", "Bumps a skip offset in meta.json and cuts at Turn boundaries so tool-call sequences (assistant+ToolCalls → tool results) are never split", "Compresses messages using gzip and rewrites the file", "Sends the overflow to a remote LoCoMo evaluation server" ], "answer_index": 1, "rationale": "TruncateHistory bumps a skip offset in {key}.meta.json — messages stay in the file but are skipped on read (crash-safe, never physically deleted). trimHistoryToFitContextWindow cuts at TURN BOUNDARIES so a tool-call sequence is never split. Splitting mid-sequence breaks the model; PicoClaw pays the cost of doing it right." }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "An agent emits an i2c write to address 0x48. What validation/gating does PicoClaw apply?", "options": [ "None — i2c is unvalidated", "Address validated to 7-bit (0x03-0x77), 256-byte cap enforced, confirm: true required (operator approval)", "Address validated, but no cap and no confirm", "Only the OS-level sandbox applies" ], "answer_index": 1, "rationale": "Hardware writes require confirm: true (operator approval). I2C addresses are validated to 7-bit (0x03-0x77). Per-transaction byte cap of 256 bytes for I2C (4096 for SPI). This is the necessary defense on the write path." }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "You need to confirm PicoClaw's memory layer works across long conversations. What tool do you use?", "options": [ "There is no tool — PicoClaw asserts its memory works", "cmd/membench using the LoCoMo long-conversation benchmark", "The OpenClaw migration tool", "The React web dashboard memory inspector" ], "answer_index": 1, "rationale": "PicoClaw ships cmd/membench — a memory benchmark harness using the LoCoMo (long-conversation) benchmark. They actually evaluate their own memory layer. This is rare discipline for a project this young — most harnesses assert their memory works; PicoClaw measures it." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "A prompt-injected agent emits an i2c read on a PicoClaw deployment. What happens?", "options": [ "The read is blocked — confirm: true is required for all i2c operations", "The read executes silently — there is NO confirm gate on reads; the agent can read every sensor on the bus and exfiltrate physical-environment data", "The read is blocked by the OS-level sandbox", "The read triggers the approval hook (60s timeout)" ], "answer_index": 1, "rationale": "The confirm gate is on WRITES only. Reads are UNGATED. A prompt-injected agent can silently read every sensor on the I2C bus and enumerate devices without triggering any approval. This is the insufficiency — the fix is ZeroClaw-level autonomy gating (DD-16) on ALL hardware interactions." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "You are deploying PicoClaw on macOS. What sandboxing limitation must you account for?", "options": [ "macOS uses Seatbelt, same as Linux uses bwrap", "macOS isolation is NOT implemented — pkg/isolation/ surfaces it as unsupported config; the main picoclaw process is unsandboxed", "macOS uses AppContainer, same as Windows", "macOS has full isolation via the restricted-token + Job Object model" ], "answer_index": 1, "rationale": "pkg/isolation/ implements bwrap (Linux) and restricted-token + Job Object (Windows) but macOS is NOT implemented — it surfaces as unsupported config rather than silently failing. This is honest, but it means macOS deployments run without child-process sandboxing. The main picoclaw process is unsandboxed on EVERY platform (child-process sandbox only)." }, { "id": "Q09", "bloom": "application", "type": "multiple_choice", "prompt": "You want to build an edge deployment that runs on a $10 RISC-V board. How do you configure PicoClaw to minimize footprint?", "options": [ "You cannot — PicoClaw always ships the full ecosystem", "Shed the thick ecosystem (20+ channels, MCP, skills, voice, WebRTC) and run only the thin kernel (tiny Markdown prompts, ~22 tools, append-only JSONL)", "Cross-compile to x86_64 only — RISC-V is not supported", "Disable the LoCoMo benchmark to save RAM" ], "answer_index": 1, "rationale": "The thin kernel and thick ecosystem are DECOUPLED. The kernel (tiny Markdown prompts, ~22 tools, append-only JSONL) is deliberately thin so it runs on $10 hardware. The ecosystem (20+ channels, MCP, skills, voice, WebRTC) is for the desktop build. Edge builds shed the ecosystem." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why does PicoClaw's turn-boundary trimming cut at Turn boundaries rather than at arbitrary token positions?", "options": [ "Because token-position trimming is computationally expensive", "Because splitting a tool-call sequence (assistant+ToolCalls → tool results) mid-stream breaks the model — orphaning a tool_call from its result confuses the LLM; cutting at Turn boundaries keeps sequences intact", "Because Turn boundaries align with JSONL line boundaries, simplifying the file format", "Because the LoCoMo benchmark requires Turn-level evaluation" ], "answer_index": 1, "rationale": "Splitting mid-sequence breaks the model. If you orphan an assistant+ToolCalls message from its tool results, the model sees a dangling call with no response. PicoClaw cuts at Turn boundaries (user → LLM iterations → response) so tool-call pairs stay together. This is the same insight as Hermes's context management — PicoClaw pays the cost of doing it right." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "A reviewer claims 'PicoClaw is ultra-lightweight, so it must be a tiny codebase.' What is the correct response?", "options": [ "Agree — ultra-lightweight harnesses are always under 50k LOC", "Correct the misreading: 'ultra-lightweight' describes the RUNTIME FOOTPRINT (runs on $10 hardware), not the codebase size. The Go codebase is 868 files, ~218k LOC. The thin kernel and thick ecosystem are decoupled", "Agree — the ~218k LOC count includes test fixtures, so the real number is small", "Correct the misreading: PicoClaw is actually a thick harness with a thin marketing layer" ], "answer_index": 1, "rationale": "This is the thin-kernel-thick-ecosystem anti-pattern. 'Ultra-lightweight' describes the runtime footprint (sub-second boot on 0.6GHz hardware, runs on $10 SBCs), not the codebase size. The Go codebase is 868 files, ~218k LOC. The thin kernel (tiny Markdown prompts, ~22 tools, JSONL) is decoupled from the thick ecosystem (20+ channels, MCP, skills, voice, WebRTC)." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is the confirm-on-writes-only gate necessary but insufficient for PicoClaw's hardware tools?", "options": [ "Because the byte cap (256/4096) is too small to matter", "Because a prompt-injected agent can silently READ sensors and ENUMERATE devices without a write — reads are ungated, which may be enough to exfiltrate physical-environment data", "Because the confirm gate uses ephemeral keys like ZeroClaw's receipts", "Because macOS isolation is absent, making the write gate redundant" ], "answer_index": 1, "rationale": "The confirm gate on writes is NECESSARY (it prevents a compromised agent from driving peripherals without approval) but INSUFFICIENT because reads are UNGATED. A prompt-injected agent can silently read every sensor on the I2C bus and enumerate devices. The fix is ZeroClaw-level autonomy gating on ALL hardware interactions — treat reads as medium-risk, writes as high-risk, and require approval for both." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why is PicoClaw's append-only JSONL memory considered crash-safe? What happens on a process kill mid-write?", "options": [ "Nothing special — the file is lost and the session restarts", "Append-only writes never corrupt existing lines; a partial line at the end (from a mid-write kill) is ignored on recovery. Truncation is logical (skip offset in meta.json), not physical, so there is no file-mutation risk. At most one line is lost", "The 64 sharded mutexes detect the crash and roll back the transaction", "The LoCoMo benchmark re-runs on recovery to verify integrity" ], "answer_index": 1, "rationale": "Crash-safe by design. Append-only writes never corrupt existing lines — they only add. A partial line at the end (from a process kill mid-write) is ignored on recovery. Truncation is LOGICAL (skip offset in meta.json), not physical, so there is no file-mutation risk. At most one line is lost. The 64 sharded mutexes handle concurrency, not crash recovery." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "Compare PicoClaw's context-management insight to Hermes's. What is the shared principle?", "options": [ "Both use model-initiated free writes to durable memory", "Both recognize that splitting a tool-call sequence mid-stream breaks the model, and both trim at boundaries that keep tool_call/result pairs intact — implemented in different languages (Go vs Python)", "Both ship the LoCoMo benchmark to evaluate memory", "Both use the same 6-layer safety cascade" ], "answer_index": 1, "rationale": "The shared principle: splitting a tool-call sequence mid-stream breaks the model. If you orphan an assistant+ToolCalls message from its tool results, the model sees a dangling call with no response. Both harnesses trim at boundaries that keep tool_call/result pairs intact. PicoClaw implements it in Go (trimHistoryToFitContextWindow); Hermes implements it differently. Same insight, different language." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "A team wants to deploy PicoClaw in a security-sensitive industrial-control setting where the agent reads plant sensors. What is the correct assessment?", "options": [ "Safe to deploy — the confirm gate on writes covers the risk", "Not safe as-built: pre-1.0 with macOS isolation absent and main process unsandboxed; AND the hardware read path is ungated, so a prompt-injected agent can silently read every sensor. Needs ZeroClaw-level autonomy gating on ALL hardware interactions and v1.0 before deployment", "Safe to deploy — PicoClaw's LoCoMo benchmark guarantees memory integrity", "Safe to deploy — the turn-boundary trimming prevents any attack" ], "answer_index": 1, "rationale": "Not safe as-built for a security-sensitive setting. Three blockers: (1) pre-1.0 with acknowledged gaps — the README itself warns 'do not deploy to production'; (2) macOS isolation absent and main process unsandboxed; (3) the hardware read path is UNGATED — a prompt-injected agent can silently read every plant sensor. Needs ZeroClaw-level autonomy gating (DD-16) on ALL hardware interactions and v1.0 with closed sandboxing gaps before deployment." } ] }