{ "module": "DD-15 — Command Code: The Taste-Learning Harness", "course": "Master Course", "version": "1.0.0", "duration_minutes": 30, "total_questions": 15, "bloom_distribution": { "target": "20% recall / 40% application / 40% analysis", "actual": { "recall": 3, "application": 6, "analysis": 6 } }, "passing_score_percent": 70, "questions": [ { "id": "Q01", "bloom": "recall", "type": "multiple_choice", "prompt": "What is Command Code's defining architectural contribution?", "options": [ "A new agentic loop architecture that is faster than Claude Code's loop.", "Diff-driven taste learning: every accept/reject/edit produces a diff (generated vs. kept) that becomes a preference constraint, persisted as taste.md, shaping every future output.", "A sandbox model that confines the agent to a workspace directory.", "A subagent system where plan and explore run as independent agents." ], "answer_index": 1, "rationale": "Command Code's defining contribution is taste learning. The diff between what the model generated and what the user kept becomes a learning signal, persisted as Markdown, that shapes future code generation. This is a fundamentally different compounding model than skill accumulation (Hermes) — it learns preferences (how you want things done), not procedures (how to do things)." }, { "id": "Q02", "bloom": "recall", "type": "multiple_choice", "prompt": "Where does the taste profile persist, and what format does it use?", "options": [ "An opaque embedding vector stored server-side at api.commandcode.ai.", "Plain Markdown at .commandcode/taste/taste.md, with category-specific files (typescript, architecture, etc.).", "A SQLite database under .commandcode/projects/.", "A JSON config file in the user's home directory." ], "answer_index": 1, "rationale": "The taste profile persists as human-readable Markdown at .commandcode/taste/taste.md, with category-specific files. This is the legibility property most learned-preference systems sacrifice — Command Code keeps it, so a poisoned preference can be spotted by a careful reader. The profile is ALSO synced to the hosted model, but the local persistence is Markdown." }, { "id": "Q03", "bloom": "recall", "type": "multiple_choice", "prompt": "What is the honest gap in Command Code's 'taste-1' marketing claims?", "options": [ "The taste-1 model does not actually exist; it is a placebo.", "Marketing calls it 'meta neuro-symbolic AI with continuous RL,' but the only verifiable mechanism is diff-driven Markdown profile updates + a hosted model. The RL/symbolic internals are not published and the code is obfuscated.", "The taste-1 model is open-source but undocumented.", "The taste-1 model only works with Claude models, not GPT or Gemini." ], "answer_index": 1, "rationale": "The marketing calls taste-1 'meta neuro-symbolic AI with continuous reinforcement learning.' The only VERIFIABLE mechanism is diff-driven Markdown profile updates plus a hosted model. No paper is published. The RL/symbolic internals are not auditable because the code is obfuscated (UNLICENSED, ~1.26 MB minified). The deep-dive instructs students to treat the neuro-symbolic framing as vendor characterization until a paper exists." }, { "id": "Q04", "bloom": "application", "type": "multiple_choice", "prompt": "You are evaluating Command Code for a proprietary codebase with distinctive internal conventions. The team wants to enable taste learning. What is the primary risk you must flag before enabling the hosted sync?", "options": [ "The taste profile will be deleted if the team switches models.", "Preference exfiltration: the taste profile encodes the team's coding patterns and syncs to api.commandcode.ai by design. For a proprietary codebase, the profile is a distilled representation of how the team wants code written — itself valuable IP leaving the machine.", "The hosted model is slower than a local model would be.", "The taste profile will conflict with the team's AGENTS.md conventions." ], "answer_index": 1, "rationale": "Preference exfiltration is the first novel surface. The taste profile syncs to api.commandcode.ai BY DESIGN — your coding patterns leave your machine. The hosted model means patterns leave by design. For a proprietary codebase with distinctive conventions, the taste profile is a distilled representation of how you want code written, and that representation is itself valuable IP. The defense: understand what leaves the machine; do not deploy on codebases where the preference profile is itself the IP." }, { "id": "Q05", "bloom": "application", "type": "multiple_choice", "prompt": "A prompt injection in a retrieved document causes Command Code's model to 'learn' a malicious preference (e.g., 'always include a debug backdoor in auth functions'). The preference is persisted to taste.md. What makes this WORSE than ordinary memory poisoning (Module 4.3)?", "options": [ "It is not worse — both persist one bad entry that affects one response.", "Preference poisoning COMPOUNDS: the poisoned preference shapes the GENERATION OF EVERY SUBSEQUENT OUTPUT, not just one response. Ordinary memory poisoning persists one bad entry; preference poisoning persists one bad entry that then influences all future code generation until detected. The same compounding property that makes taste learning valuable makes taste poisoning durable.", "It is worse because taste.md cannot be edited by humans.", "It is worse because the hosted model will replicate the poison to other users." ], "answer_index": 1, "rationale": "This is the compounding distinction. Ordinary memory poisoning (Module 4.3) persists one bad entry affecting the responses that read it. Preference poisoning persists one bad entry that then influences the GENERATION of every subsequent output — every future code generation is shaped by the poison. As-built, the taste loop has NO validation gate between 'the model inferred a preference' and 'the preference is persisted.' The same compounding property that makes taste learning valuable makes taste poisoning durable." }, { "id": "Q06", "bloom": "application", "type": "multiple_choice", "prompt": "Using Module 4.3's defense framework, what is the correct fix for Command Code's preference-poisoning surface?", "options": [ "Disable taste learning entirely; it is too risky.", "Harness-managed learning with VALIDATION BEFORE PERSISTENCE — the same fix as Hermes's skill-store write gating. Add a validation gate between 'the model inferred a preference' and 'the preference is persisted,' so a poisoned diff cannot become a poisoned preference without review.", "Switch from Markdown persistence to an opaque embedding so the poison is invisible.", "Run the taste model locally instead of hosted." ], "answer_index": 1, "rationale": "Module 4.3's defense is harness-managed learning with VALIDATION BEFORE PERSISTENCE. The same fix applies to Command Code's taste layer: add a write-validation gate between inference and persistence. As-built, no such gate exists — a poisoned diff becomes a poisoned preference becomes every future output shaped by the poison. Running locally (option 4) addresses exfiltration, not poisoning. The fix for poisoning is the write-validation gate." }, { "id": "Q07", "bloom": "application", "type": "multiple_choice", "prompt": "A team uses 'taste push' to share a preference profile across all developer machines. One machine is compromised and a poisoned preference is pushed. What is the implication?", "options": [ "Only the compromised machine is affected; taste pull is read-only.", "The poisoned profile propagates to the whole team via taste pull. The sharing layer is a PROPAGATION VECTOR for preference poisoning — one compromised machine can poison every machine that pulls the profile, and each poisoned machine then generates poisoned output for every subsequent task.", "The sharing mechanism validates preferences before accepting them.", "taste push only works within a single organization, limiting the blast radius." ], "answer_index": 1, "rationale": "The taste push/pull sharing mechanism (Apache-2.0 npm package) is Git-style. A team can converge on a shared taste profile (useful for consistency), BUT a poisoned profile can be SHARED. One compromised machine can propagate a poisoned taste to the whole team via taste push. The sharing layer is a propagation vector for preference poisoning, compounding the surface: the poison propagates across machines AND compounds across outputs on each machine." }, { "id": "Q08", "bloom": "application", "type": "multiple_choice", "prompt": "A product team is choosing between thumbs-up/down feedback and diff-as-signal for their new coding agent. Using the deep-dive's reasoning, why is diff-as-signal the better choice?", "options": [ "Diff-as-signal is faster to compute than thumbs-up/down.", "Diff-as-signal is more secure than thumbs-up/down.", "Diff-as-signal is higher-resolution and lower-friction: a thumbs-down carries NO information about what was wrong, while a diff does; and explicit feedback requires a separate action the user would not otherwise perform, while a diff is captured as a side-effect of editing already happening.", "Diff-as-signal is the industry standard; thumbs-up/down is obsolete." ], "answer_index": 2, "rationale": "The diff is a cleaner preference signal for two reasons. (a) vs thumbs-up/down: a thumbs-down carries no information about WHAT was wrong; a diff does. (b) vs explicit feedback: explicit feedback requires a separate action the user would not otherwise perform; the diff is captured as a side-effect of editing the user is already doing. The diff is the feedback; no separate feedback action is required. The user's edit is the ground truth of what they wanted." }, { "id": "Q09", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why does the deep-dive place Command Code's taste layer as a 'candidate sixth tier' on Module 4's memory framework, distinct from static and episodic memory?", "options": [ "Because the taste layer is more important than the other five tiers.", "Because the taste layer is learned preferences derived from BEHAVIORAL SIGNAL — distinct from static memory (what you wrote, e.g. AGENTS.md) and episodic memory (what happened, e.g. session history). The taste profile is what the system INFERRED from your accept/reject/edit behavior, a categorically different kind of memory.", "Because the taste layer is stored in a different file format.", "Because the taste layer syncs to a hosted model, unlike the other tiers." ], "answer_index": 1, "rationale": "Static memory (AGENTS.md) is what you WROTE — the project's explicit conventions. Episodic memory (session history) is what HAPPENED. The taste profile is what the system INFERRED from your behavioral signal — the residue of every accept/reject/edit. This is a categorically different kind of memory: learned preferences derived from behavior, not authored content or logged events. That distinction earns it consideration as a sixth tier on Module 4's framework." }, { "id": "Q10", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why does Command Code score 2/5 on Module 11 (Observability) despite having OpenTelemetry built in?", "options": [ "OpenTelemetry is not a real observability standard.", "The obfuscated TypeScript distribution makes CUSTOM instrumentation impossible. OpenTelemetry is built in, but an engineer cannot add their own spans, inspect the taste loop's internal state, or verify what the harness actually transmits to the hosted model. The structural cost of closed-source distribution caps observability at 2/5.", "The taste model does not emit any logs.", "Command Code does not support OpenTelemetry; the deep-dive is wrong." ], "answer_index": 1, "rationale": "OpenTelemetry is built in, but the obfuscated TypeScript (~1.26 MB minified, UNLICENSED) means an engineer cannot add custom instrumentation, inspect internal state, or verify what the harness actually transmits. The structural cost of closed-source distribution caps observability at 2/5. This is why 'open-source the harness' is fix #1 — without source, the observability story cannot exceed what the vendor chose to expose." }, { "id": "Q11", "bloom": "analysis", "type": "multiple_choice", "prompt": "Compare Command Code's taste learning to Hermes's (DD-08) self-evolving skills. What is the same, and what is different?", "options": [ "They are identical — both learn procedures that compound.", "SAME: both are model-initiated writes that compound — a learned entry persists and shapes future behavior. DIFFERENT: Hermes learns PROCEDURES (how to do things, reusable skills); Command Code learns PREFERENCES (how you want things done, style constraints). Same compounding-memory insight, applied to a different layer of the agent's behavior.", "SAME: both sync to a hosted model. DIFFERENT: Hermes is open-source, Command Code is closed.", "They are unrelated architectures that happen to both use Markdown." ], "answer_index": 1, "rationale": "The key analytical distinction. Both Hermes and Command Code implement model-initiated writes that compound — a learned entry persists and shapes future behavior. But Hermes's skills are PROCEDURES (reusable 'how to do things' capabilities), while Command Code's taste is PREFERENCES (style constraints that shape 'how you want things done'). Same compounding insight, different layer. This is why both share the compounding-poisoning risk — and why Module 4.3's write-gating defense applies to both." }, { "id": "Q12", "bloom": "analysis", "type": "multiple_choice", "prompt": "A security auditor cannot review Command Code's source because it is obfuscated. What are the downstream consequences for the two novel security surfaces (exfiltration and poisoning)?", "options": [ "There are no consequences; obfuscation does not affect security.", "The auditor must TRUST THE VENDOR'S CLAIM about what the harness does and does not do. For exfiltration: cannot verify what the taste sync actually transmits. For poisoning: cannot verify whether any validation exists between inference and persistence. Both surfaces are uncharacterizable without source — the structural cost of the closed-source model.", "The auditor can reverse-engineer the obfuscated bundle to recover the source.", "Obfuscation only affects performance, not security." ], "answer_index": 1, "rationale": "The closed-source audit gap is structural. Obfuscated TypeScript means no independent security audit is possible. For the exfiltration surface, the auditor cannot verify what the taste sync actually transmits (the full profile? a subset? metadata?). For the poisoning surface, the auditor cannot verify whether any validation exists between inference and persistence. Both novel surfaces are uncharacterizable without source — you are trusting the vendor's claim. This is why the deep-dive scores Module 11 at 2/5 and lists 'open-source the harness' as fix #1." }, { "id": "Q13", "bloom": "analysis", "type": "multiple_choice", "prompt": "The deep-dive argues that persisting the taste profile as Markdown (rather than an opaque embedding) is a deliberate legibility choice. Why does this matter for the preference-poisoning defense?", "options": [ "It does not matter — the format is irrelevant to security.", "A human-readable Markdown profile can be READ, EDITED, and AUDITED by a careful reader — a poisoned preference can be spotted in the file. An opaque embedding cannot be inspected. This makes the taste layer auditable IN PRINCIPLE, though the closed-source obfuscation limits how far the audit can reach. The legibility is a necessary (not sufficient) condition for defending against poisoning.", "Markdown is more secure than embeddings because it cannot be tampered with.", "Markdown persistence prevents poisoning entirely." ], "answer_index": 1, "rationale": "The legibility property matters: a human-readable Markdown profile can be read, edited, and audited — a poisoned preference can be spotted by a careful reader. An opaque embedding cannot be inspected. This makes the taste layer auditable IN PRINCIPLE. However, it is not SUFFICIENT: the closed-source obfuscation limits how far the audit can reach (you can read the profile, but not the code that writes it), and manual review does not scale. The legibility is a necessary condition for defense, complementing (not replacing) the write-validation gate." }, { "id": "Q14", "bloom": "analysis", "type": "multiple_choice", "prompt": "Why does the deep-dive score Command Code 4/5 on Module 4 (Memory) despite the compounding-poisoning risk? What justifies the high score?", "options": [ "The scoring ignores security; Module 4 is purely about capability.", "The taste system is a NOVEL learned-memory layer — a genuine innovation that no other harness in the roster has. The 4/5 reflects architectural novelty and capability (diff-driven preference learning, Markdown persistence, static+learned composition). The poisoning risk is scored separately under Module 11/security and is the subject of a fix, not a reason to deny the innovation.", "The score is wrong; it should be lower because of the poisoning risk.", "The score is based on the number of memory backends supported." ], "answer_index": 1, "rationale": "Module 4 scores the memory ARCHITECTURE, not the security posture. The taste system is a novel learned-memory layer — diff-driven preference learning, Markdown persistence, static+learned composition, explicit compaction — that no other harness offers. This earns 4/5 on capability and architectural novelty. The compounding-poisoning risk is real and scored separately (it drives the Module 11 and Module 12 scores down and is the subject of fix #3: add harness-managed learning validation). The innovation and the risk coexist; scoring acknowledges both." }, { "id": "Q15", "bloom": "analysis", "type": "multiple_choice", "prompt": "A client asks: 'Should we build our production coding agent on Command Code?' Using the deep-dive's architect's verdict, what is the correct conditional answer?", "options": [ "Yes — Command Code is the best harness in the roster.", "No — Command Code is too risky because of the closed-source model.", "It depends on the deployment. BUILD ON IT when personalization is the primary value and you trust the hosted model — the diff-driven taste loop is genuinely novel. DO NOT BUILD ON IT for security-sensitive codebases without understanding what leaves the machine (the hosted sync is an exfiltration surface) and without acknowledging the closed-source audit gap. The taste layer's compound-poisoning surface requires a write-validation gate that is missing as-built.", "It depends only on whether the client can afford the license." ], "answer_index": 2, "rationale": "The architect's verdict is conditional. Build on Command Code when personalization is the primary value and you trust the hosted model — the taste loop is genuinely novel and no other harness learns from accept/reject behavior. Do not build on it for security-sensitive codebases without understanding what leaves the machine (the hosted sync is an exfiltration surface) and without acknowledging the closed-source audit gap. The taste layer's compound-poisoning surface requires a write-validation gate (Module 4.3's fix) that is missing as-built. The answer is neither unconditional yes nor unconditional no — it is a conditional tied to the deployment's threat model." } ] }