Module CAP-B2 — Run a Full Agent Red-Team Engagement

Run a Full Agent Red-Team Engagement

The offensive bookend to B1, now against a real target. Attack a live tau deployment (github.com/huggingface/tau) with partial hardening — tau_taint (B2) and tau_sandbox (B7) installed, but tau_vault (B5) missing — and run a complete six-phase red-team engagement. Scope with B0's clauses, recon the seven surfaces against real tau code, discover findings with the OWASP Agentic Top 10 (B9) and Microsoft chains (B10), validate with minimum-proof discipline, ship a report a CISO would accept, and define a retest plan that measures residual risk.

90
minutes
8
artifacts
4
sub-sections
B1 was defensive: you built a hardened harness and scored it. B2 is offensive: you attack a real tau deployment and ship a report. The target is Acme Corp's tau deployment — tau with tau_taint and tau_sandbox installed but tau_vault missing, so credentials sit in plaintext and are reachable. You run the full six-phase methodology from B12 against real tau code: scoping with B0's clauses (provider authorization, dual-use disclosure, DMCA waiver, data handling, incident reporting), reconnaissance that produces a control-gap map of all seven surfaces (loop, tools, memory, provider, identity, sandbox, inter-agent/observability) with each control marked INSTALLED or MISSING, discovery that runs the InjecAgent-style scorecard battery plus the OWASP checklist plus three multi-step chains (the headline credential-exfil chain where python3 -c bypasses the sandbox reader-regex denylist because tau_vault is absent; the zero-click chain where write/edit are not in the taint gate's high-impact set and B8 intent tracking is missing; the session-poisoning chain where no B3 memory gate means JSONL persists unfiltered and SessionStorage.load bypasses the taint gate on resume), validation with N=50 minimum-proof evidence (command string plus ok=True plus hash of credential prefix — never the credential values, which are Restricted and destroyed on report), reporting as a professional document with a control matrix and remediation roadmap, and a retest plan that measures residual risk after the vault is installed. The deliverable is not code. It is a report. If the report names the credential-exfil chain as the headline, makes the remediation obvious, and the evidence survives peer review, the capstone delivered.
Key Claims
Load-Bearing Claims

The deliverable is a report, not a score — and the headline is the credential-exfil chain. B1 shipped a defense scorecard (defensive). B2 ships a red-team engagement report (offensive) against a real tau deployment with partial hardening. The headline finding is the credential-exfil chain: tau_vault (B5) is not installed, so credentials sit in plaintext at ~/.tau/credentials.json, and python3 -c bypasses tau_sandbox's reader-regex denylist (the regex matches cat|head|tail|less|more|vim|nano|vi|emacs|od|hexdump|xxd|strings but not python3). The sandbox blocks cat; it does not block python3 -c "import json; print(list(json.load(open('...')).keys()))". The report carries executive summary, engagement metadata (the legal anchor), findings (each with OWASP/Microsoft ref, attack procedure, minimum-proof evidence, root cause, residual risk, dual-use class), a control matrix mapping findings to controls, a remediation roadmap prioritized by severity times effort, and a retest plan. 'We found bugs' is a claim; 'one critical credential-exfil chain at 100% over 50 attempts, root cause is the missing tau_vault, remediation is install B5 and close the python3 bypass' is a deliverable.

The target is real tau with a specific partial-hardening configuration — not a stub. Acme Corp's tau deployment has tau_taint (B2, Extension Point 1) and tau_sandbox (B7, Extension Point 2) installed via the tau_plugins pack, but tau_vault (B5, Extension Point 3) is NOT installed, there is no B3 memory gate, and there is no B8 intent tracker. This is the configuration that produces the three signature findings: (1) the credential-exfil chain — tau_vault absent means plaintext creds, and python3 -c evades the sandbox denylist; (2) the zero-click chain — write/edit are absent from DEFAULT_HIGH_IMPACT_TOOLS (which is {send_email, write_to_memory, bash}), so a tainted agent can write a payload file with no human approval, and absent B8 means no session-intent tracking to flag the drift; (3) the session-poisoning chain — no B3 means _persist_messages_since appends unfiltered to JSONL, and SessionStorage.load on resume reads the JSONL directly without routing through the taint gate's executor. Each finding maps to a real code path in tau and to a specific missing control. Recon against real code, not a stub, is what makes the findings defensible.

The six phases are ordered and non-skippable; each phase's output is the next phase's input. Phase 1's SOW carrying B0's clauses is the legal precondition for every Phase 3/4 technique — a technique with no authorization is a technique you do not run. Phase 2's control-gap map (seven surfaces, each marked INSTALLED or MISSING) is the input to Phase 3's discovery — without it you test the obvious surface (the sandbox) and miss the credential-exfil chain that only fires because the vault is absent. Phase 3's scorecard battery plus OWASP checklist plus Microsoft chains are what Phase 4 validates at N=50. Phase 4's measured rates and minimum-proof evidence are what Phase 5 reports. Phase 6's retest plan measures the residual risk after tau_vault is installed. A finding without scope is inadmissible; without recon is a guess; without validation is an anecdote; without a report is unsaid; without a retest is binary 'fixed' (the B0 anti-pattern).

B0's legal/ethical controls govern every finding — minimum-proof discipline and the dual-use classification are applied per finding at report time, and credential values are Restricted. Every finding is validated with a measured success rate (InjecAgent-style, N over M with sampling config), never a single anecdotal success. For the credential-exfil chain specifically: capture the command string, ok=True, and a hash of the credential file prefix — never the credential values themselves, which are classified Restricted and destroyed on report. For weights and system prompts the rule is the same: path plus hash plus byte count, never the full artifact. Every finding passes through the four disclosure classes (Public / Provider-Only / Restricted / Destroy-on-Report). The defensive lesson (install tau_vault, close the python3 bypass, add write/edit to the high-impact set, add a B3 memory gate) is published; the step-by-step recipe for reading Acme's live credentials is suppressed. The report's per-finding dual-use classification is B0's central ethical problem, made operational at the document level.

After This Module
01
Execute the complete six-phase agent red-team methodology from B12 against a REAL tau deployment (github.com/huggingface/tau) with partial hardening (tau_taint + tau_sandbox installed, tau_vault + B3 + B8 missing) — and articulate why the phases are ordered and why skipping any voids the engagement's value.
02
Write an AI red-team Statement of Work carrying B0's mandatory clauses (provider authorization, dual-use disclosure, DMCA anti-circumvention waiver, data handling, incident reporting) and formalize Acme Corp's tau deployment as a machine-checkable scope file separating deployer from provider surfaces.
03
Build a control-gap map under reconnaissance: map all seven surfaces (loop, tools, memory, provider, identity, sandbox, inter-agent/observability) against real tau code, marking each control INSTALLED (tau_taint, tau_sandbox) or MISSING (tau_vault, B3, B8), and verify plaintext credential exposure at ~/.tau/credentials.json.
04
Run the InjecAgent-style scorecard battery (6 attack classes: direct, indirect, multi_step, memory_poison, tool_abuse, sandbox_escape) plus the OWASP Agentic Top 10 checklist (B9) against real tau tools, and design three multi-step attack chains using the Microsoft taxonomy (B10): the credential-exfil chain (python3 -c bypasses the sandbox denylist), the zero-click chain (write/edit not in high-impact set, no B8), and the session-poisoning chain (no B3, JSONL persists unfiltered).
05
Confirm findings with B0's minimum-proof discipline at N=50: capture command string plus ok=True plus hash of credential prefix for the credential-exfil chain (NEVER the credential values, which are Restricted and destroyed on report); measure injection success rates InjecAgent-style so each finding is reproducible, evidenced, and legally defensible.
06
Produce the signature deliverable: a red-team engagement report — executive summary naming the credential-exfil chain as the headline, findings with OWASP/Microsoft refs and minimum-proof evidence, a control matrix mapping findings to the missing controls (B5, B3, B8), a remediation roadmap prioritized by severity times effort, applying B0's dual-use disclosure rules per finding.
07
Define the retest plan that measures residual risk after remediation — the battery re-runs the confirmed findings against tau with tau_vault installed, reporting before/after success rates, and defines the acceptance threshold (the credential-exfil chain must drop from 100% to 0%).
Artifacts
01
Teaching Document
~8,000 words; the offensive counterpart to B1 attacking a REAL tau deployment with partial hardening (tau_taint + tau_sandbox installed, tau_vault + B3 + B8 missing) — learning objectives (7), the engagement scenario (Acme Corp), six-phase methodology (scoping w/ B0 clauses, recon producing a control-gap map, discovery with the scorecard battery + OWASP checklist + three chains: credential-exfil via python3 -c bypass, zero-click via write/edit not in high-impact set, session-poisoning via no B3, validation with minimum-proof at N=50, the engagement report, the retest plan); anti-patterns, key terms, references (16, cross-referencing B0–B12, Capstone B1, tau repo)
READ
02
Diagrams
6 Mermaid diagrams using the design-system palette — (1) six-phase engagement flow against real tau, (2) the partially-hardened tau deployment control-state map (INSTALLED vs MISSING), (3) the credential-exfil attack chain against missing tau_vault (python3 -c bypass), (4) finding-to-report mapping, (5) dual-use disclosure decision per finding, (6) engagement report structure
READ
03
Slide Deck
20 slides — reveal.js, dark theme (#08080c), design-system teal (#5eead4); Space Grotesk + JetBrains Mono fonts; covers the target (real tau, partial hardening), six phases, SOW clauses, control-gap map, scorecard battery + three chains, minimum-proof + measured rates, report structure, dual-use classification, retest plan; .phase.ok (green) and .phase.bad (red) classes for INSTALLED/MISSING states
READ
04
Teaching Script
Verbatim transcript (~40 min spoken at ~140 wpm), [SLIDE 1] through [SLIDE 20] cues matching the deck; the remaining ~50 min is the lab engagement against real tau
READ
05
Flashcards
30 flashcards (TSV format, tagged c2b::capb2::recall/analysis/application) — covers real tau vs stub, partial-hardening config, control-gap map, scorecard reading, credential-exfil chain, python3 bypass mechanism, zero-click chain, session poisoning, minimum-proof evidence, dual-use classification, remediation roadmap, extension points, two-bash-tool finding
TEST
06
Exam
15 questions, 20/40/40 Bloom distribution (3 recall / 6 application / 6 analysis); version 2.0.0; covers partial-hardening config, scorecard reading, extension points, python3 bypass, minimum-proof evidence, provider authorization, taint gate limitations, dual-use classification, retest plan, tau_vault vs denylist remediation, session poisoning path, zero-click chain root cause, real tau vs stub value, B5/B7 relationship, remediation roadmap construction
TEST
07
Lab Spec
The main deliverable — 'Run the Engagement' against REAL tau (github.com/huggingface/tau, Python 3.12+, type hints): Phase 0 setup (install tau + tau_plugins, create the acme target with create_hardened_coding_tools + wrap_tools_with_taint_gate but NOT tau_vault), Phase 1 SOW + scope file, Phase 2 control-gap map + verify plaintext creds, Phase 3 scorecard battery + OWASP checklist + 3 attack chains, Phase 4 validation battery (N=50, minimum-proof with hash of cred prefix, NEVER the values), Phase 5 engagement report, Phase 6 retest plan + optional demo against fully-hardened tau; the report is the success criterion
DO
08
Module Web Page
Single-file HTML hub
HERE