# Module S03 — Pentest and Red Team Harnesses

**Course**: 2A — Building AI Harnesses for Cybersecurity
**Module**: S03 — Pentest and Red Team Harnesses
**Duration**: 90 minutes
**Level**: Senior Engineer and above
**Prerequisites**: S00, S01, S02 complete

> *Full-engagement autonomous operations across multi-stage kill chains. Kill chain state management as an attack graph, plan correction when exploits fail, the sandbox inversion problem, and the report as the actual deliverable.*

---

## Learning Objectives

1. Design an attack graph as the harness's primary state structure, with MITRE ATT&CK technique IDs as node labels and per-node states (unexplored, attempted, succeeded, failed, out-of-scope).
2. Implement RedTeamLLM-style plan correction: detect failure, re-hypothesize, re-plan, and retry with an alternative path — and defend the choice between generalizing and specializing an attack pattern.
3. Configure dual containment (outbound scope enforcement + inbound isolation) for an offensive harness running in Docker, and verify both directions independently.
4. Build a report generator that turns structured findings into a client-ready HTML/PDF/JSON deliverable with CVSS drafts (clearly marked for human review), evidence citations, and CWE/OWASP-mapped remediation.
5. Diagnose the failure modes specific to multi-stage adversarial harnesses: intermediate state loss, plan lock-in, sandbox inversion, and report hallucination.

---

# S03.1 — Kill Chain State Management

S02 built engagement memory as a flat surface: domains, ports, services, findings. That representation works when each finding is independent — an XSS on `/search` has no structural dependency on an open port finding. A red team engagement is different. A red team kill chain is a *sequence with dependencies*: foothold requires initial access, lateral movement requires foothold, privilege escalation requires a session on the target, exfiltration requires privilege. The state is not a set of independent findings; it is a graph of dependent objectives.

## MITRE ATT&CK as the state machine

MITRE ATT&CK is the canonical taxonomy. It is also, when used correctly, a state machine for the harness loop. Each ATT&CK technique is a node; the kill chain is a path through nodes; the engagement state is the set of nodes reached so far.

```typescript
interface AttackNode {
  id: string;                        // stable UUID
  tactic: string;                    // ATT&CK tactic: e.g. "TA0001 Initial Access"
  technique: string;                 // ATT&CK technique: e.g. "T1190 Exploit Public-Facing Application"
  target: string;                    // which asset this node applies to
  state: "unexplored" | "attempted" | "succeeded" | "failed" | "out_of_scope";
  prerequisites: string[];           // node IDs that must be in "succeeded" state
  evidence_refs: string[];           // evidence record IDs
  attempts: AttemptRecord[];         // every try, with payload, result, timestamp
  last_updated: string;
}

interface AttackGraph {
  engagement_id: string;
  nodes: Record<string, AttackNode>;
  current_position: string[];        // node IDs the agent is positioned at
  edges: Record<string, string[]>;   // adjacency — enables/enables
  scope_ref: string;
}
```

The critical design decision is that `prerequisites` and `edges` are explicit. A harness that cannot tell you "why am I allowed to attempt lateral movement now" will attempt it too early (before foothold) and waste cycles, or too late (after detection) and get caught.

## Attack graph vs. linear plan

A linear plan is a checklist: step 1, step 2, ..., step N. It works when every step succeeds. In an adversarial environment, steps fail, and a linear plan has no native notion of "what else could I try." The bug bounty triage pipeline from S02.4 handles independent findings; it cannot represent "I tried `T1190 Exploit Public-Facing Application`, it failed, so now I want to try `T1078 Valid Accounts` via credential reuse instead, because both satisfy the Initial Access prerequisite for the next tactic."

The attack graph is the right primary structure for any engagement longer than a single exploit. Rules:

- **Linear plan when**: single objective, single technique, high confidence in success. A one-shot CVE validation.
- **Attack graph when**: multi-objective, multiple techniques per objective, dependencies between objectives. Any engagement the client is paying more than a day for.
- **Switch mid-engagement**: start linear, promote to a graph the moment a step fails or a second objective appears. The promotion is cheap — a linear plan is a degenerate graph (a single chain).

The most common harness failure mode here is the opposite: starting with a graph, then collapsing it back into a linear plan "for simplicity" when the graph gets dense. That collapse throws away the alternative paths that the agent will need when its primary technique fails.

## Why general harnesses lose the chain

A general agent harness (Course 1's default loop) calls a tool, gets a result, and reasons about the next step. Between tool calls, the only state is the context window. Across sessions, there is no state. For a kill chain that requires ten sequential objectives, this fails in two specific ways:

1. **Intermediate state loss between tool calls.** The model attempts initial access, succeeds, and the result goes into the context window as a tool result. Three tool calls later, the context compacts, and the fact that initial access succeeded is gone. The agent attempts lateral movement, which requires a session from initial access, and the prerequisite check fails — not because the prerequisite wasn't met, but because the harness forgot it was met.

2. **Cross-session state loss.** The engagement pauses on day one with foothold achieved. On day two, a fresh session has no memory of the foothold. The agent either re-attempts initial access (wasting time, generating noise, risking detection) or, worse, proceeds as if no foothold exists and misses the dependency.

The attack graph solves both because it is persisted outside the context window and queried before every step. The agent asks the graph: "what is my current position? what prerequisites are met? what techniques are available to me now?" The context window carries the reasoning for the current step; the graph carries the structural truth across the whole engagement.

## Node state transitions and why they are explicit

The five node states are not arbitrary labels — each represents a distinct operational condition, and the transitions between them are the harness's control flow. A node moves from `unexplored` to `attempted` only when the harness has verified prerequisites are met and the agent has selected the technique. It moves from `attempted` to `succeeded` only when the objective is confirmed — not when the exploit returns output, but when the output is verified to have achieved the objective (a shell, a credential, a file). It moves from `attempted` to `failed` on a hard failure, which triggers plan correction. It moves to `out_of_scope` when the scope check rejects the target or action — and this transition is terminal, because re-checking an out-of-scope node every cycle wastes computation.

The transition that teams get wrong is `attempted` to `attempted` — the soft-failure retry. A soft failure (timeout, rate limit) should retry the same node, but only with backoff, and only up to a configurable limit. Without the limit, a flapping target can loop the harness indefinitely. Without distinguishing soft from hard failure (S03.2), a patched exploit loops just the same. The transition table is part of the harness design, not an afterthought.

## Cross-session continuity

Pausing and resuming a multi-day engagement cleanly is a hard requirement, not a nice-to-have. The mechanism:

- The attack graph is serialized to disk (JSON or a graph database) at every state transition.
- On resume, the harness loads the graph, reconstructs `current_position`, and verifies each node's `evidence_refs` still resolve to valid evidence records.
- The agent's first action on resume is a *graph orientation prompt*: "Here is the attack graph. You are positioned at these nodes. The next available techniques are these. What is your next action?" — not a continuation of yesterday's reasoning.

The orientation prompt matters because yesterday's reasoning may be based on target state that has changed. Overnight, the target's admin may have patched the CVE you were planning to exploit, rotated the credentials you harvested, or added detection that makes your foothold noisy. The graph is structural truth about what was achieved; it does not claim that what was achieved is still valid. The orientation prompt explicitly asks the agent to re-validate its position before proceeding — a quick recon pass to confirm the foothold session is still alive, the credentials still work, the open ports are still open. If re-validation fails, the affected nodes transition back toward `failed`, and plan correction engages.

This is the difference between resuming an engagement and replaying one. A replay assumes nothing changed. A resume assumes everything may have changed and verifies before proceeding.

## Why ATT&CK and not a custom taxonomy

A senior engineer's first instinct is often to design a custom state space tailored to the specific engagement — fewer nodes, more relevant techniques, no ATT&CK overhead. This is usually a mistake. ATT&CK is verbose because it is complete; it covers the techniques you did not think of as well as the ones you did. A custom taxonomy encodes the team's prior assumptions about what the engagement will look like, and those assumptions are the first thing an adversarial environment invalidates.

ATT&CK also gives you interoperability: threat intelligence feeds, detection rules, and remediation mappings are all keyed to ATT&CK technique IDs. A finding reported as "T1190 succeeded" can be cross-referenced against the client's existing detection coverage, the team's knowledge base of technique-specific payloads, and the mitigation literature. A finding reported as "we exploited the web app" cannot. The verbosity is the value.

---

# S03.2 — Plan Correction Under Adversarial Conditions

In Course 1, an error is a tool that returned a traceback. The fix is retry, or report the error. In an offensive engagement, an "error" is an exploit that didn't land, a credential that didn't work, a path that turned out to be patched. The correct response is not retry-the-same-thing — it is *re-plan*. This is the attacker's equivalent of error handling, and it is the single biggest differentiator between a script that runs exploits and a harness that conducts an engagement.

## The RedTeamLLM plan correction mechanism

The RedTeamLLM paper (Brown et al., 2024) formalizes a four-step plan correction loop that triggers on exploit failure:

1. **Detect failure.** Distinguish a hard failure (exploit returned no result, target patched, authentication rejected) from a soft failure (timeout, rate limit, transient network error). Hard failures trigger plan correction; soft failures trigger retry-with-backoff. Conflating the two is the most common harness bug in this category — retrying a patched exploit three times before realizing it's a hard failure wastes engagement time and generates detectable noise.

2. **Re-hypothesize.** The original hypothesis was "technique T1190 will give me initial access via this CVE." The failure invalidates that hypothesis. The re-hypothesis step asks: what else could satisfy the Initial Access tactic? The harness consults its knowledge base for alternative techniques that satisfy the same tactic and whose prerequisites are met.

3. **Re-plan.** Select the next-best alternative and update the attack graph: mark the failed node as `failed`, add the new candidate node as `unexplored`, update `current_position` if the failure means the agent is no longer positioned where it thought it was.

4. **Retry with a different approach.** Execute the new technique. This is a new attempt on a new node — not a retry of the old one.

```python
async def plan_correction_loop(
    failed_node: AttackNode, graph: AttackGraph, knowledge_base, scope
) -> AttackNode:
    # 1. Already established this is a hard failure (caller verified)
    graph.mark_failed(failed_node.id)

    # 2. Re-hypothesize: what else satisfies this tactic?
    alternatives = knowledge_base.find_techniques(
        tactic=failed_node.tactic,
        prerequisites_met=graph.met_prerequisites_for(failed_node),
        exclude_failed=True,
    )
    if not alternatives:
        raise NoViablePath(f"No remaining techniques for {failed_node.tactic}")

    # 3. Re-plan: select next-best, add to graph
    next_technique = alternatives[0]  # ranked by knowledge_base heuristics
    new_node = graph.add_node(
        tactic=failed_node.tactic,
        technique=next_technique.id,
        target=failed_node.target,
        prerequisites=failed_node.prerequisites,  # same prereqs as the failed attempt
        state="unexplored",
    )

    # 4. Retry (caller executes the new node)
    return new_node
```

## Detecting failure: hard vs. soft

The detection step is the one most often implemented incorrectly. A hard failure is a definitive negative: the exploit returned no result, the target responded with a patch indicator (a version banner that excludes the CVE, an authentication rejection, a 404 on the vulnerable endpoint). A soft failure is a transient negative: a timeout, a rate-limit response, a connection reset that could indicate network conditions rather than target state.

The distinction matters because the responses are structurally different. A soft failure retries the same node — the technique is still viable, the environment was temporarily hostile. A hard failure re-plans — the technique is not viable, a different technique is required. Conflating them produces two failure modes: retrying a patched exploit burns time and generates noise (the target's logs fill with failed attempts, detection risk rises); re-planning on a timeout abandons a viable technique prematurely and may select a worse alternative.

The reliable signal is whether the failure is reproducible. A single timeout is soft; three consecutive timeouts against a target that responds to other requests is hard (the service is down or blocking you). A single authentication rejection is hard (credentials are wrong); it does not become soft by retrying. The harness should implement a small classifier — a handful of rules mapping response patterns to hard/soft — and log the classification with the attempt so an operator can audit the decisions later.

## Exploit substitution

When the primary exploit fails, the harness selects the next-best from its knowledge base. The knowledge base is not a flat list — it is ranked by target context. A web-app exploit ranks higher against a target whose attack surface is web-facing; a phishing pretext ranks higher against a target with human operators and email exposure. The ranking uses the engagement memory from S02.1 (what services are exposed, what OS, what user accounts are known).

The ranking is updated as the engagement progresses. Early in the engagement, when the attack surface is poorly characterized, the ranking is coarse — all web techniques rank similarly because the harness does not yet know the specific frameworks. After recon populates the engagement memory with version numbers and configurations, the ranking refines: a technique targeting the confirmed framework ranks above a generic technique. This is why the knowledge base query runs at plan-correction time, not at engagement start — the ranking should reflect the latest intelligence.

The anti-pattern here is *exploit lock-in*: the harness has a favorite exploit, it fails, and the harness retries it with minor variations instead of substituting a different technique. This is the harness equivalent of a penetration tester who only knows one tool. The cure is the explicit `exclude_failed=True` in the knowledge base query — failed techniques are removed from the candidate set, forcing genuine substitution. The new node added by plan correction must use a different technique ID than the failed node; the harness should assert this invariant.

## Generalize vs. specialize

When selecting an alternative, the harness faces a choice:

- **Generalize**: use a broad attack pattern that works across many targets (e.g., `T1110 Brute Force` against any exposed login form). Higher reliability, lower stealth, often noisier.
- **Specialize**: use a target-specific payload (e.g., a CVE for the exact software version identified during recon). Higher impact if it works, but fails immediately if the version is wrong or patched.

The heuristic: specialize first when you have high-confidence target intelligence (exact version, confirmed configuration). Generalize when intelligence is low or when the specialized technique has already failed and you need a fallback. The RedTeamLLM paper's ablation shows that harnesses with a generalize-on-failure fallback complete 40% more objectives than harnesses that only specialize — because in an adversarial environment, the specialized intelligence is frequently wrong.

---

# S03.3 — The Sandbox Inversion Problem

This sub-section is the load-bearing one. Course 1 taught you that the sandbox *contains* the agent to protect the host: the agent runs inside a container, its filesystem access is limited, its network is restricted. That model is correct for a coding agent. It is *inverted* for an offensive agent.

## The inversion

An offensive agent *must reach out* to attack the target. If you contain it the way Course 1 contains a coding agent, it cannot do its job. But if you remove containment entirely, a compromised agent (via prompt injection from target output, S01.3) can attack *you* — lateral movement into your host, exfiltration of your engagement credentials, pivot into your internal network.

This is the sandbox inversion: the offensive sandbox must let the agent reach out to declared targets while preventing the agent (if compromised) from reaching anywhere else, and preventing any inbound foothold from propagating back to the host.

## The dual containment model

The resolution is **dual containment** — two independent containment boundaries, each enforcing a different property:

1. **Outbound scope enforcement.** The agent may initiate connections only to declared in-scope targets. This is S01.2's scope enforcement, but applied at the network layer (firewall rules, egress proxy), not just the middleware. The middleware checks intent ("the model wants to call nmap on X"); the network layer enforces outcome ("even if the middleware is bypassed, the packet cannot leave for X"). Both are required.

2. **Inbound isolation.** Nothing the agent does on the target can establish a foothold back into the host. The agent's container has no inbound routes from the target network. Reverse shells, callbacks, and C2 channels that the agent sets up as part of the engagement must terminate inside the container's isolated segment, not on the host.

```
+---------------------------------------------------------------+
|  HOST (developer machine / CI runner)                         |
|                                                               |
|   +---------------------+        +------------------------+   |
|   | Agent container     |  egress | Egress proxy / FW     |   |
|   | (isolated network   |------->| allows ONLY declared  |   |
|   |  namespace)         |         | in-scope targets      |   |
|   |                     |<-------+                        |   |
|   | NO inbound routes   |         +------------------------+   |
|   | from target net     |                 |                  |
|   +---------------------+                 |                  |
|              ^                            v                  |
|              | (control plane only,       +--------+         |
|              |  over a separate           | TARGET |         |
|              |  mgmt channel)             | (scope)|         |
|              +----------------------------+        +---------+|
+---------------------------------------------------------------+
```

The two boundaries are independent by design. If the agent is compromised via prompt injection and attempts to scan the developer's internal network (192.168.x.x), the outbound boundary blocks it — the internal network is not in scope. If the agent sets up a reverse shell as part of a legitimate engagement technique, the inbound boundary ensures that shell terminates in the container, not on the host.

## Credential quarantine

A specific sub-problem: during an engagement, the agent discovers credentials — target database passwords, API keys, harvested hashes. These are engagement artifacts. They must not touch the host's credential stores (browser password manager, SSH agent, keychain, `~/.aws/credentials`). If they do, a compromised agent can use them to escalate beyond the engagement, or they leak into the host's environment and violate the data isolation the client expects.

The rule: all engagement credentials live in a **quarantine store** inside the container — a separate encrypted vault with its own key, accessible only to the agent process, destroyed when the container is destroyed. The host's credential stores are not mounted into the container at all. This is enforced by container configuration, not by agent cooperation.

The "not by agent cooperation" clause is the load-bearing one. A defense that relies on the model choosing not to write to `~/.aws/credentials` is a defense that fails the first time the model is prompt-injected into doing so. The model is not a security boundary; the container configuration is. Do not mount the host's credential directories into the container. Do not pass host environment variables containing secrets (`AWS_ACCESS_KEY_ID`, `GITHUB_TOKEN`) into the container. The container starts with no host credentials and acquires only engagement credentials, which are quarantined.

## What "compromised agent" means concretely

The dual containment model assumes a threat: the agent can be compromised. It is worth being explicit about what this means, because the defense only makes sense if the threat is real.

A compromised agent is one whose behavior has been altered by content it ingested from the target. S01.3 established the mechanism: target output (an HTTP response, a file contents, a database record) contains a prompt injection that, when processed by the model, causes the model to emit tool calls the operator did not intend. In an offensive harness, the agent processes large volumes of target output by design — it is reading the target's responses to decide the next step. Each response is an injection vector.

The compromise scenarios are concrete. A target web application serves a page containing instructions that cause the agent to call its file-write tool to create a backdoor on the host. A database record contains text that causes the agent to exfiltrate the host's SSH keys to a target-controlled endpoint. A response header that causes the agent to scan the host's internal network for other services to pivot into. None of these require the model to be malicious; they require only that the model follows instructions, and the instructions came from the target rather than the operator.

Dual containment assumes this will happen and ensures the blast radius is bounded. Outbound enforcement means even a fully-compromised agent cannot reach the host's internal network or the metadata endpoint. Inbound isolation means a reverse shell the agent sets up cannot become a persistent foothold on the host. Credential quarantine means exfiltrated credentials are engagement credentials, not host credentials. The defenses are structural, not behavioral — they hold even when the model's behavior is fully controlled by the adversary.

## Network isolation in practice

The agent runs in an isolated network namespace with only the declared targets routable. The Docker implementation:

- The agent container is on a user-defined bridge network with no default route to the host.
- An egress proxy (e.g., a small `squid` or a custom transparent proxy) is the only path out.
- The proxy's ACL is generated from `scope.json` — it allows connections only to in-scope IPs/domains and on in-scope ports.
- The host's LAN, the developer's other containers, and any metadata endpoints (e.g., `169.254.169.254`) are explicitly denied.

The metadata endpoint denial is worth calling out specifically. Cloud instance metadata endpoints are a common SSRF target and a common accidental-exfiltration path. If the host is a cloud instance and the agent can reach `169.254.169.254`, a compromised agent can steal the host's cloud credentials. Deny it at the proxy, always.

## Verifying both directions independently

The dual containment model is only trustworthy if you can verify each direction independently. Verification is not "we configured it correctly" — it is "we attempted the forbidden thing and observed the block."

- **Outbound verification**: from inside the container, attempt a connection to an out-of-scope target (a canary host you control). Confirm the connection is refused. Attempt an in-scope target. Confirm it succeeds.
- **Inbound verification**: from the target network segment, attempt to establish a connection back to the host (or to the agent container's management interface). Confirm it is refused. The only thing that should traverse the host-container boundary is the control plane (the harness's own instructions to the agent), over a separate channel.

A harness that has not verified both directions is making an untested safety claim. The lab in Phase 3 makes this concrete.

---

# S03.4 — Report Generation as a Harness Output

The deliverable is not the attack. The client cannot use a foothold; they cannot invoice a reverse shell. The deliverable is the report. A red team engagement that ends with "we got Domain Admin, here are the screenshots" is an incomplete engagement. The same engagement that ends with a structured report — findings, evidence, CVSS scores, remediation mapped to controls — is complete.

Report generation is therefore a first-class harness output, not an afterthought. The report is generated from structured data accumulated through the engagement, and the generation pipeline is part of the harness design from the start.

## Template-driven, finding-by-finding

The report is generated from the structured findings accumulated in the attack graph and evidence chain (S02.3). Each finding is a record; the report is the rendered view of those records. The structure is fixed; the content is model-generated from structured inputs.

```typescript
interface Finding {
  id: string;
  title: string;
  severity: "critical" | "high" | "medium" | "low" | "informational";
  cvss_draft: string;              // VECTOR_STRING — see warning below
  cvss_auto_scored: boolean;       // ALWAYS true from the harness
  description: string;             // model-written from structured data
  affected_assets: string[];
  evidence_refs: string[];         // evidence record IDs — clickable in HTML
  recreation_steps: string[];      // step-by-step reproduction
  remediation: RemediationRecord;
  cwe: string;                     // e.g., "CWE-79"
  discovered_via_node: string;     // attack graph node ID
}

interface RemediationRecord {
  short_term: string;              // immediate mitigation
  long_term: string;               // structural fix
  owasp_control: string;           // mapped OWASP ASVS / Top 10 control
  effort: "low" | "medium" | "high";
}
```

Every finding links back to evidence records and to the attack graph node that discovered it. This is what makes the report auditable: a client (or their auditor) can trace each finding to the exact tool call that produced it, with the scope authorization for that call.

The linkage is not decorative. A finding without evidence is an assertion; a finding with evidence is a demonstrated fact. A client who receives a finding that says "remote code execution on the billing server" can either trust the assertion or demand proof. A finding that links to evidence record `ev-047` — which contains the exact HTTP request, the exact response showing command output, the timestamp, the scope authorization — is a finding the client cannot dispute. The report's credibility is a function of its evidence density, and the harness accumulates that density automatically through S02.3's evidence chain.

## What gets reported and what does not

Not every succeeded node becomes a finding. A discovery technique (T1046 Network Service Discovery) that succeeds is operational progress, not a client-facing vulnerability. The report distinguishes:

- **Vulnerability findings**: nodes where the technique exploited a weakness the client can remediate (T1190 on an unpatched service, T1110 against a weak password policy, T1552 where credentials were stored insecurely). These become findings with CWE mappings and remediation.
- **Operational achievements**: nodes where the technique succeeded but does not represent a remediable weakness (successful recon, successful lateral movement using valid credentials the client already issued). These appear in the narrative as attack-path demonstration but do not get individual finding entries with remediation.
- **Failed attempts**: nodes that failed. These do not appear in the client-facing report directly, but they remain in the evidence chain and the internal engagement record. A failed attempt that generated significant noise (many requests, triggered detection) may appear as an operational note: "the harness attempted X, which was detected; this confirms the client's detection coverage for X."

The classification is part of the report pipeline — the harness tags each succeeded node as vulnerability-finding or operational-achievement based on the technique and the result.

## CVSS scoring: a draft, not a verdict

This is the most important caveat in this sub-section. CVSS scoring is a known weakness of LLMs. The model will produce a vector string that looks plausible, with attack vector, complexity, privileges required, user interaction — all filled in. The problem is that the model frequently gets these wrong in ways that matter: it overstates exploitability when the description contains scary words, understates it when the exploit path is subtle, and produces scores that disagree with human analysts by 2+ points on the 10-point scale with regularity.

The harness's policy: CVSS scores are **always** generated as drafts, clearly labeled as auto-scored, and routed to human review before they reach the client. The report renders them with a visible "DRAFT — pending analyst review" marker. The model is not the final authority on severity; the human analyst is.

The corollary: do not train the model's confidence on its own CVSS output. If the harness lets the model see its previous CVSS scores as "ground truth," the errors compound. The CVSS draft is a single-shot suggestion from structured data, reviewed by a human, and the human's adjusted score is what goes in the final report.

## Executive summary generation

The executive summary is model-written narrative from structured findings. It is the one place in the report where prose matters more than structure — the C-suite reads the summary; the engineers read the findings. The summary is generated last, after all findings are triaged and scored, because it summarizes the finding set as a whole:

- Overall posture assessment (one paragraph).
- Top 3 findings by severity, in business language (not CVE IDs).
- Aggregate metrics (findings by severity, attack paths demonstrated, time-to-domain-admin if achieved).
- Recommended strategic priorities (derive from remediation records).

The anti-pattern is the **generic executive summary** — a paragraph of security boilerplate that could describe any engagement. The cure is to ground the summary in the specific finding set: name the actual top finding, cite the actual attack path, quote the actual metric. If the summary could be swapped with another engagement's summary without detection, it failed.

## Remediation recommendation engine

For each finding, the report includes remediation mapped to CWE and OWASP controls. This mapping is lookup-driven, not generated: the harness maintains a mapping table from CWE to OWASP ASVS control to remediation template, and the report renders the mapped entry. The model fills in the *target-specific* remediation detail ("patch nginx to version X.Y"), but the *control framework mapping* is deterministic.

This split matters because the control mapping is what the client's compliance team checks. If the model hallucinates an OWASP control reference, the compliance team rejects the report. A deterministic lookup cannot hallucinate.

## Client-ready output: HTML, PDF, JSON

The report is rendered to three formats from the same structured data:

- **HTML**: the primary deliverable. Interactive — evidence references are clickable, findings are navigable, CVSS drafts are visually flagged.
- **PDF**: for clients who need a static, signable artifact. Rendered from the HTML via a headless browser.
- **JSON**: the machine-readable export. Used by the client's ticketing system to create remediation tickets automatically, and by the harness itself for cross-engagement analytics.

All three are generated from the same finding records — there is one source of truth, three renderings. A harness that maintains separate "report data" from "engagement data" has two sources of truth, and they will drift.

---

## Anti-Patterns

### The linear-plan lock-in
The harness has a 10-step plan. Step 3 fails. The harness retries step 3 with the same technique, or aborts the whole engagement. It cannot substitute an alternative technique or promote the plan to a graph. Cure: the attack graph as primary state (S03.1) and the plan correction loop (S03.2).

### The single-boundary sandbox
The offensive harness runs in a standard container with outbound access to "the internet." One boundary — the middleware scope check. A prompt injection bypasses the middleware, and the agent scans the developer's internal network. Cure: dual containment — outbound scope enforcement at the network layer plus inbound isolation (S03.3).

### The credential leak
The agent discovers target database credentials and writes them to `~/.pgpass` for convenience. They are now in the host's credential store, accessible to every process on the host, and they survive the engagement. Cure: credential quarantine — a separate encrypted vault inside the container, destroyed on container teardown.

### The hallucinated CVSS
The report ships with a CVSS 9.8 Critical on a finding that is actually a 6.5 Medium. The model was confident; no human reviewed it. The client disputes the finding, the engagement credibility drops, and the compliance team flags every subsequent score. Cure: CVSS is always a draft, always human-reviewed, always rendered with a visible "pending analyst review" marker (S03.4).

### The generic executive summary
The summary reads: "The engagement identified several security issues of varying severity. The organization should prioritize remediation based on risk." It could describe any engagement. Cure: ground the summary in the specific finding set — name the top finding, cite the attack path, quote the metric.

### The unverified sandbox
The harness is "configured for dual containment" but no one ever attempted a forbidden connection to confirm it was blocked. The first time a compromised agent tries to reach the host's metadata endpoint, it succeeds. Cure: verify both containment directions independently, every engagement (S03.3, Lab 3).

---

## Key Terms

| Term | Definition |
| --- | --- |
| **Attack graph** | The harness's primary state structure for multi-stage engagements: nodes are ATT&CK techniques with per-node states, edges are dependency relationships |
| **MITRE ATT&CK** | The canonical taxonomy of adversary tactics and techniques; used as node labels in the attack graph |
| **Plan correction** | RedTeamLLM's four-step loop on exploit failure: detect → re-hypothesize → re-plan → retry with alternative |
| **Exploit substitution** | Selecting the next-best technique from a knowledge base when the primary exploit fails |
| **Generalize vs specialize** | Broad attack pattern (reliable, noisy) vs. target-specific payload (high-impact, fragile); specialize on high-confidence intel, generalize on low-confidence or as fallback |
| **Sandbox inversion** | Course 1 contains the agent to protect the host; an offensive agent must reach out to the target while still preventing lateral movement if compromised |
| **Dual containment** | Two independent boundaries: outbound scope enforcement (network layer) + inbound isolation (no routes from target to host) |
| **Credential quarantine** | Engagement-discovered credentials live in a separate encrypted vault inside the container; never touch host credential stores |
| **CVSS draft** | Auto-generated severity score, always labeled as pending human review; LLMs are known to mis-score |

---

## Lab Exercise

See `07-lab-spec.md`. Four labs: (1) implement an attack graph as the primary state structure and run a 10-step engagement; (2) simulate an exploit failure at step 3 and verify plan correction selects an alternative path; (3) configure dual containment in Docker and verify both directions independently; (4) build a report generator that produces a client-ready HTML report from structured findings.

---

## References

1. **MITRE ATT&CK** — tactics, techniques, and procedures taxonomy; the canonical state space for the attack graph (S03.1).
2. **RedTeamLLM** (Brown et al., 2024) — the plan correction mechanism: detect failure, re-hypothesize, re-plan, retry (S03.2).
3. **S00.2** — evidence obligations; the report is the client-facing view of the evidence chain.
4. **S01.1** — the offensive state machine; S03.1 builds the attack graph that instantiates it for multi-stage operations.
5. **S01.2** — scope enforcement middleware; S03.3 extends it to the network layer for dual containment.
6. **S01.3** — prompt injection from target output; the inbound isolation boundary exists because of this threat model.
7. **S02.1** — engagement memory; the attack graph is the multi-stage generalization of the flat target-state store.
8. **S02.3** — the evidence chain; the report's evidence references link back to these records.
9. **OWASP ASVS / Top 10** — the control framework for deterministic remediation mapping.
10. **CWE** — common weakness enumeration; the canonical weakness taxonomy for finding classification.