# Module S12 — Advanced Smart Contract Harnesses

**Course**: 2A — Building AI Harnesses for Cybersecurity
**Module**: S12 — Advanced Smart Contract Harnesses
**Duration**: 60 minutes
**Level**: Senior Engineer and above
**Prerequisites**: S11 complete; the three-mode EVM audit harness (Detect/Patch/Exploit) is operational.

> *The audit harness is operational. This module makes it measurable, portable across chains, and deliverable. Three sub-sections: benchmarking against EVMbench's three scores, Solana and cross-chain security, and the audit as a client-ready product.*

---

## Learning Objectives

1. Configure an audit harness to run against an EVMbench subset and produce a scored results table across Detect, Patch, and Exploit modes.
2. Explain the 92% vs 34% gap between purpose-built agents and general LLMs on DeFi-specific vulnerability classes.
3. Adapt the harness for Solana's account model and identify the distinct vulnerability classes it creates.
4. Describe cross-chain bridge attack surfaces and the harness adaptations required for bridge security review.
5. Generate a client-ready audit report matching the structure and severity classification published by Trail of Bits, Consensys Diligence, and OpenZeppelin.

---

# S12.1 — Benchmarking Against EVMbench

EVMbench is the standard for measuring smart contract audit harnesses. It is not enough to claim "our harness finds bugs" — the claim must be scored against the same 117 vulnerabilities, across the same 40 repositories, using the same three-mode evaluation. EVMbench makes smart contract audit performance comparable, reproducible, and falsifiable.

## The three scores that matter

EVMbench scores an agent across three modes, each producing an independent score:

1. **Detect recall.** Of the 117 known vulnerabilities, how many did the harness find? A harness that finds 100 of 117 has 85% Detect recall. This is the most commonly reported score but the least complete — detection without exploitation or patching is a hypothesis, not a delivered result.

2. **Patch quality.** For each detected vulnerability, did the harness generate a patch that (a) removes the vulnerability and (b) preserves all intended behavior? Patch quality is not binary — a patch can remove the bug but break a test, or fix the specific PoC but leave the underlying flaw (the trap from S11.4). EVMbench scores patch quality against both criteria.

3. **Exploit success rate.** For each detected vulnerability, did the harness construct a working PoC that runs on a forked mainnet and successfully exploits the target? This is the hardest score. A harness can detect a vulnerability it cannot exploit — meaning it identified the pattern but cannot construct the attack transaction.

| Score | What it measures | What it does not measure |
| --- | --- | --- |
| Detect recall | Can the harness find known bugs? | Whether it can exploit or fix them |
| Patch quality | Can the harness fix bugs without breaking behavior? | Whether it can find them in the first place |
| Exploit success rate | Can the harness build working PoCs? | Whether the PoC generalizes to unseen bugs |

A harness that scores 90% Detect, 30% Patch, 30% Exploit is a detection engine, not an audit harness. A harness that scores 80% across all three is a genuine audit tool. The three scores must be reported together — reporting only Detect recall hides weaknesses in the Patch and Exploit pipelines.

### How each score is actually computed

The three scores are not abstractions — each has a concrete evaluation procedure, and a senior engineer building a harness needs to know where the measurement can go wrong.

**Detect recall** is the fraction of the 117 known vulnerabilities the harness flags. A vulnerability is "detected" when the harness emits a finding that names the vulnerable location (file + function or contract + line range) and the vulnerability class. The grading is tolerant on the *narrative* (the harness's prose description can be imprecise) but strict on the *location* — a finding that describes a reentrancy in `Vault.deposit` but points at the wrong line is a false positive, not a true positive. This is where naive harnesses inflate their own recall: they emit broad findings ("there is a reentrancy risk in this contract") and claim credit when any reentrancy exists. EVMbench's grader requires the location to match within a bounded range; a finding that points at the whole contract is not a detection. Build the harness's finding schema to require a line range, not a contract name, and the recall number becomes meaningful.

**Patch quality** is the harder score, and the one most often glossed over. For each detected vulnerability, the harness generates a patch; EVMbench scores the patch on two criteria, both of which must hold:

1. *Vulnerability removed* — the original exploit PoC no longer succeeds against the patched code. This is verified by re-running the S11.3 PoC against the patched contract on a forked mainnet. If the PoC still drains the vault, the patch failed criterion 1.
2. *Behavior preserved* — the protocol's own test suite (the repository's `forge test` or `hardhat test` suite) still passes against the patched code. This catches the S11.4 trap: a patch that blocks the exploit by disabling the vulnerable function entirely satisfies criterion 1 but breaks the protocol's intended behavior. It is not a fix; it is a denial of service.

A patch that satisfies criterion 1 but not criterion 2 is the most common Patch failure mode. The harness "fixes" the bug by adding a `require(false)` to the vulnerable path, the PoC stops working, and the harness claims a successful patch — but the protocol is broken. EVMbench's dual criterion makes this failure visible. When you tune the harness, optimize against criterion 2 specifically; criterion 1 is the easy half.

**Exploit success rate** is the hardest score because it requires the harness to *construct* an attack, not just describe one. The harness must emit a runnable PoC — a Foundry test or a script — that, when executed against a forked mainnet at the vulnerable block, performs the exploit and verifies the post-condition (the vault balance dropped, the attacker balance rose, the governance proposal executed). A PoC that "would work in theory" but does not compile, or compiles but reverts, is a failure. The score is binary per vulnerability: the PoC runs and the post-condition holds, or it does not.

The structural gap between Detect and Exploit is where most harness engineering effort goes. A harness detects a flash loan vulnerability by pattern-matching the price-feed dependency; it exploits it by constructing a multi-step transaction that borrows, manipulates the price, calls the vulnerable function, and repays — all within a single block, with the right fork block number and the right oracle state. The detection is a hypothesis; the exploit is a proof. EVMbench separates them because a harness that can do the former but not the latter is useful for triage but not for verification.

## The 92% vs 34% gap: what domain specificity buys

Purpose-built smart contract agents achieve approximately **92% detection** on DeFi-specific vulnerability classes. General-purpose frontier LLMs (e.g., GPT-5.1 used as a baseline) achieve approximately **34%** on the same classes. This gap is not a marginal improvement — it is the difference between a tool an audit team can rely on and a tool that misses two-thirds of DeFi bugs.

The gap has a structural explanation. DeFi-specific vulnerability classes — flash loan manipulation, oracle dependency, rebase token accounting, governance execution paths, liquidation logic — require domain knowledge that general LLMs lack. A general LLM can identify a reentrancy pattern (syntactic) but struggles with "this protocol assumes the spot price is honest, but a flash loan can manipulate it in one transaction" (semantic, economic). Purpose-built agents embed this domain knowledge via:

- **Heuristic scaffolds** (Heimdallr's approach) that encode DeFi-specific safety properties
- **Tool integration** with Slither, Mythril, Foundry — the LLM reasons over structured tool output, not raw source
- **Context engineering** (function-level reorganization) that presents the right code slice for the vulnerability class
- **Cascaded verification** that filters false positives specific to DeFi patterns

The benchmark gap is the quantified value of these engineering choices. When a team considers building a smart contract audit harness, the 92% vs 34% gap is the business case: a purpose-built harness finds nearly three times as many DeFi bugs as pointing a general LLM at the code.

## Benchmark as product differentiator

EVMbench results are not just an internal metric — they are a marketing artifact. An audit firm that publishes "our harness scores 88% Detect / 72% Patch / 68% Exploit on EVMbench" makes a falsifiable claim that competitors can compare against. This is how audit quality becomes legible to clients who cannot evaluate Solidity security themselves. The benchmark converts "trust us, we're good" into "here is the scored evidence."

The lab for this section configures the S11 harness to run against an EVMbench subset and produces a scored results table — the same table that would appear in a product pitch or a competitive comparison.

## Worked example: benchmarking against an EVMbench subset

Running the full 117-vulnerability benchmark is expensive (each Exploit mode run forks mainnet per target). For iteration during harness development, subset benchmarking is the standard practice: select a representative slice — say, the 12 flash-loan and 8 reentrancy vulnerabilities — run the three modes, and score against that subset. The subset scores are not the headline number, but they are the number you tune against, because they give you feedback in minutes instead of hours.

The harness configuration selects the subset by vulnerability class and runs each mode with its own scoring harness:

```python
# Subset selection: all flash-loan and reentrancy cases from EVMbench
SUBSET = [v for v in EVMBENCH_VULNERABILITIES
          if v.class_ in ("flash_loan", "reentrancy")]  # 20 cases

def run_benchmark_subset(harness, subset=SUBSET) -> ScoreCard:
    detect_results, patch_results, exploit_results = [], [], []

    for vuln in subset:
        repo = checkout(vuln.repo, vuln.commit)  # vulnerable commit

        # --- Detect mode ---
        findings = harness.detect(repo)
        hit = any(f.matches(vuln.location, vuln.class_) for f in findings)
        detect_results.append((vuln.id, hit, findings))

        if not hit:
            patch_results.append((vuln.id, None))   # cannot patch undetected
            exploit_results.append((vuln.id, None))
            continue

        # --- Patch mode (only on detected vulns) ---
        patched = harness.patch(repo, vuln)
        pox_still_works = run_poc(vuln.poc, patched)        # criterion 1: removed?
        tests_still_pass = run_test_suite(patched)          # criterion 2: preserved?
        patch_results.append((vuln.id, pox_still_works and tests_still_pass))

        # --- Exploit mode ---
        poc = harness.exploit(repo, vuln)
        exploit_ok = poc is not None and run_poc(poc, repo)  # PoC runs + post-condition holds
        exploit_results.append((vuln.id, exploit_ok))

    return ScoreCard(
        detect_recall=sum(r[1] for r in detect_results) / len(subset),
        patch_quality=mean_true(patch_results),     # over detected only
        exploit_success=mean_true(exploit_results), # over detected only
        per_vuln=tabulate(detect_results, patch_results, exploit_results),
    )
```

The scoring function encodes the three-score contract exactly: Detect runs on all cases; Patch and Exploit run only on cases Detect hit (you cannot patch or exploit what you did not detect, and scoring those as failures would conflate Detect recall with Patch quality). The `matches` predicate on the Detect line is the load-bearing grading rule — it is what stops a broad finding from claiming credit, as described above.

A realistic subset run on the 20-case flash-loan/reentrancy slice might produce:

| Metric | Score | Detail |
| --- | --- | --- |
| Detect recall | 16/20 = 80% | Missed 2 reentrancies in inherited base contracts (context window); missed 2 flash-loan paths that required cross-protocol oracle reasoning |
| Patch quality | 11/16 = 69% | 3 patches broke the test suite (over-restrictive guards); 2 patches left the PoC working (patched the symptom, not the root cause) |
| Exploit success | 9/16 = 56% | 4 detected flash-loan vulnerabilities the harness could describe but not construct a working multi-step attack for |

Reading this table is where harness development happens. The 80% Detect is decent; the 69% Patch points at the S11.4 over-restriction trap; the 56% Exploit points at the PoC-construction gap. The next iteration targets the Exploit gap specifically — adding a flash-loan PoC template scaffold that pre-builds the borrow/manipulate/repay transaction structure, so the LLM fills in the protocol-specific calls rather than constructing the whole transaction from scratch. Re-run the subset; if Exploit rises to 12/16 and Patch holds, the change worked. This tight loop — subset benchmark, read the table, target the weakest score, re-benchmark — is how the three-score methodology is actually used. The full 117-case run is the validation pass before a release; the subset run is the daily driver.

---

# S12.2 — Solana and Cross-Chain Considerations

The EVM is not the only chain. Solana's account model, the Anchor framework, and cross-chain bridges create distinct vulnerability classes that require harness adaptations. The harness architecture (Detect/Patch/Exploit, cascaded verification) ports; the tools and heuristics do not.

## Solana's account model and its vulnerability classes

Solana differs from the EVM at the execution-model level. The EVM has a single global state tree keyed by contract address; Solana has accounts that are explicitly passed into each transaction instruction. This creates a class of vulnerabilities that has no EVM analog:

| Solana vulnerability class | Description | EVM analog |
| --- | --- | --- |
| **Account confusion** | A program accepts an account of the wrong type (e.g., a token account instead of the expected vault account) because it does not verify the account's owner or type | None — EVM contracts address-reference is type-bound |
| **Missing signer check** | A program performs a privileged action without verifying the caller signed the transaction | Missing `onlyOwner` — but Solana signer checks are explicit per-account |
| **Integer arithmetic errors** | Solana/Rust defaults to checked arithmetic, but `unchecked` blocks or casting can introduce overflows | Integer overflow — same class, different defaults |
| **PDA misuse** | Program-Derived Addresses (deterministic addresses controlled by the program) are mis-derived, collision-attacked, or used without verifying the seeds | None — no EVM equivalent |

Account confusion is the signature Solana bug. An Anchor program receives accounts as parameters; if it does not check that an account is owned by the expected program (or is the expected type), an attacker can pass a malicious account that satisfies the struct layout but behaves differently. The defense is Anchor's `#[account]` constraints and explicit owner checks — but these must be applied correctly, and missing them is a common audit finding.

## The Anchor framework and its security implications

Anchor is Solana's dominant smart-contract framework — analogous to what Hardhat/Foundry are to Solidity. Anchor provides:

- **Account validation macros** (`#[account]`, `#[derive(Accounts)]`) that generate boilerplate owner/type checks
- **IDL generation** for client-side integration
- **A constraint system** (`has_one`, `constraint`, `address`) for declarative account validation

Anchor reduces account-confusion bugs when used correctly, but introduces its own footguns: a `#[derive(Accounts)]` struct that omits a `has_one` constraint is a silent security gap. The harness checks Anchor programs by static-analyzing the `#[derive(Accounts)]` blocks for missing constraints — the Solana equivalent of Slither's access-control detectors.

### Anchor vulnerability patterns, concretely

The three patterns below are the ones a Solana harness must detect, with the specific Anchor construct each breaks. These are drawn from published audit findings (Neodyme, OtterSec, Sec3) and from the Solana Sealevel Attacks repository; they are not theoretical.

**1. Missing `has_one` on a cross-account reference.** A vault program stores a `user` field on the `Vault` account that should match the `user` account passed into the instruction. Without `has_one = user`, the instruction accepts any `user` account — including the attacker's — and the vault operates on behalf of the wrong user. The vulnerable Anchor struct:

```rust
#[derive(Accounts)]
pub struct Withdraw<'info> {
    #[account(mut, has_one = authority)]  // checks authority matches — good
    pub vault: Account<'info, Vault>,
    #[account(mut)]
    pub user_token_account: Account<'info, TokenAccount>,
    pub authority: Signer<'info>,
    // BUG: no has_one linking user_token_account to vault.user
    // an attacker passes their own token account; vault.authority is checked,
    // but funds are sent to the attacker's account, not the vault owner's
}
```

The harness detects this by parsing the `#[derive(Accounts)]` struct, resolving the `Vault` data structure's fields, and flagging any `TokenAccount` or `Account` field that is marked `mut` but has no `has_one` or `constraint` linking it to a field on another account in the struct. This is a structural rule — it does not require LLM reasoning; it is the Solana analog of Slither's `suicidal` and `arbitrary-send` detectors. The LLM's role is to confirm the finding is exploitable in the specific instruction context (does the instruction actually transfer to `user_token_account`?).

**2. Signer check omitted on a privileged instruction.** An `admin_set_fee` instruction accepts a `fee_authority` account but does not mark it `Signer<'info>`. Any caller can invoke it and set the fee. The fix is `fee_authority: Signer<'info>` plus a `has_one = fee_authority` on the config account. The harness detects this by checking that every instruction handler in the program has at least one `Signer` in its `Accounts` struct *and* that the signer is bound to a field on the account being mutated. An instruction with no signer at all is flagged; an instruction with a signer that is not bound to the mutated account is also flagged (the signer is decorative).

**3. PDA seed collision or missing seed verification.** A program derives an account address with `Pubkey::find_program_address(&[b"vault", user.key.as_ref()], program_id)` but the instruction handler does not verify that the passed `vault` account's address matches this derivation. An attacker passes a different account at the same struct type; the program operates on a forged vault. The fix is Anchor's `seeds` and `bump` constraints on the account:

```rust
#[account(
    seeds = [b"vault", user.key().as_ref()],
    bump,
    has_one = user,
)]
pub vault: Account<'info, Vault>,
```

The harness detects missing PDA verification by checking that every `Account` whose address *should* be a PDA (inferred from the program's `init` logic elsewhere) carries `seeds` + `bump` in every instruction that reads or writes it. This requires cross-instruction reasoning — the `init` instruction establishes the PDA derivation; every other instruction must enforce it. This is the Solana equivalent of S11.2's cross-function invariant checking, and it is where a pure-Slither-style pattern matcher fails and the LLM's contextual reasoning earns its keep.

### What the Solana harness outputs

The Solana finding schema mirrors the EVM one (location, class, severity, PoC, patch) but the location is a program ID + instruction name + account struct field, and the PoC is a `solana-bankrun` test, not a Foundry test. The patch is an Anchor constraint addition — concrete, reviewable, one PR. The severity classification uses the same Critical/High/Medium/Low scale, with the same rule: missing `has_one` on a fund-transfer path is Critical (funds at immediate risk); missing `has_one` on a non-transfer path is High; a missing signer on a non-fund instruction is Medium. The harness's severity is a draft; the auditor confirms.

## Cross-chain bridge security

Cross-chain bridges are the highest-value attack surface in crypto. Bridges lock assets on one chain and mint representations on another; compromising the bridge's verification logic lets an attacker mint unbacked assets. The 2022 Wormhole hack ($325M), the 2022 Nomad hack ($190M), and the 2024-2025 bridge exploits collectively caused billions in losses.

Bridge security is a distinct discipline because the vulnerability is often not in a single contract but in the **cross-chain verification protocol**: how does Chain B know that Chain A actually locked the assets? Common failure modes:

- **Signature verification flaws** — the bridge accepts invalid signatures (wrong signer set, replayable signatures)
- **Message-passing manipulation** — the bridge trusts a message format that an attacker can forge
- **Relayer centralization** — a single relayer can submit fraudulent messages
- **Wrapped-asset accounting** — minting more wrapped assets than the locked collateral

The harness adapts for bridge audits: it models the cross-chain message flow as a state machine (the bridge protocol, not just one contract), traces asset flows across both chains, and checks the verification invariants ("minted amount <= locked amount"). This is beyond single-contract auditing — it requires modeling the protocol as a distributed system.

## Harness adaptations for Solana

The three-mode architecture ports. The tools change:

- **Static analysis**: Slither and Mythril do not work on Solana. The harness uses Solana-specific tools — `solana-security-txt`, Anchor's `anchor test` with fuzzing, and custom Semgrep rules for Rust/Anchor patterns.
- **Exploit PoCs**: instead of Foundry forked mainnet, the harness uses Solana's `bankrun` or `solana-test-validator` with a mainnet snapshot. The PoC structure (setup → attack → verify) is identical.
- **Invariant extraction**: the same strategy applies — extract the protocol's intended invariants and check them. Solana invariants look different ("the vault token account's balance equals the sum of user deposits") but the method is the same.

---

# S12.3 — Smart Contract Audit as a Deliverable

The audit harness produces findings; the deliverable is the audit report. A finding that lives only in a tool's JSON output is not a deliverable — it is raw material. The audit report is what the client reads, what the remediation team acts on, and what becomes part of the protocol's security record. This section covers the report's structure, severity classification, and the format top firms publish.

## Report structure

A client-ready smart contract audit report has a consistent structure across top firms (Trail of Bits, Consensys Diligence, OpenZeppelin). The harness's report generator targets this structure precisely — not a generic "security report," but the specific section ordering and field set these firms publish. Clients and compliance reviewers recognize the format on sight, which is itself a form of trust: a report that looks like a Trail of Bits report is easier to accept than one that looks bespoke.

1. **Scope.** Which contracts were audited, at which commit hash, over what dates. The scope defines what was reviewed and, by omission, what was not. This is the legal and operational boundary of the audit. Trail of Bits publishes this as a table of contract path + commit hash + line count; Consensys includes the Solidity compiler version and any in-scope dependencies. The harness populates this from the `AuditScope` record (the checkout commit and the file list passed to Detect mode) — it is deterministic, not model-generated.

2. **Methodology.** How the audit was conducted — manual review, static analysis tools used, fuzzing, formal verification. This establishes that the audit was rigorous and reproducible. The harness contributes its tool list (Slither version, Mythril version, the LLM model identifier and version) and the three-mode description. The human auditor adds the manual-review coverage statement ("the lead auditor manually reviewed the vault and router contracts"). This is the section where the harness's contribution is explicitly bounded — it says what the machine did, so the human's contribution is legible.

3. **Findings table.** A summary table listing every finding: ID, title, severity, status (fixed/open), location. This is the executive view — the client's first stop. Both Trail of Bits and Consensys render this as a table sorted by severity (Critical first), with the finding ID as a link to the detailed entry below. The harness generates this table directly from the structured findings, sorted by the human-confirmed severity.

4. **Severity breakdown.** A count of findings by severity (Critical, High, Medium, Low, Informational). This gives a quick risk posture read. Consensys renders this as a simple count table; Trail of Bits often adds a sentence of narrative ("The engagement identified one critical and three high-severity findings, all related to the liquidation logic."). The harness produces the count; the narrative is model-drafted and human-edited.

5. **Detailed findings.** For each finding: description, impact, proof of concept (the exploit from S11.3), recommendation (the patch from S11.4), and status. This is the section the remediation team works from. The per-finding structure is where the firm formats converge most tightly:

   ```
   ### [S-01] Title (Severity)
   **Location**: contracts/Vault.sol:142-167
   **Description**: What the code does and why it is vulnerable.
   **Impact**: What an attacker can achieve, with a concrete scenario.
   **Proof of Concept**: The Foundry test or script (code block).
   **Recommendation**: The patch (code block), with explanation.
   **Status**: Fixed in commit abc123 / Open
   ```

   The harness populates every field from structured data: Location from the finding's line range, Description and Impact from the LLM triage (S11.1), Proof of Concept from the Exploit mode PoC (S11.3), Recommendation from the Patch mode output (S11.4), Status from the issue tracker linkage. The auditor reviews each field, edits prose where the model was imprecise, and confirms the patch code compiles and the PoC runs.

6. **Remediation roadmap.** Prioritized action items — which findings to fix first, which are acceptable risk, and the verification path for each fix. This is the section that converts findings into a project plan. The harness drafts it by sorting findings by severity and dependency (a Critical that blocks a High's fix comes first), but the prioritization judgment is the auditor's — the harness cannot know that the client ships next week and therefore must accept the Medium risk to meet the date.

## Severity classification

| Severity | Definition | Example |
| --- | --- | --- |
| **Critical** | Funds at immediate risk. Exploitable now, without special conditions. | Reentrancy that drains the vault; unchecked mint |
| **High** | Funds at risk under conditions. Exploitable but requires a specific state or trigger. | Flash loan path that requires specific oracle state |
| **Medium** | Logic error without immediate financial impact. May cause incorrect behavior but not direct loss. | Reward calculation off by a basis point |
| **Low** | Best-practice violation. No direct risk but indicates code quality issues. | Missing event emission; non-standard pattern |
| **Informational** | Optimization or style suggestion. No security relevance. | Gas optimization; naming convention |

The severity is the finding's most consequential attribute — it drives the remediation priority and the client's risk communication. The harness assigns severity via the LLM triage step (S11.1), but every severity is human-confirmed before the report ships. The harness's severity is a draft; the auditor's is final.

## Automated report generation from structured findings

The harness produces findings as structured records (the output of S11's cascaded verification). The report generator converts these records into the report format:

```python
class AuditReportGenerator:
    def generate(self, findings: list[Finding], scope: AuditScope) -> AuditReport:
        return AuditReport(
            scope=scope,
            methodology=self.describe_methodology(),
            findings_table=self.findings_table(findings),
            severity_breakdown=self.severity_breakdown(findings),
            detailed_findings=[
                self.detailed_finding(f) for f in findings
            ],
            remediation_roadmap=self.remediation_roadmap(findings),
        )

    def detailed_finding(self, finding: Finding) -> DetailedFinding:
        return DetailedFinding(
            id=finding.id,
            title=finding.title,
            severity=finding.severity,
            description=finding.description,
            impact=finding.impact,
            poc=finding.exploit_poc,        # from S11.3
            recommendation=finding.patch,    # from S11.4
            status=finding.status,
        )
```

The generator's output is a structured document (JSON, Markdown, or HTML) that matches the firm-standard format. The human auditor reviews, edits, and finalizes — the generator does the assembly, the auditor does the judgment.

## The audit-as-product positioning

An AI-assisted smart contract audit is a product. The client pays for: the scope (which contracts), the methodology (what was checked), the findings (what was found), and the remediation support (how to fix). The benchmark scores (S12.1) are the quality proof; the report (this section) is the deliverable; the remediation support is the ongoing relationship.

Pricing an AI-assisted audit differs from a traditional manual audit. The marginal cost of running the harness is low (Heimdallr's $2.31/10K LOC), but the value delivered is high (92% detection). The pricing model captures the gap between cost and value — the client pays for the finding quality, not the compute time. This is the audit-as-product positioning: the harness is the engine, the report is the product, the benchmark scores are the proof, and the remediation support is the retention.

---

## Anti-Patterns

### Reporting only Detect recall

Publishing "92% detection" without Patch or Exploit scores. Hides that the harness cannot exploit or fix what it finds. Cure: report all three EVMbench scores together.

### Porting the EVM harness to Solana unchanged

Running Slither against Solana code. Slither does not analyze Rust/Anchor. Cure: adapt the tools — Solana-specific static analysis, `bankrun` for PoCs, Anchor-constraint checking.

### Shipping the harness's severity without human confirmation

The LLM assigns severity; the report ships it. A Medium that should be Critical misleads the client's remediation priority. Cure: every severity is human-confirmed before the report ships.

### The tool-dump report

Shipping raw Slither/Mythril output as the audit report. No triage, no PoCs, no remediation. Cure: the structured report (this section), generated from verified findings with PoCs and patches.

### The bespoke report format

The harness emits a report in a custom structure that no client or compliance reviewer recognizes. The format itself becomes a friction point — the client's remediation team has to learn the layout before they can act on the findings. Cure: match the Trail of Bits / Consensys section structure and per-finding field set exactly. Recognizability is a feature; clients trust a report that looks like the ones they have seen before.

### Patching the symptom, not the root cause

A patch that blocks the specific PoC (criterion 1) but leaves the underlying flaw exploitable via a different path. EVMbench's dual criterion catches the `require(false)` version of this trap but not the subtler one — a patch that closes the detected attack vector while leaving a sibling vector open. Cure: the Patch mode output must include a re-analysis step that re-runs Detect against the patched code to confirm the vulnerability *class* is gone, not just the specific PoC. This is the Patch-mode analog of S11.4's cascaded verification.

---

## Key Terms

| Term | Definition |
| --- | --- |
| **EVMbench three scores** | Detect recall, Patch quality, Exploit success rate — measured independently; Detect runs on all cases, Patch and Exploit run on detected only |
| **Patch dual criterion** | A patch must both remove the vulnerability (PoC fails) and preserve behavior (test suite passes); satisfying only the first is the over-restriction trap |
| **92% vs 34% gap** | Purpose-built agents vs general LLMs (GPT-5.1) on DeFi-specific vulnerability classes |
| **Account confusion** | Solana vulnerability: program accepts wrong account type without owner/type verification |
| **Missing `has_one`** | Anchor vulnerability: a `mut` account in the `Accounts` struct has no constraint linking it to a field on another account; the signature Solana audit finding |
| **PDA misuse** | Program-Derived Address mis-derivation or collision attack — Solana-specific; requires cross-instruction reasoning to detect |
| **Anchor framework** | Solana's dominant smart-contract framework; `#[derive(Accounts)]` constraints |
| **Bridge verification protocol** | How Chain B knows Chain A locked assets; the cross-chain security boundary |
| **Audit report structure** | Scope, methodology, findings table, severity breakdown, detailed findings, remediation roadmap — matches Trail of Bits / Consensys format |
| **Severity classification** | Critical (immediate risk), High (conditional risk), Medium (logic error), Low (best practice), Informational |

---

## Lab Exercise

See `07-lab-spec.md`. Three labs: (1) configure the harness against an EVMbench subset and produce a scored results table; (2) run a Solana security harness against a known-vulnerable Anchor program; (3) generate a client-ready audit report from Pillar 4 findings.

---

## References

1. **EVMbench** (arXiv 2603.04915) — the three-mode benchmark; 117 vulnerabilities, 40 repositories. The scoring methodology (Detect recall over all cases; Patch and Exploit over detected only) follows the paper's evaluation protocol.
2. **Heimdallr** (arXiv 2601.17833) — purpose-built agent architecture; the 92.45% detection anchor and the $2.31/10K LOC cost model.
3. **Anchor framework** — Solana smart-contract framework; account validation macros (`#[account]`, `#[derive(Accounts)]`, `has_one`, `seeds`, `bump`).
4. **Solana Sealevel Attacks** (coral-xyz/sealevel-attacks) — the canonical catalog of Solana vulnerability patterns; the `has_one`, signer-check, and PDA examples are drawn from this.
5. **Neodyme, OtterSec, Sec3** — published Solana audit reports; the source of the Anchor vulnerability patterns cited in S12.2.
6. **Trail of Bits / Consensys Diligence / OpenZeppelin** — published audit reports as the format reference for the report structure in S12.3.
7. **Wormhole, Nomad** — cross-chain bridge exploits; the bridge security case studies.
8. **S11** — the three-mode audit harness this module benchmarks, ports, and delivers. S11.3 is the PoC source for the report's proof-of-concept field; S11.4 is the patch source for the recommendation field.
9. **Capstone 1 / Capstone 2** — where the audit harness is applied to a full engagement.