"What is EVMbench, and what does it contribute?"	A benchmark for EVM smart contract security tools (arXiv 2603.04915, OpenAI/Paradigm). 117 vulnerabilities across 40 real-world audited repos, scored in three modes (Detect/Patch/Exploit). Its contribution is NOT a tool — it is the MEASUREMENT METHODOLOGY that makes smart contract harnesses comparable. A harness without a benchmark score is a claim; with EVMbench it is evidence.	course2a::sdd10::recall
"What are the three scoring modes, and why are they separated?"	Detect (does it flag the vuln? recall), Patch (does it produce a fix that passes tests + kills the vuln? remediation), Exploit (does it produce a tx sequence that triggers the vuln on a forked chain? adversarial depth). Separated because they are DISTINCT COMPETENCIES — a composite score would hide that a harness can detect without patching or exploit without detecting.	course2a::sdd10::analysis
"Why does EVMbench use real-world repos instead of synthetic CTF challenges?"	Synthetic challenges are clean — isolated vuln, minimal surrounding code. Real-world repos are messy: inheritance, proxy patterns, integrations. A harness that scores well on synthetic but poorly on real-world has OVER-FIT to the pattern. The 40-repo diversity prevents tuning to one codebase style. Real-world grounding is the validity decision.	course2a::sdd10::analysis
"What makes the Exploit mode honest (trustworthy)?"	EXECUTION-VERIFICATION. 'Produced an exploit' is not self-reported — the transaction sequence is run on a FORKED CHAIN by Foundry/Hardhat, and it either triggers the vuln or it does not. No plausibility judgment. This is the same standard Heimdallr's reconstructions need (SDD-09 build-on).	course2a::sdd10::analysis
"How do you read a three-mode scorecard? Give two example profiles."	Profile A: 90% Detect / 20% Patch / 20% Exploit = a FINDING-FLAGGER (use for triage, not remediation). Profile B: 50% Detect / 30% Patch / 70% Exploit = an ATTACK-RECONSTRUCTOR (needs the vuln pointed at it; deep on confirmed targets). The three scores are INDEPENDENT — the combination defines what the tool is. A composite score hides this.	course2a::sdd10::analysis
"What is data contamination, and why is it the largest benchmark-integrity risk?"	If a harness (or its underlying LLM) was trained on the vulnerable repos or their audit reports, the Detect score measures MEMORIZATION, not capability. The harness recites rather than reasons. Mitigations: hold out a private subset, use repos post-dating the model's training cutoff, or measure on novel vulns. This is EVMbench's integrity surface.	course2a::sdd10::analysis
"What is fork fidelity risk in Exploit mode?"	The Exploit verifier runs on a forked chain (Foundry/Hardhat fork), not live mainnet. If the fork diverges from mainnet behavior (state, block, protocol integration), the Exploit result can be misleading — an exploit that lands on the fork might not on mainnet, or vice versa. Fork fidelity is a surface to govern alongside data contamination.	course2a::sdd10::analysis
"State EVMbench's score on the 12-module rubric. Why are 4 modules n/a?"	30/45 (8 scored, 4 n/a). n/a: Context Mgmt, Memory, Permission, Subagents, Prompt Assembly — EVMbench is a methodology, not an agent. High on Tool Design (5, three-mode), Verification (5, execution-verified), Sandboxing (5, forked-chain). Lower: State (3, no versioning), Observability (4, no cost/precision axes).	course2a::sdd10::analysis
"Name 3 things EVMbench does better than any other evaluation approach."	(1) Three-mode separation forces honesty (Detect/Patch/Exploit are distinct competencies; composite hides tradeoffs). (2) Real-world grounding over synthetic challenges (messiness of actual code; 40-repo diversity prevents over-fitting). (3) Execution-verified Exploit mode (forked chain, not self-report).	course2a::sdd10::recall
"Name 3 axes you would add to the EVMbench scorecard."	(1) Cost axis — X% Detect at what $/10K LOC? 90% at $100 vs $2.31 (Heimdallr) are different tools. (2) Precision/FP axis — run on fixed code, count flags (recall without precision lets 'flag everything' score 100%). (3) Dataset versioning (v1, v2, refresh) to resist gaming and track progress.	course2a::sdd10::application
"Why is the benchmark's neutrality (governance) a concern?"	EVMbench must evaluate all harnesses identically — same code, same verifiers, same scoring. A benchmark that lets one harness tune to its dataset is worthless. The governance concerns are dataset/verifier INTEGRITY and protection from LEAKAGE — the 117 ground-truth fixes/exploits must be curated and shielded so a harness cannot have seen the answers during training.	course2a::sdd10::analysis
"How do you run your own harness against EVMbench?"	Invoke the harness per-vulnerability in each mode. Detect: give vulnerable code, must flag. Patch: produce fix that passes test suite + regression test. Exploit: produce tx sequence, verifier (Foundry/Hardhat) runs on forked chain. Output: three-mode scorecard, optionally by vulnerability class. Eval cost = harness cost × 117 × 3 + verifier cost.	course2a::sdd10::application
"State EVMbench's Architect's Verdict in one line."	The measurement methodology that makes smart contract harnesses comparable (117 real-world vulns, 40 repos, three-mode Detect/Patch/Exploit scoring with execution-verified exploits); the three-mode separation is the honesty decision; run your harness against it for a comparable scorecard, and add cost, precision, and versioning axes to turn correctness into a deployment decision.	course2a::sdd10::recall