Module S06 — Secure Code Review Harnesses

Secure Code Review Harnesses

Layered pipeline, false positive triage, semantic codebase memory, and approval-gated autofix.

minutes

artifacts

sub-sections

A layered pipeline that turns pull requests into verified, prioritized security findings — deterministic scanners for recall, an LLM triage layer for precision, semantic codebase memory for cross-file taint flows, and an approval-gated autofix loop that never merges without a human.

Key Claims

Load-Bearing Claims

Deterministic first, LLM last. AST, Semgrep, and CodeQL give recall cheaply and without hallucination; the LLM gives precision by judging exploitability on the filtered candidate set.

The false positive problem is an LLM triage problem. Raw scanners at scale are ~40% precision. LLM triage with cross-file context and a feedback loop shifts the curve to ~92% precision.

Single-file analysis misses cross-file taint flows. Semantic codebase memory — function-level embeddings plus the static call graph — retrieves source, sink, and sanitizer across file boundaries.

Approval is non-negotiable. LLM patches can introduce new vulnerabilities while fixing the reported one. Draft PR, human merge, always.

After This Module

Design a layered code review pipeline (AST → Semgrep → CodeQL → LLM → synthesis) and defend the ordering — deterministic first, LLM last.

Implement LLM-powered false-positive triage with confidence scoring and a human-feedback loop, and measure precision/recall/F1 against a labeled dataset.

Build a semantic codebase index (function-level vector embeddings) that retrieves cross-file taint context under a bounded context budget.

Engineer an autofix loop (patch → verify → approve → merge) and explain why human approval is non-negotiable.

Artifacts

Teaching Document

~4,500 words; 4 sub-sections — architecture, false positives, semantic memory, autofix gate

READ

Diagrams

5 diagrams — pipeline n8n, FP precision/recall, triage prompt assembly, semantic retrieval, autofix loop

READ

Slide Deck

reveal.js slide deck, dark theme, design-system teal

READ