Module SDD-07 — Secure Code Review Stack

Secure Code Review Stack

The layered AppSec pipeline: AST for structure, Semgrep for patterns, CodeQL for data-flow, LLM triage for false-positive suppression, autofix-with-approval for remediation.

minutes

artifacts

sub-sections

The reference secure code review harness, studied as a composition: tree-sitter/AST for structure, Semgrep for fast pattern rules, CodeQL for deep taint-flow analysis, an LLM for semantic false-positive triage, and autofix behind a human review gate. No single tool is sufficient — the stack's value is how each layer compensates for the failure mode of the one before it. The LLM triage is the single highest-leverage addition, because it is what makes the deep tools usable by suppressing the noise.

Key Claims

Load-Bearing Claims

The composition is the product. Semgrep is fast but shallow. CodeQL is deep but noisy. The LLM triages the noise but hallucinates. Autofix without a gate is dangerous. Each layer compensates for the failure mode of the layer before it — the stack's value is the composition, not any single tool.

LLM triage is the highest-leverage addition. CodeQL's 70-90% false-positive rate is why teams abandon it. The LLM suppression layer turns that noise into a signal a human can act on, at cents per finding. This is the economic case for the LLM in the stack.

The hallucination-mitigation constraint is suppress-with-reason, never confirm. The LLM can only reduce the queue (suppress false positives with an explanation); it cannot confirm or act. Every suppression is logged and a sample is human-audited. This bounds the blast radius to filter-only.

The human goes at two gates, both non-optional. The suppression-log sample audit catches hallucinated dismissals; the autofix PR review is the backstop against LLM error and prompt-injection-driven malicious patches. The human is not removed — the loop concentrates human attention on the decisions that require judgment.

After This Module

Build the layered pipeline (AST → Semgrep → CodeQL → LLM triage → autofix-with-approval) and justify why each layer exists.

Distinguish the three deterministic layers (AST, Semgrep, CodeQL) by what each can and cannot reason about.

Explain the false-positive problem and how LLM triage reduces it without introducing the hallucination failure mode.

Defend the autofix-with-approval gate as a non-optional human checkpoint.

Score the stack on the 12-module rubric (45/60) and identify the build-on list (incremental scanning, semantic memory, hallucination audit).

Artifacts

Teaching Document

~3,000 words; 6-phase study of the secure code review stack as the reference AppSec harness — AST/Semgrep/CodeQL/LLM-triage/autofix pipeline, the false-positive economy, suppress-with-reason constraint, double human gate, scored 45/60

READ

Diagrams

4 diagrams — the five-layer pipeline architecture, Semgrep vs CodeQL vs LLM comparison, the false-positive economy (why LLM triage exists), where the human goes (the double gate)