# Module S08 — SDLC Gate Harnesses **Course**: 2A — Building AI Harnesses for Cybersecurity **Module**: S08 — SDLC Gate Harnesses **Duration**: 75 minutes **Level**: Senior Engineer and above **Prerequisites**: S00–S07 complete > *The control plane shift from review-time checks to creation-time guardrails. Multi-scanner orchestration that runs SAST, SCA, secrets, and IaC in parallel, aggregates and deduplicates, and gates the pipeline. Trend analysis that tracks whether the codebase is getting safer. Vulnerability triage at scale using CVE, EPSS, and CISA KEV.* --- ## Learning Objectives 1. Articulate the control plane shift from review-time checks to creation-time guardrails, and design a hard-gate vs soft-gate decision matrix per scan type. 2. Orchestrate multiple scanners (SAST, SCA, secrets, IaC) in parallel, aggregate and deduplicate results, and perform LLM-driven cross-scanner triage under a token budget. 3. Build a trend and risk-scoring system that tracks finding counts over build history, attributes debt per author, and produces a single risk score that gates PR approval. 4. Implement vulnerability triage at scale: ingest CVE feeds, match against dependency manifests, score exploitability with EPSS, and prioritize with CISA KEV. --- # S08.1 — The Control Plane Shift Traditional SDLC security runs scans after the commit and reviews findings days or weeks later. A developer pushes on Tuesday; the SAST report lands in a dashboard on Thursday; the secret leaked in the commit is in git history by Friday. The review-time model assumes a slow, human-paced pipeline. That assumption is now false. ## The AI coding agent problem AI coding agents — Copilot, Cursor, autonomous agents — create code, dependencies, workflows, and infrastructure faster than review-time checks can process. An agent can introduce a new dependency with a transitive vulnerability, scaffold a new IAM role with an over-broad policy, and add a new API endpoint without authz, all in a single session. A review-time gate that runs overnight and surfaces findings in a dashboard the next morning is structurally too slow. By the time the finding is reviewed, the code is in production. The speed asymmetry is the core problem. Creation happens in seconds; review happens in hours or days. The defense must move to the creation side. ## Creation-time guardrails instead of review-time checks The control plane shift: enforce security at creation time, not review time. A creation-time guardrail runs synchronously in the developer's path — pre-commit, pre-merge, in the IDE — and blocks the insecure artifact before it lands. The developer learns immediately that the dependency has a critical CVE, that the IAM policy is public, that the secret is in the diff. The feedback loop is seconds, not days. This is not "shift left" as a slogan. It is a structural change to where the gate lives: | | Review-time check (old) | Creation-time guardrail (new) | | --- | --- | --- | | **When** | Post-commit, post-merge, nightly | Pre-commit, pre-merge, in-IDE | | **Feedback latency** | Hours to days | Seconds | | **Action** | Report to a dashboard | Block the action, inline | | **Developer sees** | A ticket weeks later | The problem at the moment of creation | ## What an SDLC gate harness enforces The harness enforces a set of controls, each mapped to a scan type: - **Secret scanning** (Gitleaks, TruffleHog) — no committed credentials - **SAST** (Semgrep, CodeQL) — no known code weaknesses - **SCA** (OWASP Dependency-Check, Snyk) — no vulnerable dependencies - **IaC policy** (Checkov, OPA) — no insecure infrastructure declarations - **License compliance** — no prohibited licenses in dependencies - **DAST for API changes** — runtime checks when the API surface changes ## Hard gate vs soft gate Not every finding should block. A hard gate blocks the pipeline (the PR cannot merge, the commit is rejected). A soft gate warns (the developer sees the finding but can proceed). The decision matrix per scan type: | Scan type | Default gate | Rationale | | --- | --- | --- | | **Secrets (validated, high confidence)** | Hard block | A leaked credential is an active breach path | | **SAST (critical, high confidence)** | Hard block | A confirmed critical weakness ships an exploit path | | **SCA (CISA KEV or EPSS > 0.7)** | Hard block | Known-exploited or high-probability-of-exploit CVE | | **IaC (public S3 bucket, 0.0.0.0/0)** | Hard block | Public exposure is an immediate risk | | **SAST (medium/low)** | Soft warn | Not every medium finding blocks; surface for review | | **SCA (low EPSS, fix available)** | Soft warn | Triage, schedule, don't block | | **License (unknown/copyleft)** | Soft warn | Legal review, not a security block | The matrix is the policy. It is configurable per repository, per environment (production gates are stricter than staging), and per risk appetite. The principle: block what is immediately dangerous; warn what needs attention but is not immediately dangerous. Over-blocking trains developers to override without reading; under-blocking ships vulnerabilities. The matrix is the calibration. --- # S08.2 — Multi-Scanner Orchestration Running each scanner type in isolation produces siloed findings: Semgrep reports code weaknesses, Snyk reports vulnerable dependencies, Gitleaks reports secrets, Checkov reports IaC issues. Each in its own dashboard, each with its own severity scale, each reviewed (or ignored) separately. The orchestrator unifies them. ## Running scanners in parallel The four scanner types are independent — they analyze different artifacts (code, manifests, diffs, IaC files). They run in parallel: ``` Input (PR diff + repo snapshot + dependency manifests + IaC files) ↓ (parallel) ┌───────────────┬──────────────────┬──────────────────┬─────────────────┐ │ SAST │ SCA │ Secrets │ IaC │ │ Semgrep │ OWASP Dep-Check │ Gitleaks │ Checkov │ │ CodeQL │ Snyk │ TruffleHog │ OPA │ └───────────────┴──────────────────┴──────────────────┴─────────────────┘ ↓ (aggregate) Unified finding list (normalized schema) ``` Parallel execution keeps wall-clock time bounded by the slowest scanner, not the sum. On a large PR, SAST might take 90 seconds, SCA 30 seconds, secrets 10 seconds, IaC 15 seconds. Sequential: 145 seconds. Parallel: 90 seconds. For a creation-time gate, that difference is the difference between a usable guardrail and one developers disable. ## Result aggregation and normalization Each scanner emits findings in its own format with its own severity vocabulary. The orchestrator normalizes to a unified schema: ```typescript interface UnifiedFinding { id: string; scanner: "sast" | "sca" | "secrets" | "iac"; tool: string; // "semgrep" | "snyk" | "gitleaks" | "checkov" severity: "critical" | "high" | "medium" | "low"; cwe?: string; // for SAST cve?: string; // for SCA location: { file: string; line?: number; resource?: string }; description: string; dedup_key: string; // cross-scanner + cross-tool dedup confidence: "high" | "medium" | "low"; } ``` Normalization includes **severity reconciliation** — different scanners use different scales (Semgrep: ERROR/WARNING/INFO; Snyk: critical/high/medium/low; Checkov: FAILED/PASSED). The orchestrator maps each to the unified critical/high/medium/low scale using a documented mapping. The mapping is policy, not improvisation. ## Cross-scanner dedup Findings overlap across scanners. A Semgrep rule and a CodeQL query may flag the same SQL injection. A Snyk finding and an OWASP Dependency-Check finding may report the same CVE in the same dependency. The orchestrator deduplicates via `dedup_key`: - **Within scan type**: Semgrep + CodeQL on the same (file, line, CWE) → one finding. - **Across scan types**: a secret detected by Gitleaks that is also a hardcoded credential flagged by SAST → one finding, cross-referenced. Dedup presents one finding to the developer, not duplicates from each tool. Duplicate findings erode trust — the developer assumes the bot is broken when it reports the same thing three times. ## LLM-powered cross-scanner triage Some findings relate across scan types in ways that simple dedup cannot catch. An SCA finding (a vulnerable library) is only exploitable if the vulnerable code path is reachable in the application — which a SAST scanner can determine. An IaC finding (a public S3 bucket) is only a real exposure if the bucket contains sensitive data — which the application code or a data classification tool can indicate. The LLM triage layer correlates across scanners: ```python async def cross_scanner_triage(findings: list[UnifiedFinding]) -> list[UnifiedFinding]: # Group findings that may relate (same file area, same dependency, same resource) groups = group_related_findings(findings) triaged = [] for group in groups: if len(group) == 1: triaged.append(group[0]) # standalone, no correlation needed continue # Multiple findings may relate — ask the LLM to correlate correlated = await llm_correlate(group) triaged.extend(correlated) return triaged ``` The correlation can upgrade or downgrade severity: an SCA finding upgraded because the vulnerable path is reachable, or an IaC finding downgraded because the exposed resource contains no sensitive data. Cross-scanner correlation is where the orchestrator produces insight no single scanner could. ## Token budget management Multi-scanner contexts are token-heavy. A PR with 40 raw findings across four scanners, each with surrounding context, can blow past a model's context window. The orchestrator prioritizes within the budget: high-severity findings first (they are the ones that block), then medium, then low. Within a severity, findings are ordered by confidence. Low-severity, low-confidence findings may be truncated entirely — they would not block regardless, and the token budget is better spent on the findings that will. --- # S08.3 — Trend Analysis and Risk Scoring A single build's findings are a snapshot. The harness's power over time is trend: is the codebase getting safer or riskier? Trend turns point-in-time findings into a directional signal. ## Build history memory The harness stores finding counts per category per build. Over a sequence of builds, this builds a history: ```typescript interface BuildRecord { build_id: string; timestamp: string; pr_id: string; branch: string; finding_counts: { sast: { critical: number; high: number; medium: number; low: number }; sca: { critical: number; high: number; medium: number; low: number }; secrets: { critical: number; high: number; medium: number; low: number }; iac: { critical: number; high: number; medium: number; low: number }; }; risk_score: number; // computed author: string; } ``` The history is the substrate for trend detection, risk scoring, and attribution. ## Risk trend detection The simplest trend: finding counts over the last N builds. Is the total critical count rising or falling? A rising trend across 10 builds is a stronger signal than any single build's count — it indicates systemic degradation, possibly from a new framework adoption, a team change, or a process breakdown. More sophisticated trend detection weights recent builds more heavily (exponential moving average) and separates "new findings introduced" from "existing findings fixed." A build that introduces 5 critical findings and fixes 7 is net-positive; a build that introduces 5 and fixes 0 is net-negative. Net direction matters more than absolute count. ## Per-author attribution Finding counts attributed per author identify where security debt concentrates. This is for coaching, not punishment — the framing matters. An author who introduces many SAST findings may need a 30-minute session on the common patterns; a team that introduces many IaC findings may need the gate matrix explained. Attribution without coaching is surveillance; coaching without data is guesswork. The harness provides the data. The framing guardrail: attribution data is visible to the author and their lead, not broadcast. It feeds into coaching conversations, not performance reviews. The harness policy should make this explicit — misuse of attribution as a punishment tool destroys the psychological safety that makes the gate effective. ## Risk score per PR The trend history produces a single risk score per PR that can gate approval: ```python def compute_risk_score(build: BuildRecord, history: list[BuildRecord]) -> float: """A single number 0-100 representing this PR's risk contribution.""" # New findings introduced by this PR, weighted by severity new_findings = build.finding_counts - baseline(history) severity_weights = {"critical": 25, "high": 10, "medium": 3, "low": 1} score = sum( count * severity_weights[sev] for scan_type in new_findings.values() for sev, count in scan_type.items() ) # Trend modifier: rising trend adds risk if trend_is_rising(history, window=10): score *= 1.2 return min(score, 100) ``` The risk score gates approval: above a threshold (say 30), the PR requires a security reviewer's explicit approval; above a higher threshold (say 60), the PR is blocked until findings are resolved. The thresholds are policy. The score is computed, consistent, and auditable — not a gut call. --- # S08.4 — Vulnerability Triage at Scale A large application has hundreds of dependencies, each with a CVE history. The NVD publishes thousands of CVEs per year. Dependency scanners match dependencies to CVEs and produce a list — often hundreds of findings for a mature application. Raw, untriaged, the list is overwhelming. The harness triages it to a prioritized remediation queue. ## The triage pipeline ``` CVE feed ingestion (NVD) ↓ Dependency matching (against manifest: package.json, requirements.txt, go.mod) ↓ Exploitability scoring (EPSS — probability of exploitation in 30 days) ↓ Environment relevance (is the vulnerable code path reachable in our app?) ↓ Priority queue (CISA KEV first, then EPSS × reachability × severity) ``` Each stage narrows the list. Matching removes CVEs that do not apply to any dependency. Exploitability scoring removes CVEs that are theoretical. Environment relevance removes CVEs in code paths the application does not use. The priority queue is the output: a ranked list of what to fix first. ## EPSS integration EPSS (Exploit Prediction Scoring System) is a probability score from 0 to 1 representing the likelihood that a CVE will be exploited in the wild in the next 30 days. It is produced by FIRST from real-world exploit data. A CVE with EPSS 0.9 is almost certainly being exploited; a CVE with EPSS 0.01 is almost certainly not, regardless of its CVSS severity. EPSS inverts a common failure mode: prioritizing by CVSS severity alone. A CVSS 9.8 (critical) CVE with EPSS 0.01 is less urgent than a CVSS 7.5 (high) CVE with EPSS 0.8. Severity measures impact if exploited; EPSS measures probability of exploitation. Together they prioritize correctly. ## CISA KEV as a first-priority signal CISA's Known Exploited Vulnerabilities (KEV) catalog is the highest-confidence signal: these are CVEs with confirmed, active exploitation in the wild. A dependency matching a KEV entry is not a theoretical risk — it is a present-tense breach path. KEV findings jump to the top of the priority queue regardless of EPSS or CVSS. The harness checks KEV first and surfaces matching findings for immediate remediation. ## Bug report intake (external signals) Beyond automated CVE matching, the harness ingests external vulnerability reports from bug bounty programs (HackerOne, Bugcrowd) and security researchers. These submissions require triage: - **Parse** the submission (structured: target, vulnerability type, PoC, severity claim). - **Deduplicate** against existing findings (internal scanner findings, prior reports) — duplicate reports are common. - **Detect duplicates** via semantic similarity between the report and known findings. - **Auto-route P1 findings** (critical, confirmed, in scope) to immediate response. The intake pipeline ensures that an external report of a critical vulnerability does not sit in a queue unread. The same triage discipline applied to scanner output applies to human-submitted reports. ## The prioritized remediation list The output is a single prioritized queue: 1. **CISA KEV matches** — remediate immediately (hours). 2. **EPSS > 0.7, reachable in our environment** — remediate urgently (days). 3. **EPSS > 0.7, not confirmed reachable** — verify reachability, then remediate. 4. **High CVSS, low EPSS** — schedule in the normal backlog. 5. **Low CVSS, low EPSS** — accept risk or batch-fix opportunistically. This queue turns "hundreds of CVEs" into "three things to fix this week." That is the harness's value: not finding more vulnerabilities, but making the existing flood actionable. --- ## Anti-Patterns ### Review-time-only gates Scans run post-commit, findings in a dashboard reviewed days later. A leaked secret is in git history by the time anyone sees it. Cure: creation-time guardrails that block synchronously in the developer's path. ### One-gate-fits-all Blocking every finding regardless of severity or exploitability. The pipeline never merges; developers override without reading. Cure: the hard-gate/soft-gate decision matrix — block the immediately dangerous, warn the rest. ### Siloed scanners Each scanner in its own dashboard with its own severity scale. The developer sees three reports of the same SQL injection from three tools and assumes the bot is broken. Cure: the orchestrator normalizes and deduplicates across scanners. ### CVSS-only prioritization Prioritizing CVEs by CVSS severity alone. A CVSS 9.8 with EPSS 0.01 is less urgent than a CVSS 7.5 with EPSS 0.8. Cure: EPSS (exploitation probability) and CISA KEV (confirmed exploitation) must factor into priority, not CVSS alone. ### Attribution as punishment Using per-author finding counts as a performance metric. Destroys psychological safety; developers stop engaging with the gate. Cure: attribution is for coaching conversations, visible to author and lead only, never broadcast or tied to review. --- ## Key Terms | Term | Definition | | --- | --- | | **Creation-time guardrail** | Synchronous security check in the developer's path (pre-commit/pre-merge/IDE); seconds of feedback, not days | | **Hard gate vs soft gate** | Block (cannot merge) vs warn (can proceed); the decision matrix per scan type | | **Multi-scanner orchestration** | SAST, SCA, secrets, IaC run in parallel; results normalized and deduplicated | | **Risk score per PR** | A single computed number from finding counts + trend; gates approval at thresholds | | **EPSS** | Exploit Prediction Scoring System; probability (0-1) of exploitation in next 30 days | | **CISA KEV** | Known Exploited Vulnerabilities catalog; confirmed active exploitation; first-priority signal | --- ## Lab Exercise See `07-lab-spec.md`. Four labs: (1) the gate decision matrix; (2) multi-scanner orchestration with aggregation and dedup; (3) trend analysis and risk scoring with a simulated build history; (4) CVE triage harness with EPSS and KEV. --- ## References 1. **CISA KEV** (Known Exploited Vulnerabilities catalog) — confirmed active exploitation; the first-priority triage signal. 2. **EPSS** (FIRST) — Exploit Prediction Scoring System; probability of exploitation in 30 days. 3. **CVSS v3.1** (FIRST) — severity scoring; measures impact, not exploitation probability. 4. **Semgrep, CodeQL** — SAST; **Snyk, OWASP Dependency-Check** — SCA; **Gitleaks, TruffleHog** — secrets; **Checkov, OPA** — IaC. 5. **NVD** (National Vulnerability Database) — the CVE feed source. 6. **S06.2** — false-positive triage and precision/recall measurement; this module's cross-scanner triage uses the same patterns.