Module B12 — Harness Security Assessments as a Service

Harness Security Assessments as a Service

The capstone methodology module. B0–B11 are techniques, controls, and frameworks; B12 is the operational layer that turns them into a scoped, priced, reported, retestable engagement. The six-phase methodology (PTES / NIST SP 800-115 adapted) — Scoping, Reconnaissance, Discovery, Exploitation/Validation, Reporting, Retesting — with each phase tied to the module that supplies its content. The buyer is the CISO / AI security lead, and the deliverable is the scored artifact B9's checklist executor produces, packaged into a five-section report a board can read.

60
minutes
8
artifacts
3
sub-sections
Every prior module gave you a technique, a control, or a framework. None of them gave you an engagement. A CISO who hires you does not want a technique — they want a defensible answer to 'is my agent safe to ship, and what is the residual risk?' That answer is an artifact, produced by a method, bounded by a scope, priced as a service. This module turns B0–B11 into that artifact. The six phases are a loop, not a pipeline: a retest feeds the next engagement's scoping. Run B9's checklist and hand over the output is half an engagement — scoping sets the legal plane, reporting makes it a deliverable, retesting measures the residual. B12 introduces no new technique; it introduces the operational layer that makes B0–B11 a profession.
Key Claims
Load-Bearing Claims

The six-phase methodology is a loop, not a pipeline — and only two phases are the testing B9/B10 supply. Scoping (B0 legal plane → SOW + scope file), Reconnaissance (B1 surface map), Discovery (B9 checklist + B10 chains), Exploitation/Validation (B0 minimum-proof), Reporting (B9 output packaged), Retesting (B0.2 residual-risk). An engagement that runs only the testing phases is 'run B9 and hand over the output' — half an engagement, illegal without scope, unsellable without a report, unverifiable without a retest. The retest feeds back into scoping for the next engagement; that loop is the recurring value.

Scoping is where B0's legal control plane gets written into the engagement contract, and the four-point enumeration is what makes it complete. Surfaces (B1 template), pinned model versions, provider authorizations (B0's three conditions — ToS / waiver / self-hosted; fail all three and the surface is out of scope), and exclusions. The most common scoping mistake is a client who says 'jailbreak everything' without checking their provider's terms — the deployer cannot authorize what the provider forbids, and the provider_authorization check runs at scoping, not at testing. The SOW carries seven clauses, three of them AI-specific (provider authorization, dual-use/disclosure, DMCA § 1201 waiver).

The engagement report is the same scored artifact B9's checklist executor produces, packaged into five sections for a CISO. B9's 8 PASS/FAIL + 2 MEASURED rows become findings F-01..F-10; B10's chains become F-11+ that the checklist alone misses; the B9 risk-to-module mapping becomes the control matrix. The report generator enforces the honesty at the output layer — it rejects a finding with no taxonomy reference, a MEASURED finding without a success rate, and a Critical finding without a remediation route. The strictness is what prevents the '10/10 PASS' lie a client may demand.

Retesting measures residual risk, never binary 'fixed' — and the service's long-term value is comparable artifacts across releases. The verdict is Resolved / Improved / Unchanged / Regressed under identical conditions (same harness, same sampling, same pinned model version). The retest re-runs the FULL B9 checklist and re-attempts the B10 chains because a patch can open a new gap or reroute a chain. The four-layer packaging (methodology, tooling, templates, evidence) is what makes artifacts comparable across releases — the injection-success-rate trend over six releases is what a CISO buys. Drift in any layer breaks the comparability.

After This Module
01
Run a six-phase agent security assessment — Scoping, Reconnaissance, Vulnerability Discovery, Exploitation/Validation, Reporting, Retesting — as a repeatable professional engagement, with each phase tied to the module that supplies its content.
02
Scope and price an agent assessment: enumerate the surfaces (B1), pin the model versions, verify provider authorizations (B0), and weight the cost drivers (number of agents, surface complexity, depth of testing) into a defensible fee.
03
Write an AI-specific Statement of Work incorporating B0's dual-use clause and provider-authorization clause, plus the techniques permitted/prohibited, data-handling rules, and residual-risk measurement protocol.
04
Produce the engagement report — the scored artifact B9's checklist executor produces — with an executive summary, findings (each tagged to OWASP/Microsoft taxonomy, with attack procedure, evidence, residual risk), a control matrix, and a remediation roadmap.
05
Retest a remediated agent using residual-risk measurement rather than binary 'fixed/unfixed,' and produce a before/after retest report with Resolved / Improved / Unchanged / Regressed verdicts.
06
Package the service — methodology, tooling, templates, evidence — into a repeatable assessment practice that produces comparable results across engagements and releases.
Artifacts
01
Teaching Document
~3,900 words prose + report-generator code; 3 sub-sections — six-phase methodology + scoping/pricing (B0/B1/B9/B10 cross-refs, seven-clause SOW), the engagement report (five sections, the scored B9 artifact packaged), retesting + packaging (residual-risk retest, four-layer service stack); with anti-patterns, key terms, references
READ
02
Diagrams
5 Mermaid diagrams — the six-phase methodology loop (phase→module), the scoping-to-SOW flow with B0's seven clauses (3 AI-specific), the five-section report structure with the validation layer, the retest residual-risk before/after with four verdicts, the four-layer service packaging stack
READ
03
Slide Deck
15 slides — reveal.js, dark theme, design-system teal; covers techniques-to-service, six phases, four-point scoping, three pricing drivers, seven-clause SOW, five-section report, finding field set, residual-risk retest, four-layer packaging, capstone synthesis
READ
04
Teaching Script
Verbatim transcript, ~140 wpm, [SLIDE N] cues; ~35 min spoken
READ
05
Flashcards
22 flashcards (TSV) — mix recall and analysis; six phases, four-point enumeration, seven SOW clauses, five report sections, finding field set, retest verdicts, packaging layers, synthesis tracing
TEST
06
Exam
15 exam questions — 20/40/40 Bloom (3 recall / 6 application / 6 analysis); scoping, pricing, SOW clauses, report generator validation, retest verdicts, B9/B10 complementarity, service packaging
TEST
07
Lab Spec
Build the Assessment Report Generator — ingest B9's checklist-executor JSON, emit the five-section report with strict validation (no unclassified findings, no MEASURED without a rate, no Critical without remediation); add the retest-delta comparison (Resolved/Improved/Unchanged/Regressed) with the version-bump guard; write a sample seven-clause SOW. Runnable, type hints, no GPU; ~75-90 min
DO
08
Module Web Page
Single-file HTML hub
HERE