# Lab Specification — Module B11: Governance and Compliance

**Course**: Course 2B — Securing & Attacking Harnesses and LLMs
**Module**: B11 — Governance and Compliance
**Duration**: 60–75 minutes
**Environment**: Python 3.10+. No GPU. No external API calls. A text editor, `pytest`, and the standard library. This lab builds the **three governance artifacts** a regulator or auditor asks for first: an AI BOM generator, a policy-as-code engine, and a tamper-evident audit-trail writer.

---

## Learning objectives

By the end of this lab you will have built:

1. **An AI BOM generator** that reads a sample agent's model version, tool registry, dependencies, and config, and emits a machine-readable AI Bill of Materials — the inventory that makes an agent auditable (the precondition for every other governance artifact).
2. **A policy-as-code engine** that evaluates agent actions against a constitution-derived policy set, with default-deny, policy-aware redaction, and an audit entry emitted for every evaluation — the governance-to-engineering bridge realized in code (cf. DD-20 IronCurtain deterministic compilation and DD-09 NemoClaw governance-beneath-the-agent).
3. **A tamper-evident audit-trail writer** with append-only semantics and hash-chaining — the evidence store that proves controls are enforced, not just documented (the compliance artifact EU AI Act Art 12 and HIPAA § 164.312(b) require).

The lab is the governance layer. A red-team or build team without these three artifacts is a team operating without a control plane a governance review can read.

---

## Phase 0 — Setup (3 min)

```bash
mkdir b11-governance-lab && cd b11-governance-lab
python3 -m venv .venv && source .venv/bin/activate
# No third-party dependencies for the core lab — standard library only.
# Optional: pytest for running the test cases.
pip install pytest
```

No GPU, no network. This is a governance and code lab. Create the file structure:

```
b11-governance-lab/
├── ai_bom.py          # Phase 1 — AI BOM generator
├── policy_engine.py   # Phase 2 — policy-as-code engine
├── audit_trail.py     # Phase 3 — tamper-evident audit trail
├── constitution.md    # Phase 2 input — the plain-English policy
├── sample_agent.json  # Phase 1 input — a sample agent definition
└── tests/             # test cases for each phase
```

---

## Phase 1 — The AI BOM generator (20 min)

Build `ai_bom.py` — a generator that reads a sample agent's definition and emits an AI Bill of Materials in machine-readable JSON.

### 1.1 The sample agent definition

Create `sample_agent.json` representing a realistic deployed agent:

```json
{
  "agent_id": "support-triage-agent",
  "model": {
    "provider": "anthropic",
    "model_id": "claude-opus-4-1",
    "checkpoint": "claude-opus-4-1-20260605",
    "modality": ["text"],
    "license": "anthropic-api-tos"
  },
  "training_data_sources": [
    {"name": "internal-tickets-2024", "provenance": "internal", "license": "proprietary", "pii": true},
    {"name": "public-support-kb", "provenance": "internal-curation", "license": "cc-by-4.0", "pii": false}
  ],
  "system_prompt": {
    "version": "v3.2",
    "hash": "sha256:9f2a...",
    "guardrail_version": "nemoguardrails-0.10.0"
  },
  "tools": [
    {"name": "query_prod_db", "type": "mcp", "endpoint": "mcp://db.internal", "capability_scope": "read", "risk_tier": "high"},
    {"name": "send_email", "type": "mcp", "endpoint": "mcp://mail.internal", "capability_scope": "write", "risk_tier": "high"},
    {"name": "run_code", "type": "builtin", "endpoint": "sandbox.local", "capability_scope": "exec", "risk_tier": "low"}
  ],
  "frameworks": [
    {"name": "langgraph", "version": "0.2.40"},
    {"name": "anthropic-sdk", "version": "0.30.0"}
  ],
  "external_services": [
    {"name": "anthropic-api", "type": "llm-provider"},
    {"name": "pgvector", "type": "vector-db"}
  ]
}
```

### 1.2 The generator spec

```python
from __future__ import annotations
from typing import Any

def generate_ai_bom(agent: dict[str, Any]) -> dict[str, Any]:
    """Generate an AI Bill of Materials from an agent definition.

    The AI BOM extends the NTIA minimum SBOM elements with the AI-specific
    component classes: model, training data, system prompt/config.
    The output must be machine-readable JSON, generated from the running
    system's state (not maintained by hand — a wiki and reality drift).
    """
    # Return a dict with keys:
    #   bom_version, generated_at (UTC ISO 8601), agent_id,
    #   components: {model, training_data_sources, system_prompt,
    #                tools, frameworks, external_services}
    # Each component carries the fields needed for auditability:
    #   - model: provider, id, checkpoint, modality, license
    #   - training_data_sources: name, provenance, license, pii flag
    #   - system_prompt: version, hash, guardrail_version
    #   - tools: name, type, endpoint, capability_scope, risk_tier
    #   - frameworks: name, version (the SBOM subset)
    #   - external_services: name, type
    pass
```

### 1.3 Your task

Implement `generate_ai_bom`. Then:

- Run it on `sample_agent.json` and write the output to `ai_bom_output.json`.
- Validate the JSON (`python3 -m json.tool ai_bom_output.json`).
- Add a function `verify_ai_bom(bom: dict) -> list[str]` that checks completeness and returns a list of warnings (e.g., a tool with no `risk_tier`, a model with no `checkpoint`, a training-data source with `pii: true` and no documented handling). An empty list means the BOM is complete.

### 1.4 The point

An agent without an AI BOM cannot be audited: a vulnerability in a dependency cannot be traced, a model-version dispute cannot be resolved, a compliance assertion cannot be evidenced. The generator — not a wiki — is the source of truth, because reality drifts and the auditor reads reality. This BOM is the input to B4's supply-chain response and B12's assessment.

---

## Phase 2 — The policy-as-code engine (25 min)

Build `policy_engine.py` — the engine that evaluates agent actions against a constitution-derived policy set. This is the governance-to-engineering bridge.

### 2.1 The constitution

Create `constitution.md` — the plain-English policy a governance council would own (the Govern output):

```markdown
# Agent Governance Constitution

## P-001: Production data requires human approval
No agent may access production data without human approval.

## P-002: High-risk tool calls require a second reviewer
Any tool call classified as high-risk requires a second human reviewer.

## P-003: Sandboxed execution is permitted
Code execution within the sandbox is permitted without approval.

## P-004: No tool outside the AI BOM
An agent may not call any tool not listed in its current AI BOM.
(Supply-chain control: an unlisted tool is an unaudited dependency.)
```

### 2.2 The engine spec

Implement the policy engine from the teaching document, extended with:

- **Default-deny**: an action no rule matches is DENIED with reason "no policy matched; default-deny."
- **Policy-aware redaction**: production-data surfaces log argument keys, not values (B0 retention discipline).
- **AI BOM integration**: rule P-004 DENIES any tool not in the AI BOM (the Phase 1 output is a constructor argument).
- **Audit emission**: every evaluation (including default-deny) emits an `AuditEntry` to the sink.

```python
from __future__ import annotations
from dataclasses import dataclass, asdict
from datetime import datetime, timezone
from enum import Enum
from typing import Any, Callable

class Decision(str, Enum):
    ALLOW = "ALLOW"
    DENY = "DENY"
    ESCALATE = "ESCALATE"

@dataclass(frozen=True)
class AgentAction:
    agent_id: str
    model_version: str
    tool: str
    arguments: dict[str, Any]
    surface: str          # 'production_data' | 'internal_api' | 'sandbox' | ...
    risk_tier: str        # 'low' | 'medium' | 'high'

@dataclass(frozen=True)
class PolicyRule:
    policy_id: str
    description: str
    matches: Callable[[AgentAction], bool]
    decide: Callable[[AgentAction], Decision]

@dataclass(frozen=True)
class AuditEntry:
    timestamp: str
    policy_id: str
    agent_id: str
    model_version: str
    action_summary: dict[str, Any]   # redacted
    decision: str
    reason: str

class PolicyEngine:
    def __init__(self, rules: list[PolicyRule], audit_sink: Callable[[AuditEntry], None]) -> None:
        self._rules = rules
        self._audit = audit_sink

    def evaluate(self, action: AgentAction) -> tuple[Decision, str]:
        # Iterate rules; first match wins.
        # If a rule matches, evaluate decide(), emit audit, return.
        # If no rule matches, default-deny, emit audit, return.
        pass

    @staticmethod
    def _redact(action: AgentAction) -> dict[str, Any]:
        # Production-data surfaces: log arg keys only (B0 retention discipline).
        # Other surfaces: log full arguments.
        pass
```

### 2.3 Implement the four rules from the constitution

Compile each `P-00X` from `constitution.md` into a `PolicyRule`. For P-004 (AI BOM integration), the engine needs the BOM's tool list — pass it as a constructor argument:

```python
class PolicyEngine:
    def __init__(self, rules, audit_sink, ai_bom_tools: set[str] | None = None) -> None:
        ...
        self._ai_bom_tools = ai_bom_tools or set()
```

P-004's `matches` returns True if `action.tool not in self._ai_bom_tools`; its `decide` returns `Decision.DENY`.

### 2.4 Test cases

Write at least six test cases in `tests/test_policy.py`:

```python
# 1. sandbox run_code (low risk, in BOM) -> ALLOW (P-003)
# 2. query_prod_db (production_data, high) -> ESCALATE (P-001)
# 3. send_email (high risk) -> ESCALATE (P-002)
# 4. a tool NOT in the AI BOM -> DENY (P-004)
# 5. an action no rule matches -> DENY (default-deny)
# 6. production-data action: verify the audit entry has arg KEYS not VALUES (redaction)
```

For each, assert both the Decision and that an `AuditEntry` was emitted (use a list as the audit sink and inspect it). Print the decisions and the audit entries.

### 2.5 The point

This engine is the governance-to-engineering bridge. Three properties matter: **default-deny** (missing policy = safety), **audit on every evaluation** (evidence the control fired), **policy-aware redaction** (B0 retention discipline at the audit layer). The engine sits in the harness execution path — NOT the agent process (DD-09 NemoClaw): if the agent can reach it, a prompt injection can disable it. The rules are deterministic if/then (DD-20 IronCurtain): no LLM judgment at runtime.

---

## Phase 3 — The tamper-evident audit-trail writer (15 min)

Build `audit_trail.py` — the append-only, hash-chained store that proves controls are enforced. This is the compliance artifact EU AI Act Article 12 and HIPAA § 164.312(b) require.

### 3.1 The spec

```python
from __future__ import annotations
import hashlib
import json
from datetime import datetime, timezone
from pathlib import Path

class AuditTrail:
    """Append-only, hash-chained audit trail.

    Properties required for compliance:
    - Append-only: no entry modified or deleted once written.
    - Hash-chained: each entry includes hash of the previous entry's
      canonical JSON, so tampering breaks the chain and is detectable.
    - Complete: every event is logged (not sampled).
    - Tamper-evident: verify_chain() detects any modification.
    """
    def __init__(self, path: Path) -> None:
        self._path = path
        self._prev_hash: str | None = None  # loaded from the last line if the file exists
        # On init, if the file exists, load the last entry's hash as _prev_hash.

    def append(self, entry: dict) -> str:
        """Append an entry. Returns the entry's hash.
        Each entry gets: timestamp (UTC ISO 8601), prev_hash (chain link),
        and entry_hash (SHA-256 of canonical JSON of {timestamp, prev_hash, entry})."""
        pass

    def verify_chain(self) -> bool:
        """Walk the file and verify every hash link. Return True if intact,
        False if any entry was modified or removed (chain broken)."""
        pass

    def __len__(self) -> int:
        pass
```

### 3.2 Implement it

Key implementation details:

- **Canonical JSON**: serialize entries with `json.dumps(entry, sort_keys=True, separators=(',', ':'))` so the hash is deterministic (key order must not affect the hash).
- **Chain link**: each entry stores `prev_hash` (the `entry_hash` of the previous entry, or `None` for the first). The `entry_hash` is `sha256(canonical_json({timestamp, prev_hash, payload}))`.
- **Append-only**: open the file in append mode (`'a'`); never rewrite. Each entry is one JSON line (JSON Lines format) for easy streaming and verification.
- **Verify**: read every line, recompute each entry's hash, check it matches the stored `entry_hash`, and check each `prev_hash` matches the previous entry's `entry_hash`. Any mismatch returns `False`.

### 3.3 Test cases

```python
# 1. Append 5 entries; verify_chain() returns True.
# 2. Manually edit one entry in the file (simulate tampering); verify_chain() returns False.
# 3. Confirm append-only: attempting to "modify" an entry by re-writing breaks the chain.
# 4. Confirm completeness: every PolicyEngine evaluation in Phase 2 produces exactly one entry.
```

### 3.4 The point

An audit trail you can edit is not an audit trail — it is a draft. The hash chain makes tamper detectable: change one byte in the middle and every subsequent `prev_hash` link breaks. This is the integrity property a regulator requires. Combined with completeness (every event logged) and retention (a defined period), it satisfies the logging requirements of EU AI Act Article 12 and HIPAA § 164.312(b). The observability dashboard (B8, sampled) and this trail (complete) are separate systems serving different audiences.

---

## Phase 4 — Integration: wire the three together (10 min)

Now connect the three components into one governance pipeline:

1. Generate the AI BOM from `sample_agent.json` (Phase 1).
2. Construct the `PolicyEngine` with the four rules, passing the BOM's tool names as `ai_bom_tools` (Phase 2).
3. Construct an `AuditTrail` at `audit.jsonl` and use its `append` method as the engine's audit sink (Phase 3), wrapped to convert `AuditEntry` → dict.
4. Evaluate a sequence of actions (the six test cases from Phase 2) and confirm each produces a decision AND a hash-chained audit entry.

```python
# Sketch of the integration
from ai_bom import generate_ai_bom
from policy_engine import PolicyEngine, RULES
from audit_trail import AuditTrail

bom = generate_ai_bom(agent)
bom_tools = {t["name"] for t in bom["components"]["tools"]}
trail = AuditTrail(Path("audit.jsonl"))

def sink(entry):  # AuditEntry -> dict -> trail
    trail.append({
        "policy_id": entry.policy_id,
        "agent_id": entry.agent_id,
        "model_version": entry.model_version,
        "action_summary": entry.action_summary,
        "decision": entry.decision,
        "reason": entry.reason,
    })

engine = PolicyEngine(RULES, sink, ai_bom_tools=bom_tools)
# Evaluate actions...
print("Audit trail intact:", trail.verify_chain())
print("Entries written:", len(trail))
```

### 4.1 The end-to-end test

After running the integration:

- `audit.jsonl` exists and contains one entry per evaluation.
- `trail.verify_chain()` returns `True` (no tampering).
- Each entry's `action_summary` for production-data actions shows arg keys, not values (redaction works).
- The AI BOM tool `send_email` is allowed to be proposed (it's in the BOM); a non-BOM tool is DENIED by P-004.

---

## Phase 5 — Stretch (optional, 10 min)

If time remains:

1. **Approval-state check**: extend P-001 so that if an `approval` dict is attached to the action (with `approver` and `timestamp`), the rule returns ALLOW instead of ESCALATE — but only if the approval is fresh (e.g., < 10 minutes old). Add a test for an expired approval.
2. **RMF mapping generator**: write a function that, given the AI BOM and the audit trail, emits a markdown report mapping the agent's controls to the four RMF functions (Govern/Map/Measure/Manage), with evidence pointers (e.g., "Measure: injection success rate 4% over 500 attempts, audit entries AT-7841–AT-8340"). This is the artifact you'd hand a governance reviewer.
3. **SBOM integration**: if `pip-audit` or `cyclonedx-py` is available, extend the AI BOM generator to read the actual dependency list from a `requirements.txt` or the environment, merging it with the `frameworks` field.

---

## Deliverables

- `ai_bom.py` — the AI BOM generator + `verify_ai_bom` (Phase 1)
- `ai_bom_output.json` — the generated BOM for the sample agent (Phase 1)
- `policy_engine.py` — the policy-as-code engine with default-deny, redaction, BOM integration (Phase 2)
- `constitution.md` — the plain-English policy (Phase 2 input)
- `audit_trail.py` — the append-only, hash-chained writer + `verify_chain` (Phase 3)
- `audit.jsonl` — the hash-chained audit trail produced by the integration (Phase 4)
- `tests/` — the test cases for each phase

## Success criteria

- [ ] `generate_ai_bom` produces valid JSON with all six component classes; `verify_ai_bom` returns no warnings for the sample agent.
- [ ] The policy engine returns the correct Decision for all six test cases, including default-deny.
- [ ] The audit entry for a production-data action shows arg KEYS, not VALUES (redaction works).
- [ ] A tool not in the AI BOM is DENIED by P-004.
- [ ] `AuditTrail.append` writes hash-chained entries; `verify_chain()` returns True for an intact trail and False after tampering.
- [ ] The Phase 4 integration runs end-to-end: every `engine.evaluate` produces one audit entry; the chain verifies.
- [ ] Every artifact ties back to a specific principle from the teaching document (AI BOM = auditable inventory; audit trail = enforcement evidence; policy-as-code = governance-to-engineering bridge).

## What this lab is NOT

- It is not a red-team. No injection, no jailbreak. The lab builds the **governance layer** the red-team's findings are reported against.
- It is not a production system. The audit trail's hash chain is tamper-*evident*, not tamper-*proof* (a sophisticated attacker with file write access could rewrite the whole file). Production uses WORM storage, hardware roots of trust, or an external notary. The principle is what matters here.
- It does not call any real model API. The `model_version` is a string field, not a live call. The governance layer is model-agnostic.