# Lab Specification — Module FT21: HIPAA and BAA Elimination **Course**: Course 3 — LLM Fine-Tuning Masterclass **Module**: FT21 — HIPAA and BAA Elimination **Lab title**: The HIPAA Architecture **Duration**: 60–90 minutes (design and defense — no GPU required) **Environment**: A diagramming tool (Mermaid, Excalidraw, draw.io) and a markdown editor. No code execution. This is an architecture and judgment lab. --- ## Why this lab is a design lab, not a build lab Modules FT20 and earlier gave you the operational capability to serve a model. This lab asks the *upstream* question — *should* you serve it locally or call it via API, and what does each choice obligate you to build around it? Getting this wrong wastes millions (wrong architecture, retrofit) or creates a breach surface (right architecture, missing safeguards). The lab is deliberately artifact-first: you produce an architecture diagram and a written defense, the two documents a real HIPAA-LLM program lives or dies by. By the end you will have a defensible, documented architecture for a clinical-decision-support LLM that handles PHI — the same deliverable a CISO, a compliance officer, and a clinical informatics board would review. --- ## The scenario You are the lead architect for **Meridian Health**, a regional health system running 14 hospitals and ~200 clinics. Meridian wants to deploy a clinical-decision-support (CDS) LLM with three workflows: 1. **Differential suggestions.** A clinician pastes a de-identified case summary (symptoms, labs, history) and the model returns a structured differential diagnosis with cited reasoning patterns. 2. **Note summarization.** The model ingests long discharge summaries and produces a concise structured summary for the care team. 3. **Imaging triage assist (phase 2).** The model receives a wound or skin-lesion photo alongside text and returns a triage suggestion. (Phase 2 — not in initial deploy, but the architecture must not preclude it.) The data is unambiguously PHI: patient identifiers, clinical narratives, dates, and (in phase 2) identifiable body images. Meridian's compliance office has stated plainly: *"We will not deploy anything that routes PHI through a surface our BAA does not provably cover, and we will not sign a BAA we have to police endpoint-by-endpoint for the life of the product."* Your job: design the architecture. Choose local-only or API-under-BAA. Defend the choice. Specify the supporting stack. --- ## Learning objectives By the end of this lab you will have: 1. **Chosen a deployment posture** (one of the four ways to run an LLM on PHI) and defended it against the scenario's constraints — especially the phase-2 imaging workflow. 2. **Specified the serving stack** (if local) — model, quant format, server, VRAM envelope — drawing on FT20. 3. **Specified the de-identification pipeline** for the fine-tuning corpus — tool choice, identifier classes handled, residual-risk handling. 4. **Specified the audit-logging approach** — what is logged, where, retention, who can read it. 5. **Documented the technical safeguards** — encryption, access controls, network boundary. 6. **Produced two deliverables**: an architecture diagram and a written defense. Both at the standard a CISO would sign. --- ## Phase 0 — The decision framework (15 min) Before you draw anything, fill in the decision framework below. This is the structured reasoning that *produces* the architecture choice. Do not skip it — the defense document references it directly. ### The decision matrix Score each deployment option (1–4) against each criterion (1 = weak, 5 = strong). The lowest-scoring criterion is your binding constraint — not the average. | Criterion | Cloud BAA | Dedicated | Self-host (VPC) | On-prem | | --- | --- | --- | --- | --- | | Phase-2 imaging (vision endpoint coverage under BAA) | | | | | | Stateful workflow coverage (note summarization needs context) | | | | | | Endpoint-coverage maintenance burden (life of product) | | | | | | Latency to clinician (sub-2s for CDS) | | | | | | Egress risk (PHI leaving the trust boundary) | | | | | | Operational cost (you own the GPUs vs vendor owns them) | | | | | | Audit-log completeness (you control every touch) | | | | | | **Binding constraint (lowest score across the row)** | | | | | **Questions to answer in your decision document**: 1. Which option is eliminated outright by phase-2 imaging, and why? (Hint: re-read Section 21.3 of the teaching document on vision-endpoint coverage.) 2. Which option is eliminated by the compliance office's "we will not police endpoint-by-endpoint" constraint? 3. Of the surviving options, which has the strongest *evidence* posture — i.e., which lets you *prove* to an auditor that PHI never left a covered path? 4. What is the binding constraint for your final choice, and how does your architecture address it? --- ## Phase 1 — The architecture template (15 min) Use this template as your starting structure. Fill in every box. Mark every boundary where PHI crosses, and every boundary where it does *not*. ``` ARCHITECTURE TEMPLATE — Meridian CDS LLM [ TRUST BOUNDARY: ___________________________ ] Data sources : [ EHR / notes / imaging ] De-identification : [ tool / method / identifier classes ] Fine-tuning corpus : [ PHI-scarce? yes/no; how verified ] Base model : [ which open-weights base; why ] Adapter / steer : [ LoRA config; what behavior steered ] Serving stack : [ server / quant / VRAM envelope (FT20) ] Inference path : [ prompt flow; PHI enters where, leaves where ] Audit logging : [ what / where / retention / access ] Encryption at rest : [ what is encrypted, with what ] Encryption in transit : [ TLS to what endpoint ] Access control : [ RBAC model; who can call the model ] [ END TRUST BOUNDARY ] Crosses the boundary outbound: - [ list everything; PHI should be NONE ] Crosses the boundary inbound: - [ model download, log ship-out, etc. ] ``` Fill in every line. If a line is "N/A," say why. An incomplete template is an incomplete architecture. --- ## Phase 2 — The architecture diagram (20 min) Produce a diagram (Mermaid preferred — match the style of artifact 02). It must show: - [ ] The **trust boundary** (dashed box around everything that touches PHI). - [ ] The **PHI flow** — from data source, through de-identification, into the model, out to the clinician. Use a distinct color for PHI-bearing arrows. - [ ] The **serving stack** components (from FT20): the server (vLLM/TGI/Ollama), the model artifact, the quant format. - [ ] The **de-identification pipeline** as a distinct stage before fine-tuning. - [ ] The **audit-log store** and its relationship to every inference call. - [ ] **Everything that crosses the trust boundary** — and confirmation that none of it carries PHI. - [ ] The **encryption surfaces** (at rest, in transit) annotated. If you chose local: the only outbound crossings should be (a) the one-time model download (no PHI), and (b) optionally the audit-log ship to a SIEM (under BAA, if external). If you chose API-under-BAA: the PHI flow must cross the boundary to the vendor — annotate the BAA chain and the coverage gaps you are accepting. --- ## Phase 3 — The written defense (25 min) Write a 600–900 word defense document structured as follows. This is the document a CISO reads. ### 3.1 — The choice and the binding constraint (150–200 words) State the chosen posture in one sentence. State the binding constraint (the criterion that eliminated the other options or that your architecture must address). Reference your Phase 0 matrix. ### 3.2 — Why this posture, not the alternatives (150–200 words) Name the strongest alternative and explain why you rejected it. Be specific about the failure mode (e.g., "rejected API-under-BAA because phase-2 imaging routes PHI through a vision endpoint whose ZDR coverage is, per Protecto's analysis, typically excluded — creating a silent uncovered surface"). ### 3.3 — The supporting stack (150–200 words) Specify the serving stack (model, quant, server, VRAM envelope — draw on FT20). Specify the de-identification pipeline (tool, identifier classes, method). Specify the audit-logging approach (what, where, retention). One paragraph each. ### 3.4 — The residual risks and how you address them (150–200 words) Honest accounting. What can still go wrong? (Memorization despite de-identification? Insider extraction? GPU firmware?) What control addresses each? This is the defense-in-depth section — no single control trusted alone. ### 3.5 — What you would *not* do (100–150 words) Name the anti-patterns you explicitly rejected (Section 21.6 of the teaching document). This signals to the reviewer that you have thought about the failure modes, not just the happy path. --- ## Deliverables Submit a single `ft21-lab-architecture.md` containing: - [ ] **Phase 0**: the completed decision matrix + answers to the four questions. - [ ] **Phase 1**: the completed architecture template (every line filled). - [ ] **Phase 2**: the architecture diagram (Mermaid code block, or a link to an Excalidraw/draw.io export). - [ ] **Phase 3**: the written defense (~600–900 words across the five sub-sections). --- ## Solution key A correct submission has the following properties. (There is legitimate variation in tool/model choice; the key evaluates *judgment*, not a single right answer.) ### The choice - **Posture: local (Option 3 or Option 4).** Almost every defensible submission chooses local for this scenario. The phase-2 imaging workflow is the binding constraint: vision endpoints under mainstream BAAs are frequently excluded from ZDR, creating a silent uncovered surface. The compliance office's "we will not police endpoint-by-endpoint for the life of the product" constraint eliminates Options 1 and 2 outright. Option 4 (on-prem GPUs) is the strongest; Option 3 (self-hosted in VPC) is acceptable if the team cannot operate metal. - **Rejecting API-under-BAA**: a correct defense cites at least one specific coverage gap (vision endpoint, Assistants/Threads, or fine-tuning endpoint ZDR ineligibility), not a vague "we don't trust the vendor." ### The serving stack (local) - **Model**: an open-weights base appropriate to clinical reasoning (e.g., Llama 3.x, Qwen 2.5, Mistral — the specific choice draws on FT03/FT20). The defense should note that *open-weights* is the precondition for the local path. - **Quant**: AWQ or GPTQ for vLLM production; GGUF only if serving via Ollama/llama.cpp for a smaller-scale deployment. (FT20.) - **Server**: vLLM for throughput; TGI as an alternative; Ollama only for prototyping. (FT20.) - **VRAM envelope**: derived from model size + quant (FT01/FT20). A 70B AWQ-4bit needs ~40GB; a 8B AWQ-4bit needs ~6GB. The defense should state the envelope and the GPU it implies. ### The de-identification pipeline - **Tool**: Presidio (open source), Tonic Textual (commercial), or a custom spaCy + medical-NER + regex pipeline. The choice should be justified, not arbitrary. - **Identifier classes**: direct identifiers (names, MRNs, dates, contact, SSN), quasi-identifiers (rare ZIP, rare profession, age > 89), and free-text PHI in clinical narrative. - **Method**: Safe Harbor (the 18-identifier list) or Expert Determination. Safe Harbor is the more defensible default; Expert Determination requires a qualified statistician. - **Verification**: the corpus is "PHI-scarce," verified by a re-identification scan before training. ### The audit-logging approach - **What**: every prompt, every response, every model invocation, every access — who, when, model version, prompt hash, response hash. - **Where**: an append-only log store inside the trust boundary; optionally shipped to a SIEM under BAA. - **Retention**: per the organization's HIPAA security-rule posture (commonly 6 years). - **Access**: restricted; the logs themselves contain PHI and must be access-controlled. ### The residual risks - **Memorization despite de-identification**: addressed by eval-for-memorization (canary insertion, extraction attacks) before deploy; DP-SGD for high-sensitivity subsets. - **Insider extraction**: addressed by access controls, audit logging, and the local boundary (no external adversary in the data path). - **Operational failure**: GPU failure, model regression, prompt-injection from clinical text — addressed by failover, version pinning, and input sanitization. ### What a correct submission explicitly rejects (the anti-patterns) - "We signed the BAA, so we're fine" (the coverage-gap anti-pattern). - "We're local, so we can train on raw PHI" (the memorization anti-pattern). - "We're local, so we don't need to log" (the audit anti-pattern). - "We're on-prem, so we're compliant" (the safeguards anti-pattern — encryption, access controls, risk analysis still required). A submission that chooses API-under-BAA *can* be defensible if and only if it (a) restricts to text-only workflows (no phase-2 imaging), (b) maintains a documented endpoint-coverage matrix, (c) accepts the egress risk explicitly, and (d) explains why local is infeasible (latency, cost, capability). Such a submission is rare and should be marked down relative to a local choice for this scenario. --- ## Stretch goals 1. **Add the fine-tuning-data evaluation stage.** Specify the memorization-evaluation suite: canary strings inserted into the training corpus, extraction-prompt battery, membership-inference test. Describe what pass/fail looks like. (Sets up Pillar 1 evaluation rigor.) 2. **Cost-model the local path.** GPU capex (or reserved-cloud opex), power, cooling, failover headcount, vs the API path's per-token cost at Meridian's expected call volume. At what volume does local break even? (Sets up FT22 operational depth.) 3. **Write the incident-response runbook entry.** If the audit log shows an anomalous extraction-style prompt pattern, what is the response sequence? This is the operational complement to the architecture — it is what makes the audit log useful rather than merely present.