Readiness fixes: fail-closed gaps, claims true-up, repo hygiene (audit 2026-08-24)

.github/workflows/ci.yml

@@ -16,8 +16,8 @@ jobs:
   check:
     runs-on: ubuntu-latest
     steps:
-      - uses: actions/checkout@v4
-      - uses: actions/setup-python@v5
+      - uses: actions/checkout@11d5960a326750d5838078e36cf38b85af677262 # v4.4.0
+      - uses: actions/setup-python@a26af69be951a213d495a4c3e4e4022e16d87065 # v5.6.0
         with:
           python-version: "3.12"
           cache: pip

.gitignore

@@ -2,8 +2,18 @@
 synthetic-ambient-fhir-25/index.html
 synthetic-ambient-fhir-25/synthetic-ambient-fhir-25.json
 
+# Secrets — never commit key material (.env.example is the committed template)
+.env
+.env.*
+!.env.example
+*.pem
+*.key
+
 # Build/demo artifacts
+.venv/
 __pycache__/
 *.egg-info/
+*.egg-link
 out/
 .DS_Store
+.coverage

CLAUDE.md

@@ -6,7 +6,8 @@ anything committed here at any time.
 
 ## Start of session — do this first
 
-1. Read `STATUS.md`. Claim a lane (add/edit a row, commit + push it) **before** writing code.
+1. Read `docs/HACKATHON_LOG.md` (the coordination board, archived post-hackathon at that
+   path). Claim a lane (add/edit a row, commit + push it) **before** writing code.
 2. `make setup` once per machine. `make check` must be green before every push.
 
 ## Commands
@@ -21,12 +22,12 @@ anything committed here at any time.
 
 Several Claude Code sessions share this repo. To keep git history clean:
 
-- **One terminal = one lane.** Scope is a set of files/dirs, recorded in `STATUS.md`. Do not edit
-  files outside your lane; if you must, coordinate via a note in `STATUS.md` first.
+- **One terminal = one lane.** Scope is a set of files/dirs, recorded in `docs/HACKATHON_LOG.md`.
+  Do not edit files outside your lane; if you must, coordinate via a note there first.
 - **Commit small, commit often.** Stage files **by name** — never `git add -A` or `git add .`.
 - **Before every push:** `git pull --rebase origin main`, re-run `make check`, then push.
 - If a rebase conflicts in files outside your lane, stop — don't resolve someone else's work.
-- Shared files (`STATUS.md`, `README.md`, `pyproject.toml`) get one-line focused commits so
+- Shared files (`docs/HACKATHON_LOG.md`, `README.md`, `pyproject.toml`) get one-line focused commits so
   rebases stay trivial.
 
 ## Hard rules

CONTRIBUTING.md

@@ -0,0 +1,15 @@
+# Contributing
+
+HALO is a solo-maintained research project; issues and small PRs are welcome.
+
+- Setup: `make setup` (venv + dev deps), then `make check` (ruff + pytest,
+  offline) — it must be green before and after your change.
+- Stage files by name (never `git add -A`); keep commits focused.
+- Synthetic data only. Never commit `.env`, keys, or anything derived from a
+  real patient record.
+- Fail-closed is the house rule: on model refusal, truncation, bad JSON, or a
+  missing final answer, raise `halo.llm.LLMFailure` — never return a partial.
+- Type checking is strict on `src` (`.venv/bin/python -m mypy`); keep it clean.
+- Metrics claims need N and method (see CLAUDE.md).
+
+Security reports: see [SECURITY.md](SECURITY.md).

README.md

@@ -102,9 +102,10 @@ verification layer over every agent output. Full inventory: [docs/AGENTS.md](doc
 | Any identity match | At most a *candidate*; confirmation is a human act |
 
 **Cardinal metric: under-triage false negatives = 0.** Current results on the synthetic
-goldset (N=12 notes, claude-opus-4-8, single run, method in `src/halo/mci/demo.py`):
+goldset (single run, 2026-07-18, claude-opus-4-8, on 12 of the current 13 goldset cases —
+`mci-013` was added after that run; rerun pending; method in `src/halo/mci/demo.py`):
 11/12 category agreement, **0 under-triage FNs**, 78/84 extraction fields correct; the sole
-disagreement was safe-direction over-triage. 275 offline tests (including full input-space totality sweeps over the triage and
+disagreement was safe-direction over-triage. 297 offline tests (including full input-space totality sweeps over the triage and
 surge rule tables) gate CI.
 
 ## Quickstart
@@ -120,7 +121,7 @@ Live surfaces (need `ANTHROPIC_API_KEY`):
 
 ```sh
 .venv/bin/python -m halo.mci.demo            # goldset eval: extraction + triage + FN gate
-.venv/bin/python -m halo.mci.demo --handoff  # 4 scripted end-to-end scenarios (see below)
+.venv/bin/python -m halo.mci.demo --handoff  # 6 scripted end-to-end scenarios (see below)
 .venv/bin/python -m halo.mci.demo --surge    # offline: reverse-triage the census, print plan
 make serve                                    # then open http://127.0.0.1:8000
 ```
@@ -131,7 +132,7 @@ care tethers, and the surge action per bed) plus a **door triage** tab (paste or
 note, run the handoff, read the evidence-quoted observations, identity candidates with care
 flags, the agent tool trail, and the FHIR write-back preview).
 
-The three scripted scenarios mirror the published failure modes:
+The six scripted scenarios mirror the published failure modes; three examples:
 
 1. **Route 91 pattern** — self-transported, partial identity, head strike; resolves
    deterministically and surfaces the clopidogrel occult-hemorrhage flag.
@@ -154,12 +155,13 @@ The three scripted scenarios mirror the published failure modes:
 | `docs/GOVERNANCE.md` | Medical, legal, and ethical posture; AI risk register; limitations. |
 | `docs/WORKFLOW.md` | Who actually does MCI triage (nurses) and where each surface sits. |
 | `docs/INTEGRATION.md` | Epic/FHIR incorporation: SMART launch, alias write-back, CDS Hooks, chart bloat. |
-| `CLAUDE.md` / `STATUS.md` | Working charter and live multi-terminal coordination board. |
+| `CLAUDE.md` / `docs/HACKATHON_LOG.md` | Working charter and the archived hackathon coordination board. |
 
 ## Working agreements
 
-Several agent terminals work in this repo concurrently (full rules in [CLAUDE.md](CLAUDE.md)):
-claim a lane in [STATUS.md](STATUS.md) first; stage files by name; `git pull --rebase` +
+During the hackathon several agent terminals worked in this repo concurrently (full rules
+in [CLAUDE.md](CLAUDE.md)): claim a lane in the coordination board (now archived at
+[docs/HACKATHON_LOG.md](docs/HACKATHON_LOG.md)) first; stage files by name; `git pull --rebase` +
 `make check` before every push; synthetic data only; never read or print `.env` contents.
 
 ## Scope & claims

SECURITY.md

@@ -0,0 +1,19 @@
+# Security policy
+
+This repository is research software. It is not a medical device and must not
+be used for clinical decision-making. All data in this repository is synthetic.
+
+## Reporting a vulnerability
+
+Email b@thegoatnote.com with the repository name, a description, and
+reproduction steps. You will receive an acknowledgement within 5 business days.
+Please do not open a public issue for security reports.
+
+## Scope
+
+- Code in this repository and its GitHub Actions workflows.
+- Out of scope: third-party services referenced in docs, synthetic demo data.
+
+## Supported versions
+
+Only the default branch is supported.

docs/ECOSYSTEM.md

@@ -16,7 +16,8 @@ these repos.** They interoperate by shared *data formats* and shared
                   ▼
    ┌────────────────────────────┐
    │  HALO edu / CME engine      │  depth: step-graded drills, computable
-   │  (hand-built, MD-signed)    │  dosing, fail-closed refusal
+   │  (hand-built; all cards     │  dosing, fail-closed refusal
+   │   draft, sign-off pending)  │
    └────────────┬───────────────┘
                 │ hash-chained CME evidence (identical primitive)
                 ▼
@@ -26,6 +27,9 @@ these repos.** They interoperate by shared *data formats* and shared
             LostBench / SafeShift
 ```
 
+CME engine content status: hand-built; physician sign-off pending (all four cards
+carry `review.status="draft"`).
+
 ## Seam 1 — OpenEM as citation & provenance backbone  *(wired)*
 
 OpenEM supplies breadth and citations; HALO supplies depth. The boundary is

STATUS.md → docs/HACKATHON_LOG.md

@@ -1,4 +1,9 @@
-# STATUS — live coordination board
+# HACKATHON LOG — archived coordination board
+
+> Archived 2026-08-31. This was the live multi-terminal lane board for the
+> 2026-07-18 hackathon build (formerly `STATUS.md` at the repo root),
+> preserved verbatim below as the build log. Counts and claims reflect the
+> moment each lane closed, not the current tree.
 
 Every terminal updates this file when claiming or finishing work. Commit it with your changes —
 it is the source of truth for who is working where.

pyproject.toml

@@ -7,7 +7,7 @@ requires-python = ">=3.11"
 license = "Apache-2.0"
 authors = [{ name = "GOATnote Inc." }]
 dependencies = [
-    "anthropic>=0.116",
+    "anthropic>=1.0,<2",
     "pydantic>=2",
     "fastapi>=0.110",
     "uvicorn>=0.29",

src/halo/app.py

@@ -1,6 +1,7 @@
 """API surface + demo UI. Run locally: ``make serve`` -> http://127.0.0.1:8000"""
 
 from dataclasses import asdict
+from datetime import UTC, datetime
 from pathlib import Path
 from typing import Any
 
@@ -351,7 +352,10 @@ class HandoffRequest(BaseModel):
     """
 
     note: str = Field(min_length=1)
-    incident_date: str = Field(default="2026-07-18", pattern=r"^\d{4}-\d{2}-\d{2}$")
+    incident_date: str = Field(
+        default_factory=lambda: datetime.now(tz=UTC).date().isoformat(),
+        pattern=r"^\d{4}-\d{2}-\d{2}$",
+    )
     likely_survivable: bool | None = None
 
 

src/halo/edu/drill.py

@@ -20,11 +20,14 @@
 
 from __future__ import annotations
 
+import logging
 import re
 
 from halo.edu.corpus import module_version
 from halo.edu.models import DecisionPoint, DrillResult, ProcedureModule, StepGrade
 
+logger = logging.getLogger(__name__)
+
 
 def phrase_hits(phrase: str, text: str) -> bool:
     """Left word boundary always; right boundary for digit-final or short phrases.
@@ -70,7 +73,10 @@ def _llm_adjudicate(point: DecisionPoint, answer: str) -> bool:
     )
     try:
         result = llm.structured(prompt, schema)
-    except Exception:  # noqa: BLE001 — fail closed: any LLM/parse error must not grant credit
+    except Exception as exc:  # noqa: BLE001 — fail closed: any LLM/parse error must not grant credit
+        logger.warning(
+            "LLM adjudication unavailable (%s: %s); no credit granted", type(exc).__name__, exc
+        )
         return False
     return result.get("credit") is True
 

src/halo/edu/lookup.py

@@ -10,12 +10,15 @@
 
 from __future__ import annotations
 
+import logging
 import re
 from dataclasses import dataclass
 
 from halo.edu.corpus import load_corpus
 from halo.edu.models import ProcedureModule
 
+logger = logging.getLogger(__name__)
+
 _ALIAS_SUBSTRING = 3.0
 _ALIAS_TOKEN = 2.0
 _TEXT_TOKEN = 1.0
@@ -146,7 +149,10 @@ def route_with_claude(query: str) -> str | None:
     )
     try:
         result = llm.structured(prompt, schema)
-    except Exception:  # noqa: BLE001 — fail closed: any LLM/parse error routes to no match
+    except Exception as exc:  # noqa: BLE001 — fail closed: any LLM/parse error routes to no match
+        logger.warning(
+            "LLM lookup unavailable (%s: %s); routing to no match", type(exc).__name__, exc
+        )
         return None
     module_id = result.get("module_id")
     return module_id if module_id in ids else None

src/halo/llm.py

@@ -91,7 +91,9 @@ def agent_loop(
                 trail.append({"tool": block.name, "input": block.input})
     if last is None:
         raise LLMFailure("fail-closed: agent loop produced no message")
-    if last.stop_reason in ("refusal", "max_tokens", "pause_turn"):
+    # ``tool_use`` as the *final* stop reason means the runner hit ``max_iterations``
+    # mid-loop: the model still wanted tools and never produced a final answer.
+    if last.stop_reason in ("refusal", "max_tokens", "pause_turn", "tool_use"):
         raise LLMFailure(f"fail-closed: stop_reason={last.stop_reason}")
     return _text(last), trail
 
@@ -115,7 +117,10 @@ def structured(
     )
     if response.stop_reason in ("refusal", "max_tokens"):
         raise LLMFailure(f"fail-closed: stop_reason={response.stop_reason}")
-    data = json.loads(_text(response))
+    try:
+        data = json.loads(_text(response))
+    except json.JSONDecodeError as exc:
+        raise LLMFailure(f"fail-closed: invalid JSON: {exc}") from exc
     if not isinstance(data, dict):
         raise LLMFailure(f"expected JSON object, got {type(data).__name__}")
     return data

src/halo/mci/census.py

@@ -55,8 +55,9 @@ def open_beds(self) -> int:
 
 
 def load_census(path: str | Path | None = None) -> Census:
-    p = Path(path or os.environ.get("HALO_CENSUS_PATH", DEFAULT_CENSUS_PATH))
-    return _load_census_cached(str(p.resolve()))
+    if path is None:
+        path = os.environ.get("HALO_CENSUS_PATH") or DEFAULT_CENSUS_PATH
+    return _load_census_cached(str(Path(path).resolve()))
 
 
 @lru_cache(maxsize=4)

src/halo/mci/compliance.py

@@ -144,6 +144,11 @@ def report_findings(findings_json: str, summary: str) -> str:
         max_iterations=8,
     )
 
+    # Fail closed: a review that never reached report_findings must not render as a
+    # clean pass with zero findings — refuse instead of reporting nothing.
+    if not summary_holder:
+        raise llm.LLMFailure("fail-closed: agent never called report_findings")
+
     # Deterministic verification: rule must exist; evidence must actually appear.
     known_rules = {r["rule_id"] for r in RULES}
     haystack = "\n".join(_log_lines()) + "\n" + json.dumps(_snapshot())

src/halo/mci/demo.py

@@ -4,7 +4,7 @@
 
 ``python -m halo.mci.demo``            goldset eval — extraction accuracy vs gold
                                        observations and the under-triage FN gate (target 0).
-``python -m halo.mci.demo --handoff``  three scripted scenarios mirroring published MCI
+``python -m halo.mci.demo --handoff``  six scripted scenarios mirroring published MCI
                                        failure modes (Route 91 / Beirut / Boston patterns):
                                        triage + identity reconciliation + care flags.
 """
@@ -36,7 +36,7 @@ def _rule(title: str) -> None:
 
 
 def run_handoff_scenarios() -> int:
-    """Three scripted end-to-end scenarios — the judged demo path."""
+    """All scripted end-to-end scenarios (``SCENARIOS``) — the judged demo path."""
     from halo.mci.panel import care_flags
     from halo.mci.reconcile import reconcile
     from halo.mci.scenarios import INCIDENT_DATE, SCENARIOS

src/halo/mci/extract.py

@@ -79,7 +79,7 @@ def extract_observations(note: str) -> tuple[Observations, dict[str, str]]:
         EXTRACTION_SCHEMA,
         system=SYSTEM,
     )
-    values: dict[str, bool | int | None] = {}
+    values: dict[str, Any] = {}
     evidence: dict[str, str] = {}
     for name in _FIELDS:
         entry = data.get(name)

src/halo/mci/panel.py

@@ -80,8 +80,9 @@ class IdentityCues:
 
 
 def load_panel(path: str | Path | None = None) -> tuple[PanelPatient, ...]:
-    p = Path(path or os.environ.get("HALO_PANEL_PATH", DEFAULT_PANEL_PATH))
-    return _load_panel_cached(str(p.resolve()))
+    if path is None:
+        path = os.environ.get("HALO_PANEL_PATH") or DEFAULT_PANEL_PATH
+    return _load_panel_cached(str(Path(path).resolve()))
 
 
 @lru_cache(maxsize=4)

src/halo/mci/triage.py

@@ -47,10 +47,13 @@
 _RR_DISTRESS_THRESHOLD = 30
 
 
+def _invert(v: bool | None) -> bool | None:
+    return None if v is None else not v
+
+
 def _screen_answers(obs: Observations) -> tuple[dict[str, bool | None], tuple[str, ...]]:
-    invert = lambda v: None if v is None else not v
     derivations: list[str] = []
-    no_distress = invert(obs.respiratory_distress)
+    no_distress = _invert(obs.respiratory_distress)
     if no_distress is None and obs.respiratory_rate is not None:
         no_distress = obs.respiratory_rate < _RR_DISTRESS_THRESHOLD
         comparator = "<" if no_distress else ">="
@@ -62,7 +65,7 @@ def _screen_answers(obs: Observations) -> tuple[dict[str, bool | None], tuple[st
         "obeys_commands": obs.obeys_commands,
         "peripheral_pulse": obs.peripheral_pulse,
         "no_respiratory_distress": no_distress,
-        "hemorrhage_controlled": invert(obs.major_hemorrhage_uncontrolled),
+        "hemorrhage_controlled": _invert(obs.major_hemorrhage_uncontrolled),
     }
     return answers, tuple(derivations)
 

synthetic-ambient-fhir-25/NOTICE

@@ -0,0 +1,14 @@
+NOTICE — synthetic-ambient-fhir-25 dataset
+
+This directory contains the "synthetic-ambient-fhir-25" dataset: 25 fully
+synthetic patients (Synthea-simulated records with LLM-generated transcripts
+and notes; no real patient data). It was provided by Abridge for use during
+the Future of Agentic AI in Healthcare hackathon (Abridge x Anthropic x
+Lightspeed, San Francisco, 2026-07-18).
+
+This dataset is NOT covered by the repository's Apache-2.0 license. Its
+redistribution terms have not been confirmed with Abridge. Until they are,
+treat the dataset as provided for hackathon use only: do not redistribute it
+separately from this repository, and do not rely on it under Apache-2.0.
+
+Questions: b@thegoatnote.com