Skip to content

Readiness fixes: fail-closed gaps, claims true-up, repo hygiene (audit 2026-08-24) - #2

Merged
bGOATnote merged 3 commits into
mainfrom
fix/readiness-p1-p2
Aug 31, 2026
Merged

bGOATnote merged 3 commits into
mainfrom
fix/readiness-p1-p2

Conversation

@bGOATnote

Copy link
Copy Markdown
Member

Implements the P1 findings and P2 polish from the 2026-08-24 public-repo readiness audit.

P1

  • P1-1 .gitignore now excludes .env, .env.* (keeping .env.example), .venv/, *.pem, *.key, .coverage — the quickstart creates .env at the repo root and nothing guarded it. Folds in the .gitignore lines PR Test coverage analysis + gitignore .venv/.coverage #1 proposed.
  • P1-2 agent_loop treats stop_reason="tool_use" on the final message (SDK runner exhausted max_iterations with the model still requesting tools) as LLMFailure; run_compliance_review raises LLMFailure("...agent never called report_findings") when the summary holder is empty. A compliance audit can no longer render as a clean pass with zero findings.
  • P1-3 structured() wraps json.loads so malformed JSON raises LLMFailure (the documented contract) instead of escaping as a raw JSONDecodeError → HTTP 500.
  • P1-4 README metrics line now names the run explicitly (single run, 2026-07-18, on 12 of the current 13 goldset cases; rerun pending) and the test count matches the collected suite (297). Deferred to owner: rerun python -m halo.mci.demo on the 13-case goldset and commit evals/goldset_<date>.json (needs a live key; not run here).
  • P1-5 docs/ECOSYSTEM.md CME box no longer says "MD-signed": hand-built; physician sign-off pending (all four cards review.status="draft").
  • P1-6 synthetic-ambient-fhir-25/NOTICE states the dataset was provided by Abridge for hackathon use and that redistribution terms are unconfirmed. Deferred to owner: confirm terms with Abridge; the data itself is untouched.

P2

  • SECURITY.md (disclosure contact b@thegoatnote.com) + short CONTRIBUTING.md.
  • CI actions pinned to commit SHAs (checkout v4.4.0, setup-python v5.6.0).
  • anthropic>=1.0,<2 (code depends on 1.x beta surfaces).
  • The 5 mypy-strict errors in src/halo/mci fixed — mypy strict now clean on all 36 source files.
  • Scenario-count consistency: 6 scripted scenarios (README + demo.py).
  • STATUS.md archived to docs/HACKATHON_LOG.md; README/CLAUDE.md references updated.
  • HandoffRequest.incident_date defaults to today (UTC), not the hard-coded hackathon date.
  • edu/drill.py / edu/lookup.py log at warning level before failing closed.

Verification

  • 297/297 tests pass offline (was 291; +6 new fail-closed tests)
  • ruff check + ruff format --check clean
  • mypy strict: no issues in 36 source files
  • New .gitignore verified with git check-ignore (.env.example still tracked)

🤖 Generated with Claude Code

https://claude.ai/code/session_01142EwVwkXFRGnYu7YgHSYL

bGOATnote and others added 3 commits August 31, 2026 16:06
…view) + ignore .env

P1-1: .gitignore now excludes .env/.env.* (keeping .env.example), .venv/,
*.pem, *.key, .coverage — the quickstart creates .env at the repo root and
nothing guarded it (folds in PR #1's .venv/.coverage lines).

P1-2: agent_loop treats stop_reason="tool_use" on the final message (the SDK
runner exhausted max_iterations with the model still requesting tools) as
LLMFailure, and run_compliance_review raises LLMFailure when the agent never
called report_findings — a compliance audit can no longer render as a clean
pass with zero findings.

P1-3: structured() wraps json.loads so malformed JSON raises LLMFailure (the
documented contract) instead of escaping as a raw JSONDecodeError / HTTP 500.

Tests: fake tool-runner coverage for agent_loop (success, iteration cap, no
messages), invalid-JSON structured(), and both compliance fail-closed paths.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01142EwVwkXFRGnYu7YgHSYL
…et NOTICE

P1-4: README metrics line now states the run explicitly (single run,
2026-07-18, on 12 of the current 13 goldset cases; rerun pending) and the
test count matches the collected suite (297). The goldset rerun itself is
deferred to the owner: python -m halo.mci.demo, then commit
evals/goldset_<date>.json.

P1-5: docs/ECOSYSTEM.md no longer says "MD-signed" — the CME cards are
hand-built with physician sign-off pending (all four carry
review.status="draft").

P1-6: synthetic-ambient-fhir-25/NOTICE states the dataset was provided by
Abridge for hackathon use and that redistribution terms are unconfirmed
(not covered by the repo's Apache-2.0). Confirming terms with Abridge is
deferred to the owner.

Also: scenario count consistency (6 scripted scenarios, README + demo.py).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01142EwVwkXFRGnYu7YgHSYL
…mypy strict clean

- SECURITY.md (disclosure contact, scope, supported versions) and a short
  CONTRIBUTING.md.
- CI actions pinned to commit SHAs (checkout v4.4.0, setup-python v5.6.0).
- anthropic bounded to >=1.0,<2 (code depends on 1.x beta surfaces).
- Fix the 5 mypy-strict errors in src/halo/mci (env-default Path handling in
  panel/census, typed _invert helper in triage, extract values dict) — mypy
  strict now clean on all 36 source files, matching the log's claim.
- STATUS.md archived to docs/HACKATHON_LOG.md (hackathon is over); README and
  CLAUDE.md references updated.
- HandoffRequest.incident_date defaults to today (UTC) instead of the
  hard-coded hackathon date.
- edu/drill.py and edu/lookup.py log at warning level before failing closed,
  so an auth/config outage is distinguishable from a model decline.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01142EwVwkXFRGnYu7YgHSYL
@bGOATnote
bGOATnote merged commit ce25d9e into main Aug 31, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant