Repository navigation
Honest README, strict-by-default preflight, scrubbed internals, full Apache-2.0 - #3
Merged
Merged
Conversation
…Apache-2.0
P1-1: README rewritten into Implemented (v0.1) vs Planned; the four
subsystems that exist only as package stubs (shift gauntlet, adjudication,
calibration, receipts) are labelled planned; "hardened", "audit-grade", and
"the moat" removed from README, CLAUDE.md, ARCHITECTURE.md, pyproject
description, module docstrings. States plainly that no end-to-end eval of a
real model has been run yet.
P1-2: judge preflight is strict by default — missing keys or zero canaries
attempted exit non-zero; --allow-missing added for keyless local dev; the
old --strict flag is now the default behaviour. Also fixes a latent bug the
new test exposed: an empty canaries dict fell through to live SDK calls
(`canaries or {...}`); now `if canaries is None`.
P1-3: owner-environment references scrubbed: dotenv path hints in
preflight.py and CLAUDE.md generalised; healthcraft/scribegoat memory
references in dual_judge.py and openai_judge.py rewritten as self-contained
rationale.
P1-5: LICENSE now carries the full Apache-2.0 text (was a 17-line stub that
GitHub reported as NOASSERTION); NOTICE added.
P2-1: SDK floors raised to anthropic>=1.0, openai>=2 to match the API
shapes the adapters use.
Tests: 54 passed (3 new: strict-default main, --allow-missing, zero-canary
strict failure); ruff check/format, bandit clean; coverage 76.6% (gate 70).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01142EwVwkXFRGnYu7YgHSYL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes from the 2026-08-24 public-repo readiness audit (P1-1, P1-2, P1-3, P1-5, P2-1; P1-4 tag follows after merge).
P1-1 — README tells the truth
Rewritten into Implemented (v0.1) vs Planned (not yet built). The four subsystems that exist only as one-line package stubs (shift gauntlet/CSGG, physician adjudication, calibration ECE, receipts attestation) are labelled planned; "hardened", "audit-grade", and "the moat" are gone from README, CLAUDE.md, docs/ARCHITECTURE.md, the pyproject description, and module docstrings. The README now states plainly that no end-to-end eval of a real model has been run and that the judge adapters have never been exercised against a live judge from this repo.
P1-2 — preflight fails closed by default
python -m medimage_eval.judges.preflight(andmake preflight) is now strict by default: missing keys exit 3, zero canaries attempted exits 3, auth failure exits 2.--allow-missingreplaces the old opt-in--strictfor keyless local dev. Writing the test for the zero-canary path exposed a latent bug —canaries or {...}treated an injected empty dict as "use live SDK canaries" — fixed with an explicitis Nonecheck.P1-3 — internal references scrubbed
The
~/lostbench/.envhint in preflight.py, the/Users/kiteboard/...path in CLAUDE.md, and the healthcraft-V9 / scribegoat2 memory references in dual_judge.py and openai_judge.py are rewritten as generic, self-contained rationale.P1-5 — real license file
LICENSE now carries the full Apache-2.0 text (the previous 17-line stub made GitHub report NOASSERTION); NOTICE added.
P2-1 — SDK floors match the API shapes used
anthropic>=1.0(the adapter usesoutput_config.format),openai>=2.Verification
ruff check,ruff format --check,banditcleanAfter merge: tag
v0.1.0.dev0and push it (P1-4); the consumer repos pin to that tag in their own PRs.🤖 Generated with Claude Code
https://claude.ai/code/session_01142EwVwkXFRGnYu7YgHSYL