docs(n50): a clean N50' cannot be sourced from repo data — deferred to human-led curation

docs/research/dead-ends.md

@@ -456,7 +456,7 @@ Artifacts: `scripts/probe_liver_zonation.py`, `tests/integration/test_liver_zona
 
 **Why the freeze 5.25 is not a generalization number:** contamination (above) + composition confound — 16 IV drugs (32%; the oral-tuned pipeline under-predicts all ~10×), 8 deliberately-adversarial OATP substrates (subset AAFE 8.45), 1 prodrug outlier (sepiapterin ~4900×). The cleanest extractable slice (hard-clean ∩ oral, N=24 novel oral NMEs) is AAFE ≈4.0 vs the 107-holdout ~2.62 on the same local stack — which **corroborates the documented prospective OOD degradation** (prospective 3.27, novel-chemotype F under-prediction), not a new signal.
 
-**Telltale if it returns:** any citation of "N50 generalization AAFE 5.25" (or a subset of it) as evidence the 2.743 headline is cherry-picking-inflated. The instrument is invalid; the number is quarantined. Fix shipped: `scripts/build_n50_exclusion.py` now keys on InChIKey-14 across all corpora + DrugBank with an `--audit` gate; `holdout_n50.json` + the freeze carry an `invalidated` block; `run_n50_benchmark.py` refuses a freeze on an invalidated file. Full record: [n50_2026q2_invalidation.md](./n50_2026q2_invalidation.md). Re-curation of a clean N50' is deferred.
+**Telltale if it returns:** any citation of "N50 generalization AAFE 5.25" (or a subset of it) as evidence the 2.743 headline is cherry-picking-inflated. The instrument is invalid; the number is quarantined. Fix shipped: `scripts/build_n50_exclusion.py` now keys on InChIKey-14 across all corpora + DrugBank with an `--audit` gate; `holdout_n50.json` + the freeze carry an `invalidated` block; `run_n50_benchmark.py` refuses a freeze on an invalidated file. Full record: [n50_2026q2_invalidation.md](./n50_2026q2_invalidation.md). Re-curation of a clean N50' is deferred. **Status 2026-07-07 — a clean N50' cannot be sourced from repo data (feasibility assessment, [n50_prime_feasibility_2026-07-07.md](./n50_prime_feasibility_2026-07-07.md)):** the genuinely-clean candidate pool from `clinical_pk.json` is **0** (331 → 177 with Cmax → 16 after removing holdout+train → 0 valid: the 2 nominal survivors are guanfacine ER = holdout HCl-salt IK14 leak and lanthanum carbonate = inorganic combo), with **0/≥3** OATP1B1 substrates. N=50 is infeasible from repo sources (margin −50) because the clinical reference *is* the training source (167/331 names in holdout+train) and DrugBank covers 14,154 IK14 blocks. A real N50' needs ~50 fresh novel molecules with primary-source Cmax — a **human-led** curation (an agent must not fabricate primary values for a never-touch instrument). The E4 rule could relax to hard-corpora-only *if* the benchmark commits to public-clone (DrugBank hidden doesn't leak), but even so the repo pool is ~2–5. Deferred to a separately-authorized human curation cycle; tooling + spec are ready.
 
 ---
 

docs/research/experiment-log.md

@@ -10,6 +10,33 @@ Reverse-chronological. The project README carries only the **current** headline
 
 ---
 
+## 2026-07-07 (cont.) — N50' clean re-curation: infeasible from repo data (pool=0), deferred to human-led curation
+
+Scoped the clean N50' secondary-holdout curation (the unbiased-generalization instrument the
+cherry-picking caveat calls for; the 2026Q2 attempt was invalidated for contamination, DE-53). A
+3-agent investigation (spec/requirements, IK14 tooling, candidate-pool feasibility) answered the
+make-or-break question first: **is a clean N=50 even sourceable? No.**
+
+- **Clean pool = 0.** `clinical_pk.json`: 331 drugs → 177 with Cmax → 16 after removing the
+  107-holdout + 76-train (name **and** IK14) → 2 nominal clean → **0 valid** (guanfacine ER = holdout
+  HCl-salt IK14 leak; lanthanum carbonate = inorganic combo). OATP1B1 non-statin substrates in the
+  clean pool: **0 / ≥3 required.** N=50 infeasible from repo sources by −50; even N=1 not achievable.
+- **Why:** the clinical reference *is* the training source (167/331 names in holdout+train); of the 16
+  survivors, 11 are in hard corpora and 3 have unparseable SMILES. E4 (DrugBank-absent) is near-impossible
+  — DrugBank covers 14,154 IK14 blocks (the 2026Q2 set was 47/50 in DrugBank).
+- **E4 / public-clone reframe:** the 2.743 headline is public-clone (DrugBank hidden → no enrichment
+  leak), so E4 could relax to hard-corpora-only *if* the benchmark commits to public-clone — but even
+  relaxed the repo pool is ~2–5, not 50.
+- **Deferred to a human-led cycle.** A real N50' needs ~50 fresh novel molecules with primary-source
+  Cmax (no back-calc), CID-verified SMILES, ≥3 non-statin OATP1B1 substrates, oral-majority. **Integrity
+  constraint:** an agent must not generate primary-source Cmax/SMILES/DOI for a never-touch instrument —
+  a fabricated value is undetectable by the IK14 gate and would be worse than the 2026Q2 contamination.
+  The tooling (#96) and spec are ready; the blocker is human-verified data sourcing.
+
+Also flagged a tooling gap: the IK14 gate is defeated by salt forms (guanfacine HCl IK14 ≠ free-base
+IK14), a false-negative-only leak a curator would catch but worth a salt-strip fix. Full record:
+[n50_prime_feasibility_2026-07-07.md](./n50_prime_feasibility_2026-07-07.md); DE-53 status line.
+
 ## 2026-07-07 — Two DE-44 kill-tests run: adaptive PI walled (DE-55), invivo-F-prior killed via placebo (DE-56)
 
 Ran the two cheap pre-registered kill-tests DE-44 flagged as "survivors to test," on the 107-holdout

docs/research/n50_prime_feasibility_2026-07-07.md

@@ -0,0 +1,99 @@
+# N50' Clean Re-Curation — Feasibility Assessment (2026-07-07)
+
+**Status: infeasible from repository data → deferred to a human-led curation cycle.**
+
+Follow-on to the 2026Q2 invalidation ([n50_2026q2_invalidation.md](./n50_2026q2_invalidation.md),
+dead-ends.md DE-53). After the exclusion tooling was fixed to key on InChIKey-14 (PR #96), the
+next step was to curate a genuinely-clean N50' (the unbiased-generalization instrument the
+cherry-picking caveat calls for). This assessment answers the make-or-break question **before**
+any curation: *is a clean N=50 even sourceable?* It is not.
+
+## Verdict
+
+A clean N50' secondary permanent holdout **cannot be sourced from the existing repository data**.
+The genuinely-clean candidate pool is **0**. N=50 is infeasible from repo sources by a margin of
+**−50**; even N=1 is not cleanly achievable. Building N50' requires importing ~50 fresh novel
+molecules with primary-source clinical Cmax from outside the current reference set — a
+**human-verified** curation effort, not an automated/agent execution.
+
+## The feasibility funnel (`data/reference/clinical_pk.json`)
+
+| Step | Filter | Count |
+|---|---|---|
+| 0 | Drugs in `clinical_pk.json` | 331 |
+| 1 | Non-null clinical `cmax_mg_L` | **177** |
+| 2 | After removing the 107-holdout + 76-train (by name **and** InChIKey-14) | **16** |
+| 3 | After removing hard-corpus IK14 hits + unparseable SMILES → nominal clean | 2 |
+| **3′** | **After inspection → genuinely valid** | **0** |
+| — | OATP1B1 non-statin substrates in the clean pool (spec requires ≥3) | **0** |
+
+- **Why step 2 is brutal:** 167 of the 331 `clinical_pk` names are literally in holdout+train —
+  the clinical reference and the holdout corpus are nearly the same file. Of the 16 survivors,
+  **11 are in hard training corpora** and **3 have unparseable SMILES**.
+- **The 2 nominal survivors both fail on inspection:**
+  - `guanfacine er` — FALSE CLEAN. Its stored SMILES is the **HCl salt** (IK14 `DGFYECXYGUIODH`),
+    whose connectivity block differs from the free base (`INJOMKTZOLKMBF`), so it dodged the
+    holdout-side IK14 match — but **guanfacine is explicitly in the holdout list**. This is the exact
+    salt/synonym leak class the tooling exists to catch; the counterion in the stored SMILES defeated
+    the IK14 gate (a tooling gap — see Follow-ups).
+  - `lanthanum carbonate` — inorganic multi-fragment salt (2×La³⁺ + 3×carbonate), not a single-active
+    small molecule, and present in DrugBank.
+
+## Why the constraints bind
+
+- **E1–E3** (107-holdout / MMPK-Cmax / TDC-CLint): the clinical reference **is** the training source.
+- **E4** (DrugBank-absent): DrugBank covers **14,154** InChIKey-14 blocks; the 2026Q2 attempt had
+  **47/50** in DrugBank. "Not in DrugBank" ≈ "not a catalogued drug" → only very-new molecules qualify.
+- **E5** (not in any validation file): the prospective N=28 novel-drug set is already consumed (it
+  lives in `data/validation/`), so those cannot be reused.
+
+## The E4 / public-clone reframe (a yield lever, not a rescue)
+
+The 2.743 headline is **public-clone** (DrugBank hidden). In that regime a drug being *in* DrugBank
+does **not** leak into its prediction — the fup/pKa/logP enrichment is disabled. So E4's
+"DrugBank-absent" rule is over-conservative *for a public-clone benchmark*. **If** the N50 benchmark
+commits to running public-clone (DrugBank hidden at freeze — note `run_n50_benchmark.py` does **not**
+currently hide it), E4 could relax to **hard-corpora-clean only** (the actual fitted-target leakage).
+But even with E4 relaxed, the repo pool is only ~2–5 — still far short of 50. The reframe improves a
+*fresh* curation's admissible yield; it does not rescue the repo pool.
+
+## What a real N50' requires (deferred, human-led)
+
+1. ~50 fresh novel molecules (likely 2024–2026 NMEs **not** already in the prospective N=28 set), each with:
+   - Primary-source observed Cmax — **no back-calculation from AUC + t½** (A2), ≥2 independent sources
+     (the prospective adversarial-verification discipline);
+   - PubChem-CID-verified canonical SMILES (A3), canonicalized to the free base;
+   - Source DOI / table reference (A5);
+   - ≥3 **non-statin** OATP1B1 substrates;
+   - oral-majority, with any IV / adversarial-transporter drugs quarantined into a **separately-reported
+     subset** (the 2026Q2 composition-confound lesson).
+2. Each candidate gated **pre-curation** through `scripts/build_n50_exclusion.py --audit` (zero
+   hard-corpus hits) + a DrugBank IK14 review.
+
+## Integrity constraint — why this is human-led, not agent-automated
+
+An agent must **not** generate the primary-source Cmax / SMILES / DOI values for a *never-touch*
+generalization instrument. A single hallucinated value would invalidate the instrument's entire
+purpose — **worse** than the 2026Q2 contamination, because a fabricated "primary source" is
+undetectable by the IK14 gate (which checks structure membership, not value provenance). The sourcing
+and verification of the actual clinical values must be human-performed. An agent may scaffold (draft
+candidate lists, run the IK14 audit, check SMILES parse) but may not be the source of record for the
+observed values.
+
+## Follow-ups (tooling, non-blocking)
+
+- **Salt canonicalization in the IK14 gate.** The guanfacine case shows a counterion in a stored
+  SMILES yields a different IK14 than the free base, silently defeating both the exclusion match and
+  the audit. `build_n50_exclusion.py` should strip salts / take the largest organic fragment before
+  computing IK14 on both the corpus and the candidate side. (Low priority — it made a *contaminated*
+  drug look clean here, i.e. it only ever produces false-negatives on exclusion, which a human curator
+  would catch, but it is a real gap.)
+
+## Status
+
+**Deferred to a separately-authorized, human-led curation cycle.** The tooling (IK14 exclusion +
+`--audit` gate, PR #96) and the governance spec
+(`docs/_internal/specs/2026-04-22-n50-secondary-holdout-design.md`) are ready. The blocker is the
+sourcing of fresh, primary-verified clinical data, which is out of scope for automated curation. Until
+then, the cherry-picking caveat stands: the 107-holdout AAFE (2.743) is a point estimate whose
+true-generalization value is not independently instrumented.