diff --git a/book/assets/figures/publication/crpto_fig13_alpha_gamma_funded_set.pdf b/book/assets/figures/publication/crpto_fig13_alpha_gamma_funded_set.pdf index cf6cd64..97d6814 100644 Binary files a/book/assets/figures/publication/crpto_fig13_alpha_gamma_funded_set.pdf and b/book/assets/figures/publication/crpto_fig13_alpha_gamma_funded_set.pdf differ diff --git a/book/assets/figures/publication/crpto_fig13_alpha_gamma_funded_set.png b/book/assets/figures/publication/crpto_fig13_alpha_gamma_funded_set.png index 5f2fe7c..1a8c519 100644 Binary files a/book/assets/figures/publication/crpto_fig13_alpha_gamma_funded_set.png and b/book/assets/figures/publication/crpto_fig13_alpha_gamma_funded_set.png differ diff --git a/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.pdf b/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.pdf index 909c0ad..338c797 100644 Binary files a/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.pdf and b/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.pdf differ diff --git a/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.png b/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.png index 38e757a..6bfdabc 100644 Binary files a/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.png and b/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.png differ diff --git a/docs/refactor/NEXT_WORK_PLAN_2026-06.md b/docs/refactor/NEXT_WORK_PLAN_2026-06.md index 7ab49cf..5b5a362 100644 --- a/docs/refactor/NEXT_WORK_PLAN_2026-06.md +++ b/docs/refactor/NEXT_WORK_PLAN_2026-06.md @@ -95,6 +95,36 @@ Resultado local: `just validate-champion`, `just drift-gate`, correctamente y subió `1 file` (el nuevo artefacto faltante en el remote tras A2 fase 4). +## Ejecución Claude/Codex (2026-06-14, paper polish sin freeze) + +Claude cerró en PR #72 la **Proposición A.1**: bajo Assumption 1 sola, +Markov es el statement correcto y ningún argumento de segundo momento agnóstico +mejora el umbral del body. Codex tomó el siguiente paso natural de la nota de +Claude: + +- **A.2 cluster-aware formalizada en el supplement.** La sección A21 ahora + contiene una Proposición A.2 con prueba Hoeffding condicional: si los + agregados de clusters son independientes tras fijar calibración, partición y + allocation, entonces el bound depende de `sum_g W_g^2`. También cuantifica el + umbral de tightening (`sum_g W_g^2 < 0.0070` para `alpha=0.01`, + `delta=0.10`) y muestra por qué los clusters observados no aprietan Markov + (`0.2407`, `0.3572`, `0.0914`). Body y TEX quedaron sincronizados: A.1/A.2 + se leen como frontera explícita entre "sin estructura" y "con estructura". +- **Figuras 13/14 listas para B/N.** `scripts/build_crpto_journal_package.py` + ya no depende solo del color: Fig. 13 usa estilos de línea y marcadores + redundantes; Fig. 14 usa colormap secuencial de grises, texto dinámico + negro/blanco según luminancia y champion marker con contorno. Se regeneraron + los PNG/PDF en `reports/crpto/figures/` y + `book/assets/figures/publication/`; las vistas convertidas a escala de grises + fueron inspeccionadas y siguen legibles. +- **Reproducibility capitalizado.** La sección de reproducibilidad del QMD y el + TEX de submission mencionan que el champion feature contract vive como + YAML/Parquet en vez de pickle opaco; solo modelo y calibrador permanecen como + binarios. + +No se ejecutó freeze ni submission. No se reabrió búsqueda, HPO, champion, +intervalos conformal, validación conformal ni optimización portfolio. + --- ## AUDITORÍA POST-EJECUCIÓN (2026-06-13, Claude) — leer antes de continuar diff --git a/dvc.lock b/dvc.lock index 8d6b63c..e149e48 100644 --- a/dvc.lock +++ b/dvc.lock @@ -492,8 +492,8 @@ stages: size: 2271 - path: scripts/build_crpto_journal_package.py hash: md5 - md5: 7a6ba1b71f97856e75c1aa715237a66d - size: 41609 + md5: e4b53b9a0a8a6580fc0a532725177255 + size: 42507 outs: - path: models/crpto_journal_package_status.json hash: md5 diff --git a/paper/CRPTO_ijds.qmd b/paper/CRPTO_ijds.qmd index 4e5ca0e..0024fc7 100644 --- a/paper/CRPTO_ijds.qmd +++ b/paper/CRPTO_ijds.qmd @@ -447,12 +447,10 @@ it gives the clean reading "miscoverage exceeds $\sqrt{\alpha}$ with probability at most $\sqrt{\alpha}$" (for $\alpha = 0.01$, a $0.10$ excess with probability at most $0.10$). Markov is deliberately the weakest defensible argument: it uses only the first moment. Supplement -Proposition A.1 proves the inverse is sharp: under Assumption 1 alone the -best second-moment (Cantelli) threshold is *worse* than Markov, so Markov -is optimal for the stated guarantee and useful Hoeffding/Bernstein-style -tightenings require additional independence, variance, or martingale -structure [@hoeffding1963; @boucheron2013concentration]. -We +Propositions A.1--A.2 separate the boundary: under Assumption 1 alone the +best second-moment (Cantelli) threshold is *worse* than Markov, while explicit +cross-cluster structure is the extra condition under which Hoeffding-style +tightening becomes available [@hoeffding1963; @boucheron2013concentration]. We keep those tightenings in the online supplement (A21) rather than in the body, because the contribution here is the auditable decision construction, not the sharpest possible tail bound. The exact certificate in this paper is @@ -896,9 +894,12 @@ Prosper and Freddie/Mendeley. # Reproducibility and Companion The project is built as an executable research bundle. Source code, Quarto -manuscript files, tests, DVC metadata, tables, figures, and status reports are -versioned together. Heavy data and model artifacts are stored outside Git and -verified through manifest hashes. The paper-facing commands regenerate tables, +manuscript files, tests, DVC metadata, the YAML/Parquet champion feature +contract, tables, figures, and status reports are versioned together. The +feature contract is no longer an opaque preprocessing pickle; only the model +and calibrator remain binary artifacts. Heavy data and model artifacts are +stored outside Git and verified through manifest hashes. The paper-facing +commands regenerate tables, figures, evidence summaries, and HTML/PDF manuscript surfaces from frozen inputs; protected champion stages are not rerun without an explicit drift validation plan. The external-replication summaries are stored locally under diff --git a/paper/submission/CRPTO_ijds_submission.tex b/paper/submission/CRPTO_ijds_submission.tex index 15b3375..013a020 100644 --- a/paper/submission/CRPTO_ijds_submission.tex +++ b/paper/submission/CRPTO_ijds_submission.tex @@ -438,11 +438,10 @@ \section{Theory}\label{sec:theory} reads cleanly as ``miscoverage exceeds $\sqrt{\alpha}$ with probability at most $\sqrt{\alpha}$'' (for $\alpha=0.01$, a $0.10$ excess with probability at most $0.10$). Markov is deliberately the weakest defensible argument: it uses only -the first moment. Supplement Proposition~A.1 proves the inverse is sharp: +the first moment. Supplement Propositions~A.1--A.2 separate the boundary: under Assumption~1 alone the best second-moment (Cantelli) threshold is -\emph{worse} than Markov, so Markov is optimal for the stated guarantee and -useful Hoeffding/Bernstein-style tightenings require additional independence, -variance, or martingale structure +\emph{worse} than Markov, while explicit cross-cluster structure is the extra +condition under which Hoeffding-style tightening becomes available \citep{hoeffding1963,boucheron2013concentration}. Those tightenings are kept in the online supplement (A21) rather than the body, because the contribution is the auditable decision construction, not the sharpest tail bound. The exact @@ -514,8 +513,9 @@ \section{Experimental Design}\label{sec:design} claims rest on---is the part the harness certifies end to end. All primary artifacts are represented as files with explicit ownership: model -binaries, calibration objects, conformal intervals, portfolio allocations, tables, -figures, and status JSON files. The anonymous submission describes the bundle without +binaries, calibration objects, conformal intervals, portfolio allocations, a +YAML/Parquet champion feature contract rather than an opaque preprocessing pickle, +tables, figures, and status JSON files. The anonymous submission describes the bundle without revealing author identity. Repository and remote-storage URLs will be disclosed according to the journal's double-anonymous and data/code-disclosure policy. diff --git a/paper/supplement_ijds.qmd b/paper/supplement_ijds.qmd index 11e93e3..818a7ed 100644 --- a/paper/supplement_ijds.qmd +++ b/paper/supplement_ijds.qmd @@ -175,28 +175,62 @@ transcribed table. ## Cluster-Aware Conditional Tightening -Let clusters `g = 1, ..., G` represent period, grade, or period-grade cells, and -let +Let clusters $g = 1,\ldots,G$ represent period, grade, or period-grade cells, +and define -```text -Z_g = sum_{i in g} w_i 1{Y_i > u_i(alpha)}. -``` +$$ +Z_g(\alpha)=\sum_{i\in g} w_i\mathbf{1}\{Y_i>u_i(\alpha)\},\qquad +W_g=\sum_{i\in g}w_i . +$$ Within each cluster, defaults and conformal misses may be arbitrarily -dependent. If, after conditioning on the calibration sample and fixed funded -allocation, the cluster aggregates are independent or conditionally -independent across `g`, then the weighted noncoverage sum admits a -cluster-level concentration bound. A Hoeffding-style version is -[@hoeffding1963; @boucheron2013concentration]: - -```text -P(V - E[V] >= t) <= exp(-2 t^2 / sum_g W_g^2), -``` +dependent. The useful structure, if one is willing to assert it, is +cross-cluster independence after conditioning on the calibration sample and the +fixed funded allocation. + +**Proposition A.2 (cluster-aware Hoeffding under cross-cluster independence).** +Let $\mathcal F$ contain the calibration sample, the frozen conformal recipe, +the declared cluster partition, and the selected funded allocation. Suppose +that, conditional on $\mathcal F$, the cluster aggregates +$Z_1(\alpha),\ldots,Z_G(\alpha)$ are independent, satisfy +$0\le Z_g(\alpha)\le W_g$, and obey conditional weighted validity +$\sum_g E[Z_g(\alpha)\mid\mathcal F]\le\alpha$ (for example, it is sufficient +that $E[Z_g(\alpha)\mid\mathcal F]\le\alpha W_g$ for every cluster). Then, for +every $\delta\in(0,1)$, + +$$ +P\!\left( + V(\alpha)\ge + \alpha + \sqrt{\frac{1}{2}\left(\sum_g W_g^2\right)\log\frac{1}{\delta}} + \;\middle|\;\mathcal F +\right)\le\delta . +$$ + +*Proof.* Let $\mu=\sum_g E[Z_g(\alpha)\mid\mathcal F]\le\alpha$ and +$S_2=\sum_g W_g^2$. Hoeffding's inequality for independent bounded summands +gives + +$$ +P\{V(\alpha)-\mu\ge s\mid\mathcal F\}\le \exp(-2s^2/S_2). +$$ -where `W_g = sum_{i in g} w_i` is the cluster exposure share. This proposition -does not replace the main Markov bound because the cross-cluster assumption is -additional. Table A14 reports the relevant exposure and miscoverage -concentration so reviewers can audit where that assumption would matter. +Taking $s=\sqrt{S_2\log(1/\delta)/2}$ and using $\mu\le\alpha$ gives the +displayed bound. Integrating over $\mathcal F$ gives the same unconditional +statement. $\blacksquare$ + +Proposition A.2 is therefore the natural complement to Proposition A.1. Under +Assumption 1 alone, A.1 shows why Markov is the sharp distribution-free claim; +under an explicit cross-cluster structure, A.2 shows exactly when a +Hoeffding-style tightening becomes available [@hoeffding1963; +@boucheron2013concentration]. At the paper level $\alpha=0.01$ with matched +tail probability $\delta=\sqrt{\alpha}=0.10$, the cluster-aware threshold is +tighter than Markov only if $\sum_g W_g^2<0.0070$. The frozen funded set is much +more concentrated: period, grade, and period-grade partitions have +$\sum_g W_g^2=0.2407$, $0.3572$, and $0.0914$, respectively, so the corresponding +thresholds are `0.5365`, `0.6512`, and `0.3344`, all looser than Markov's +`0.1000`. This proposition does not replace the main theorem; it names the +extra structure a reviewer would have to accept and makes the empirical +concentration cost transparent in A21. ### How much does the distribution-free bound leave on the table? diff --git a/reports/crpto/figures/crpto_fig13_alpha_gamma_funded_set.pdf b/reports/crpto/figures/crpto_fig13_alpha_gamma_funded_set.pdf index ac53afc..97d6814 100644 Binary files a/reports/crpto/figures/crpto_fig13_alpha_gamma_funded_set.pdf and b/reports/crpto/figures/crpto_fig13_alpha_gamma_funded_set.pdf differ diff --git a/reports/crpto/figures/crpto_fig13_alpha_gamma_funded_set.png b/reports/crpto/figures/crpto_fig13_alpha_gamma_funded_set.png index 5f2fe7c..1a8c519 100644 Binary files a/reports/crpto/figures/crpto_fig13_alpha_gamma_funded_set.png and b/reports/crpto/figures/crpto_fig13_alpha_gamma_funded_set.png differ diff --git a/reports/crpto/figures/crpto_fig14_robust_region_heatmap.pdf b/reports/crpto/figures/crpto_fig14_robust_region_heatmap.pdf index 53f5b19..338c797 100644 Binary files a/reports/crpto/figures/crpto_fig14_robust_region_heatmap.pdf and b/reports/crpto/figures/crpto_fig14_robust_region_heatmap.pdf differ diff --git a/reports/crpto/figures/crpto_fig14_robust_region_heatmap.png b/reports/crpto/figures/crpto_fig14_robust_region_heatmap.png index 38e757a..6bfdabc 100644 Binary files a/reports/crpto/figures/crpto_fig14_robust_region_heatmap.png and b/reports/crpto/figures/crpto_fig14_robust_region_heatmap.png differ diff --git a/scripts/build_crpto_journal_package.py b/scripts/build_crpto_journal_package.py index 1b0fac9..4d56850 100644 --- a/scripts/build_crpto_journal_package.py +++ b/scripts/build_crpto_journal_package.py @@ -709,6 +709,10 @@ def _plot_alpha_gamma_funded_set(bound_eval: pd.DataFrame, promotion: dict[str, data["alpha"], data["gamma_cp"], marker="o", + markerfacecolor="white", + markeredgewidth=1.2, + linestyle="-", + linewidth=2.0, label=r"$\Gamma_{\mathrm{CP}}$", color="#0B5CAD", ) @@ -716,6 +720,10 @@ def _plot_alpha_gamma_funded_set(bound_eval: pd.DataFrame, promotion: dict[str, data["alpha"], data["weighted_miscoverage_V"], marker="s", + markerfacecolor="white", + markeredgewidth=1.2, + linestyle="-.", + linewidth=2.0, label=r"$V(\alpha)$", color="#B00020", ) @@ -723,6 +731,11 @@ def _plot_alpha_gamma_funded_set(bound_eval: pd.DataFrame, promotion: dict[str, data["alpha"], data["sqrt_alpha"], linestyle="--", + marker="D", + markersize=4.8, + markerfacecolor="white", + markeredgewidth=1.0, + linewidth=1.8, label=r"$\sqrt{\alpha}$", color="#616161", ) @@ -744,8 +757,17 @@ def _plot_alpha_gamma_funded_set(bound_eval: pd.DataFrame, promotion: dict[str, ax1.grid(alpha=0.25) ax1.legend(loc="best", frameon=False) - ax2.plot(data["alpha"], data["n_funded"], marker="^", color="#2E7D32", linewidth=2.0) - ax2.fill_between(data["alpha"], data["n_funded"], alpha=0.14, color="#2E7D32") + ax2.plot( + data["alpha"], + data["n_funded"], + marker="^", + markerfacecolor="white", + markeredgewidth=1.2, + linestyle="-.", + color="#263238", + linewidth=2.0, + ) + ax2.fill_between(data["alpha"], data["n_funded"], alpha=0.12, color="#78909C") ax2.axvline(0.01, color="#263238", linestyle=":", linewidth=1.1) ax2.set_xlabel(r"Conformal level $\alpha$") ax2.set_ylabel("Funded loans") @@ -776,7 +798,8 @@ def _plot_robust_region_heatmap(shortlist: pd.DataFrame, promotion: dict[str, An aggfunc="max", ).sort_index(ascending=False) fig, ax = plt.subplots(figsize=(8.4, 5.4)) - im = ax.imshow(pivot.to_numpy(), cmap="viridis", aspect="auto") + cmap = plt.get_cmap("Greys") + im = ax.imshow(pivot.to_numpy(), cmap=cmap, aspect="auto") ax.set_xticks(np.arange(len(pivot.columns))) ax.set_xticklabels([f"{x:.2f}" for x in pivot.columns]) ax.set_yticks(np.arange(len(pivot.index))) @@ -788,7 +811,18 @@ def _plot_robust_region_heatmap(shortlist: pd.DataFrame, promotion: dict[str, An for j in range(pivot.shape[1]): value = pivot.iloc[i, j] if pd.notna(value): - ax.text(j, i, f"{value / 1000:.0f}K", ha="center", va="center", color="white") + rgba = cmap(im.norm(float(value))) + luminance = (0.2126 * rgba[0]) + (0.7152 * rgba[1]) + (0.0722 * rgba[2]) + text_color = "white" if luminance < 0.45 else "#111111" + ax.text( + j, + i, + f"{value / 1000:.0f}K", + ha="center", + va="center", + color=text_color, + fontweight="bold", + ) champion = promotion["final_champion"] champion_gamma = float(champion["gamma"]) champion_tau = float(champion["risk_tolerance"]) @@ -800,9 +834,9 @@ def _plot_robust_region_heatmap(shortlist: pd.DataFrame, promotion: dict[str, An row, marker="*", s=360, - color="#FDD835", - edgecolor="#263238", - linewidth=1.1, + color="white", + edgecolor="#111111", + linewidth=1.4, zorder=4, ) ax.annotate(