diff --git a/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.pdf b/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.pdf index 338c797..816f9df 100644 Binary files a/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.pdf and b/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.pdf differ diff --git a/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.png b/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.png index 6bfdabc..23c63e3 100644 Binary files a/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.png and b/book/assets/figures/publication/crpto_fig14_robust_region_heatmap.png differ diff --git a/dvc.lock b/dvc.lock index e149e48..71d32bf 100644 --- a/dvc.lock +++ b/dvc.lock @@ -492,8 +492,8 @@ stages: size: 2271 - path: scripts/build_crpto_journal_package.py hash: md5 - md5: e4b53b9a0a8a6580fc0a532725177255 - size: 42507 + md5: 2b75b74c8ab4f7f051c0f550d72b5e5a + size: 43045 outs: - path: models/crpto_journal_package_status.json hash: md5 diff --git a/paper/CRPTO_ijds.qmd b/paper/CRPTO_ijds.qmd index 3fcedcc..a129779 100644 --- a/paper/CRPTO_ijds.qmd +++ b/paper/CRPTO_ijds.qmd @@ -41,10 +41,11 @@ single-family mortgages preserve the conformal gates and produce positive robust LP objectives, so the result is not confined to the Lending Club panel. Across these external panels the price of robustness is a positive premium that grows with the panel default rate (from `+1.0%` to `+9.5%`). The contribution is -a conformal-robust credit-portfolio decision certificate: it connects real -credit data, calibrated predictive models, robust funding decisions, and a drift -harness that regenerates the prediction-to-decision chain bit-exactly from -frozen artifacts while keeping the statistical guarantee boundary explicit. +a conformal-robust credit-portfolio decision certificate with a distribution-free +funded-set risk bound: it connects real credit data, calibrated predictive +models, robust funding decisions, and a drift harness that regenerates the +prediction-to-decision chain bit-exactly from frozen artifacts while keeping the +statistical guarantee boundary explicit. **Keywords:** conformal prediction; robust optimization; predict-then-optimize; credit risk; portfolio optimization; reproducible data science. @@ -97,15 +98,21 @@ out-of-sample and out-of-time splits. These replications are not new champions; they test whether the same PD-to-conformal-to-LP recipe remains economically usable on different credit products. -The paper makes four contributions. First, it gives a CRPTO construction for +The paper makes five contributions. First, it gives a CRPTO construction for credit portfolios: frozen calibrated PD, Mondrian conformal uncertainty, and -robust budgeted optimization as a post-hoc decision audit. Second, it locates -that construction relative to data-driven robust optimization, P2P lending -portfolio models, conformal credit scoring, and decision-focused learning. -Third, it provides an artifact-backed empirical study where every table and -figure is generated from frozen outputs rather than manually transcribed -summaries. Fourth, it adds external economic replications on Prosper and -Freddie/Mendeley, separating the methodological claim from one P2P panel. The +robust budgeted optimization as a post-hoc decision audit. Second, it proves a +distribution-free funded-set risk bound (Theorem 1) that splits realized +portfolio loss into the conformal upper-endpoint budget +$B_u(\alpha) = \tau + (1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha)$ and the weighted +miscoverage $V(\alpha)$, with supplement propositions showing that Markov is +optimal under the stated assumption (A.1) and locating the cluster structure +that would tighten it (A.2). Third, it locates that construction relative to +data-driven robust optimization, P2P lending portfolio models, conformal credit +scoring, and decision-focused learning. Fourth, it provides an artifact-backed +empirical study where every table and figure is generated from frozen outputs +rather than manually transcribed summaries. Fifth, it adds external economic +replications on Prosper and Freddie/Mendeley, separating the methodological +claim from one P2P panel. The key claim is narrow: CRPTO maps frozen calibrated PD artifacts into a robust funded set, reports the portfolio-level conformal premium $\Gamma_{\mathrm{CP}}$, and verifies exact alpha-safe weighted miscoverage on the @@ -163,7 +170,7 @@ feeds an action, the relevant target can become the assignment rule rather than only the intermediate effect-size estimate [@fernandezloria2022causaldecision]. SPO+ and modern decision-focused learning ask models to respect the loss surface induced by the downstream decision -[@elmachtoub2022; @donti2017; @mandi2024]. CRPTO is intentionally more +[@elmachtoub2022; @donti2017; @mandi2024]. CRPTO is more conservative. It does not retrain the PD model end-to-end through the optimizer. Instead, it asks what can be achieved when a calibrated predictive system is already frozen and the decision layer must remain explainable to credit-risk @@ -188,7 +195,7 @@ policies [@guo2016p2p; @zhao2016p2pportfolio; @serrano2016profitscoring; @chi2019p2p; @babaei2020p2p; @aior2025lendingclub]. Recent ordinal conformal credit-scoring work also means that the safe claim is not "no conformal prediction in credit" [@kawasumi2026ordinal]. CRPTO does not compete on raw -ranking against this literature; its champion AUC is deliberately mid-range. +ranking against this literature; its champion AUC is mid-range. The contribution is the auditable bridge from a calibrated, frozen PD model to a conformal robust portfolio decision, not another point on the credit-scoring leaderboard. @@ -262,7 +269,7 @@ single global interval. The promoted uncertainty artifact uses a score-decile-based Mondrian partition selected by out-of-time interval quality, while grade-based partitions remain the natural governance baseline. -The resulting conformal summary is intentionally more than a scalar coverage +The resulting conformal summary is more than a scalar coverage number. The paper-facing metrics include 90% coverage `0.9297`, 95% coverage `0.9664`, average 90% interval width `0.7842`, minimum group 90% coverage `0.9190`, and 90% Winkler score `1.1107` for the promoted conformal winner. @@ -414,24 +421,42 @@ subportfolio, so the assumption is stated, audited empirically after the frozen selection, and never silently upgraded to a guarantee. **Theorem 1 (distribution-free funded-set risk bound).** -Suppose the robust constraint holds at the chosen allocation, -$\sum_i w_i u_i(\alpha) \leq \tau$. Then: +Let $B_u(\alpha) = \sum_i w_i u_i(\alpha)$ be the weighted conformal +upper-endpoint budget of the funded set. Then: -(i) *(deterministic)* $\;\sum_i w_i Y_i \leq \tau + V(\alpha)$ always; +(i) *(deterministic)* $\;\sum_i w_i Y_i \leq B_u(\alpha) + V(\alpha)$ always; (ii) *(statistical)* under Assumption 1, for every $t > 0$, $P(V(\alpha) \geq t) \leq \alpha / t$, and in particular $$ -P\!\left(\sum_i w_i Y_i \;\geq\; \tau + \sqrt{\alpha}\right) +P\!\left(\sum_i w_i Y_i \;\geq\; B_u(\alpha) + \sqrt{\alpha}\right) \;\leq\; \sqrt{\alpha}. $$ *Proof sketch.* Since $Y_i \leq u_i(\alpha) + Z_i(\alpha)$ for every loan, -part (i) follows by taking the $w$-weighted sum and applying the constraint. -Part (ii) is Markov's inequality applied to the nonnegative variable -$V(\alpha)$ with $E[V(\alpha)] \leq \alpha$ [@ghosh2002], combined with (i). -The full proof is in Online Supplement Appendix A. $\square$ +part (i) follows by taking the $w$-weighted sum; it is portfolio accounting and +needs no probability. Part (ii) is Markov's inequality applied to the +nonnegative variable $V(\alpha)$ with $E[V(\alpha)] \leq \alpha$ [@ghosh2002], +combined with (i). The full proof is in Online Supplement Appendix A. $\square$ + +**The optimizer's cap versus the endpoint budget.** The robust layer +does not constrain $B_u(\alpha)$ directly; it caps the $\gamma$-blended PD, +$\sum_i w_i \tilde p_i(\alpha,\gamma) \leq \tau$, with +$\tilde p_i = \hat p_i + \gamma(u_i(\alpha) - \hat p_i)$ and $\gamma \in [0,1]$. +Because $\Gamma_{\mathrm{CP}}(\alpha) = \sum_i w_i(u_i(\alpha) - \hat p_i)$, the +endpoint budget decomposes exactly as +$$ +B_u(\alpha) = \sum_i w_i \tilde p_i(\alpha,\gamma) + (1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha) + \;\leq\; \tau + (1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha), +$$ +with equality when the risk cap binds. The term +$(1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha)$ is the conformal robustness premium the +optimizer leaves un-internalized at $\gamma < 1$. For the promoted policy +($\tau = 0.175$, $\gamma = 0.45$, binding cap), +$B_u(0.01) = 0.175 + 0.55\,(0.187987) = 0.278393$, so the deterministic bound is +$\sum_i w_i Y_i \leq 0.278393 + V(0.01) = 0.307268$, well above the realized +weighted default rate $0.032875$. **Remark 1 (why $t = \sqrt{\alpha}$, and why Markov).** The choice $t = \sqrt{\alpha}$ is made for interpretability, not optimality: @@ -485,7 +510,7 @@ future information leaks into the funded-set decision. The displayed periods are monthly vintage labels; split assignment is row-disjoint in code, so the shared March 2017 label marks the internal cutoff rather than duplicated loans. -The out-of-time design is deliberately adversarial to the method: the test window +The out-of-time design is adversarial to the method: the test window spans an expansion (2018--2019) and a regime break (2020 COVID), so coverage and the funded-set certificate are measured under a documented distribution shift rather than on a random split that would let the model see the future. @@ -518,7 +543,7 @@ the IJDS body focused while preserving the audit trail that reviewers need. ## Multi-Dataset External Replication Protocol -The external-replication layer is deliberately narrower than a new benchmark +The external-replication layer is narrower than a new benchmark campaign. We reuse the frozen CRPTO recipe--CatBoost PD, calibration, train-only WOE/IV feature screening, Mondrian conformal intervals, and the same bound-aware robust LP--on two credit datasets with economic fields. Prosper @@ -540,7 +565,7 @@ address whether the method survives two materially different credit products. # Results The core metric table summarizes the frozen paper-facing metrics. The -calibrated PD layer is deliberately not sold as a leaderboard model: AUC +calibrated PD layer is not sold as a leaderboard model: AUC `0.7139` is sufficient only because the downstream decision consumes calibrated probabilities, not rankings alone. Its Brier score `0.1544` and ECE near `0.0070` are therefore as important as discrimination. The conformal layer @@ -563,7 +588,7 @@ turns this uncertainty into a robust allocation with exact alpha-safe checks. : Frozen paper-facing metrics by layer. -The exact certificate in the body is intentionally small. Here "exact" means +The exact certificate in the body is a narrow claim. Here "exact" means that the quantities are computed directly on the frozen OOT funded set rather than approximated by a surrogate table or visual proxy. The statistical reading still rests on the weighted funded-set validity assumption stated in the theory @@ -583,7 +608,7 @@ $\Gamma_{\mathrm{CP}} = 0.187987$ as the price of carrying interval uncertainty decision, and $V = 0.028875$ as the realized weighted noncoverage audit on the same funded loans. -The promoted policy is deliberately the economic champion inside the exact-safe +The promoted policy is the economic champion inside the exact-safe region, not the tightest certificate available. The next table makes that choice auditable rather than implicit. @@ -797,7 +822,7 @@ columns prevent a one-dimensional reading. Mean regret comes from a separate decision-regret experiment (A19/PyEPO) run on small synthetic optimization instances (50 items, budget 15, five seeds), which scores each method on a normalized decision-loss scale rather than on the `$1M` funded portfolio. That -experiment is intentionally not the same object as the funded-set economics: it +experiment is not the same object as the funded-set economics: it isolates training-time decision quality, so SPO+ is the low-regret method by construction because it is trained to minimize exactly that loss. The favorable price of robustness reported above and the higher CRPTO regret here @@ -840,7 +865,7 @@ policies under a decision-time CVaR$_{95}$ cap (computed from the conformal uppe endpoints, supplement A22) traces an explicit return-versus-tail trade-off. The tightest admissible tail cap selects a challenger earning `$160,978` (`-5.57%` versus the champion) at CVaR$_{95}$ `0.406`, while the frozen economic champion -deliberately sits at the high-return, high-tail corner. Supplement A20 makes the +sits at the high-return, high-tail corner. Supplement A20 makes the same point from the committee side: once all 45 alpha-safe robust-region policies pass the satisficing screen, the useful contrast is the lowest realized-CVaR policy. It cuts CVaR$_{95}$ by `22.58%` at a return cost of only `1.99%`. The @@ -917,7 +942,7 @@ calibration split seed, score scaling, floor multipliers) and reproduces every published interval endpoint and per-cell coverage *exactly* (maximum absolute difference zero under the locked dependency stack), including re-learning identical group floor multipliers from the calibration holdout. -The boundary is stated as honestly as the rest of the paper: gradient-boosted +The boundary is concrete: gradient-boosted retraining is not bit-reproducible across runs, which is precisely why the predictive layer is distributed as a frozen binary artifact with manifest hashes rather than as a "retrain it yourself" recipe, and why the @@ -953,7 +978,7 @@ not a fixed toll but a panel-specific premium that the method measures rather than assumes. For a risk committee this turns the governance question from "can we afford robustness?" into "how large is the coverage premium on this book?", which CRPTO answers with a bounded figure per panel. It also tempers the -contribution honestly. The favorable Lending Club value reflects champion +contribution. The favorable Lending Club value reflects champion selection, so the transferable claim is the bounded, measurable premium under blind application, not a universal free lunch; the recipe carries its own price tag wherever it is applied. @@ -972,7 +997,7 @@ conformal guarantee is marginal or partitioned by the chosen Mondrian design, not exact conditional coverage for every borrower profile. The external panels make this concrete: on Freddie the all-group minimum coverage is driven by tiny sparse Mondrian cells, and the high-default red segment misses the strict -$\alpha = 0.01$ gate at `0.9850`; both are reported as honest sensitivity +$\alpha = 0.01$ gate at `0.9850`; both are reported as sensitivity evidence rather than promoted as conditional guarantees. Finally, this is not an online deployment study: there are no new post-2020 Lending Club retail originations, no live monitoring loop, and no end-to-end utility-directed diff --git a/paper/submission/CRPTO_ijds_submission.tex b/paper/submission/CRPTO_ijds_submission.tex index 2817d05..d7933d7 100644 --- a/paper/submission/CRPTO_ijds_submission.tex +++ b/paper/submission/CRPTO_ijds_submission.tex @@ -80,10 +80,11 @@ not confined to the Lending Club panel. Across these external panels the price of robustness grows with the panel default rate (from $+1.0\%$ to $+9.5\%$). The contribution is a conformal-robust -credit-portfolio decision certificate: it connects real credit data, calibrated -predictive models, robust funding decisions, and a drift harness that regenerates -the prediction-to-decision chain bit-exactly from frozen artifacts while keeping -the statistical guarantee boundary explicit.% +credit-portfolio decision certificate with a distribution-free funded-set risk +bound: it connects real credit data, calibrated predictive models, robust funding +decisions, and a drift harness that regenerates the prediction-to-decision chain +bit-exactly from frozen artifacts while keeping the statistical guarantee boundary +explicit.% } \KEYWORDS{conformal prediction; robust optimization; predict-then-optimize; @@ -133,15 +134,21 @@ \section{Introduction}\label{sec:intro} allocation. It is a reproducible bridge from calibrated probabilistic learning to robust, auditable credit portfolio choice. -The paper makes four contributions. First, it gives a CRPTO construction for +The paper makes five contributions. First, it gives a CRPTO construction for credit portfolios: frozen calibrated PD, Mondrian conformal uncertainty, and -robust budgeted optimization as a post-hoc decision audit. Second, it locates -that construction relative to data-driven robust optimization, P2P lending -portfolio models, conformal credit scoring, and decision-focused learning. -Third, it provides an artifact-backed empirical study where every table and -figure is generated from frozen outputs rather than manually transcribed -summaries. Fourth, it adds external economic replications on Prosper and -Freddie/Mendeley, separating the methodological claim from one P2P panel. The +robust budgeted optimization as a post-hoc decision audit. Second, it proves a +distribution-free funded-set risk bound (Theorem~\ref{thm:funded-set-bound}) +that splits realized portfolio loss into the conformal upper-endpoint budget +$B_u(\alpha)=\tau+(1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha)$ and the weighted +miscoverage $V(\alpha)$, with supplement propositions showing that Markov is +optimal under the stated assumption (A.1) and locating the cluster structure +that would tighten it (A.2). Third, it locates that construction relative to +data-driven robust optimization, P2P lending portfolio models, conformal credit +scoring, and decision-focused learning. Fourth, it provides an artifact-backed +empirical study where every table and figure is generated from frozen outputs +rather than manually transcribed summaries. Fifth, it adds external economic +replications on Prosper and Freddie/Mendeley, separating the methodological +claim from one P2P panel. The key claim is narrow: CRPTO maps frozen calibrated PD artifacts into a robust funded set, reports the portfolio-level conformal premium $\Gamma_{\mathrm{CP}}$, and verifies exact alpha-safe weighted miscoverage on the promoted Lending Club @@ -196,7 +203,7 @@ \section{Related Work}\label{sec:related} by the downstream decision \citep{elmachtoub2022,donti2017,mandi2024}. IJDS work on causal decision making sharpens the same warning: once an estimate feeds an action, the relevant target can become the assignment rule rather than only the -intermediate effect-size estimate \citep{fernandezloria2022causaldecision}. CRPTO is intentionally more +intermediate effect-size estimate \citep{fernandezloria2022causaldecision}. CRPTO is more conservative. It does not retrain the PD model end-to-end through the optimizer. Instead, it asks what can be achieved when a calibrated predictive system is already frozen and the decision layer must remain explainable to credit-risk @@ -217,7 +224,7 @@ \section{Related Work}\label{sec:related} \citep{zhao2016p2pportfolio,serrano2016profitscoring}, and operations-research treatments of the same investment-decision problem CRPTO studies \citep{aior2025lendingclub}. CRPTO does not compete on raw ranking against this -literature; its champion AUC is deliberately mid-range. The contribution is the +literature; its champion AUC is mid-range. The contribution is the auditable bridge from a calibrated, frozen PD model to a robust portfolio decision, not another point on the credit-scoring leaderboard. @@ -288,7 +295,7 @@ \subsection{Mondrian Conformal Layer}\label{sec:method-cp} Mondrian partition selected by out-of-time interval quality, while grade-based partitions remain the natural governance baseline. -The resulting conformal summary is intentionally more than a scalar coverage number. +The resulting conformal summary is more than a scalar coverage number. The paper-facing metrics include 90\% coverage $0.9297$, 95\% coverage $0.9664$, average 90\% interval width $0.7842$, minimum group 90\% coverage $0.9190$, and 90\% Winkler score $1.1107$ for the promoted conformal winner. Because the outcome is @@ -406,25 +413,43 @@ \section{Theory}\label{sec:theory} guarantee. \begin{theorem}[Distribution-free funded-set risk bound]\label{thm:funded-set-bound} -Suppose the robust constraint holds at the chosen allocation, -$\sum_i w_i u_i(\alpha)\le\tau$. Then -(i) deterministically, $\sum_i w_i Y_i \le \tau + V(\alpha)$; and +Let $B_u(\alpha)=\sum_i w_i u_i(\alpha)$ be the weighted conformal +upper-endpoint budget of the funded set. Then +(i) deterministically, $\sum_i w_i Y_i \le B_u(\alpha) + V(\alpha)$; and (ii) under Assumption~\ref{asm:weighted-validity}, for every $t>0$, $\mathbb{P}(V(\alpha)\ge t)\le\alpha/t$, and in particular \[ -\mathbb{P}\Bigl(\textstyle\sum_i w_i Y_i \;\ge\; \tau+\sqrt{\alpha}\Bigr) +\mathbb{P}\Bigl(\textstyle\sum_i w_i Y_i \;\ge\; B_u(\alpha)+\sqrt{\alpha}\Bigr) \;\le\; \sqrt{\alpha}. \] \end{theorem} \proof{Proof sketch.} Since $Y_i\le u_i(\alpha)+Z_i(\alpha)$ for every loan, part~(i) follows by -taking the $w$-weighted sum and applying the constraint. Part~(ii) is Markov's -inequality applied to the nonnegative variable $V(\alpha)$ with -$\mathbb{E}[V(\alpha)]\le\alpha$ \citep{ghosh2002}, combined with~(i). The -full proof is in Online Supplement Appendix~A. \Halmos +taking the $w$-weighted sum; it is portfolio accounting and needs no +probability. Part~(ii) is Markov's inequality applied to the nonnegative +variable $V(\alpha)$ with $\mathbb{E}[V(\alpha)]\le\alpha$ \citep{ghosh2002}, +combined with~(i). The full proof is in Online Supplement Appendix~A. \Halmos \endproof +\begin{remark}[The optimizer's cap versus the endpoint budget]\label{rem:endpoint-budget} +The robust layer does not constrain $B_u(\alpha)$ directly; it caps the +$\gamma$-blended PD, $\sum_i w_i \tilde p_i(\alpha,\gamma)\le\tau$, with +$\tilde p_i=\hat p_i+\gamma(u_i(\alpha)-\hat p_i)$ and $\gamma\in[0,1]$. Because +$\Gamma_{\mathrm{CP}}(\alpha)=\sum_i w_i(u_i(\alpha)-\hat p_i)$, the endpoint +budget decomposes exactly as +\[ +B_u(\alpha)=\sum_i w_i \tilde p_i(\alpha,\gamma)+(1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha) + \;\le\; \tau+(1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha), +\] +with equality when the cap binds. The term $(1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha)$ +is the conformal robustness premium the optimizer leaves un-internalized at +$\gamma<1$. For the promoted policy ($\tau=0.175$, $\gamma=0.45$, binding cap), +$B_u(0.01)=0.175+0.55\,(0.187987)=0.278393$, so the deterministic bound is +$\sum_i w_i Y_i\le 0.278393+V(0.01)=0.307268$, well above the realized weighted +default rate $0.032875$. +\end{remark} + \begin{remark}[Why $t=\sqrt{\alpha}$, and why Markov]\label{rem:sqrt-alpha} The choice $t=\sqrt{\alpha}$ is made for interpretability, not optimality: it reads cleanly as ``miscoverage exceeds $\sqrt{\alpha}$ with probability at most @@ -487,7 +512,7 @@ \section{Experimental Design}\label{sec:design} }% \end{table} -The out-of-time design is deliberately adversarial: the test window spans an +The out-of-time design is adversarial: the test window spans an expansion (2018--2019) and a regime break (2020 COVID), so coverage and the funded-set certificate are measured under a documented distribution shift rather than on a random split that would let the model see the future. @@ -527,7 +552,7 @@ \section{Experimental Design}\label{sec:design} \subsection{Multi-Dataset External Replication Protocol} -The external-replication layer is deliberately narrower than a new benchmark +The external-replication layer is narrower than a new benchmark campaign. We reuse the frozen CRPTO recipe---CatBoost PD, calibration, train-only WOE/IV feature screening, Mondrian conformal intervals, and the same bound-aware robust LP---on two credit datasets with economic fields. Prosper contributes @@ -572,7 +597,7 @@ \section{Results}\label{sec:results} \end{tabular} \end{table} -The exact certificate in the body is intentionally small. Here ``exact'' means +The exact certificate in the body is a narrow claim. Here ``exact'' means that the quantities are computed directly on the frozen OOT funded set rather than approximated by a surrogate table or visual proxy. The statistical reading still rests on the weighted funded-set validity assumption stated in the theory section; @@ -592,7 +617,7 @@ \section{Results}\label{sec:results} \end{tabular} \end{table} -The promoted policy is deliberately the economic champion inside the exact-safe +The promoted policy is the economic champion inside the exact-safe region, not the tightest certificate available. Table~\ref{tab:champion-comparators} makes that choice auditable rather than implicit. @@ -952,7 +977,7 @@ \section{Discussion}\label{sec:discussion} than assumes. For a risk committee this turns the governance question from ``can we afford robustness?'' into ``how large is the coverage premium on this book?'', which CRPTO answers with a bounded figure per panel. It also tempers the -contribution honestly. The favorable Lending Club value reflects champion +contribution. The favorable Lending Club value reflects champion selection, so the transferable claim is the bounded, measurable premium under blind application, not a universal free lunch; the recipe carries its own price tag wherever it is applied. @@ -963,7 +988,7 @@ \section{Discussion}\label{sec:discussion} static historical panels rather than live deployment evidence. The external panels also make the conditional-coverage caveat concrete: on Freddie the all-group minimum coverage is driven by tiny sparse Mondrian cells, and the high-default red segment misses the strict $\alpha = 0.01$ -gate at $0.9850$; both are reported as honest sensitivity evidence rather than promoted +gate at $0.9850$; both are reported as sensitivity evidence rather than promoted as conditional guarantees. CRPTO does not claim legal fairness certification, because the public data lack direct protected attributes. It does not assert that robust conformal policies dominate all decision-focused learners on regret. diff --git a/paper/supplement_ijds.qmd b/paper/supplement_ijds.qmd index 51d1025..e21eb15 100644 --- a/paper/supplement_ijds.qmd +++ b/paper/supplement_ijds.qmd @@ -67,8 +67,11 @@ the optional tightening arguments. | `x_i` | Funding decision or allocation weight. | | `a_i` | Loan amount or exposure. | | `w_i` | Normalized funded-set weight, `x_i a_i / sum_j x_j a_j`. | -| `gamma` | Robust-policy tuning parameter in the optimization rule. | +| `gamma` | Robust-policy blend parameter in the optimization rule, `gamma in [0,1]`. | +| `p_tilde_i(alpha,gamma)` | Blended PD used by the optimizer, `p_hat_i + gamma (u_i - p_hat_i)`. | +| `tau` | Risk-tolerance cap the optimizer applies to the blended PD, `sum_i w_i p_tilde_i <= tau`. | | `Gamma_CP` | Portfolio-level conformal risk metric after allocation: `sum_i w_i (u_i - p_hat_i)`, with clipping at one on the PD scale. | +| `B_u(alpha)` | Weighted upper-endpoint budget of the funded set, `sum_i w_i u_i(alpha)`; equals `tau + (1-gamma) Gamma_CP` when the cap binds. | | `V(alpha)` | Weighted funded-set miscoverage quantity. | For a fixed allocation, conformal coverage controls the expected interval miss @@ -91,10 +94,14 @@ funded-set validity). We record the full argument here. Funded-set weights satisfy $w_i \geq 0$ and $\sum_i w_i = 1$. Outcomes and endpoints live on the PD scale, $Y_i \in [0,1]$ and $u_i(\alpha) \in [0,1]$, with miscoverage indicators $Z_i(\alpha) = \mathbf{1}\{Y_i > u_i(\alpha)\}$ -and $V(\alpha) = \sum_i w_i Z_i(\alpha)$. The robust constraint holds at the -allocation: $\sum_i w_i u_i(\alpha) \leq \tau$. Probabilities and expectations -are over the exchangeable calibration/test draw conditional on the frozen -recipe, partitions, and allocation rule. +and $V(\alpha) = \sum_i w_i Z_i(\alpha)$. Write +$B_u(\alpha) = \sum_i w_i u_i(\alpha)$ for the weighted upper-endpoint budget of +the funded set. The robust layer caps the $\gamma$-blended PD, +$\sum_i w_i \tilde p_i(\alpha,\gamma) \leq \tau$ with +$\tilde p_i = \hat p_i + \gamma(u_i(\alpha) - \hat p_i)$; the deterministic +identity below does not use this cap and holds for any allocation. Probabilities +and expectations are over the exchangeable calibration/test draw conditional on +the frozen recipe, partitions, and allocation rule. **Step 1 (pointwise domination).** For every funded loan, @@ -107,17 +114,17 @@ indicator. If $Z_i(\alpha) = 1$ then $Y_i \leq 1 \leq u_i(\alpha) + 1$ because $u_i(\alpha) \geq 0$. Both cases give the displayed inequality. **Step 2 (deterministic identity, Theorem 1(i)).** Taking the $w$-weighted -sum of Step 1 and applying the robust constraint, +sum of Step 1, $$ \sum_i w_i Y_i \;\leq\; \sum_i w_i u_i(\alpha) + \sum_i w_i Z_i(\alpha) - \;\leq\; \tau + V(\alpha). + \;=\; B_u(\alpha) + V(\alpha). $$ -This step uses no probability: it is portfolio accounting, valid for every -realization, and it is exactly the identity verified by the frozen exact -funded-set audit. +This step uses no probability and no optimality: it is portfolio accounting, +valid for every realization, and it is exactly the identity verified by the +frozen exact funded-set audit. **Step 3 (Markov, Theorem 1(ii)).** $V(\alpha)$ is a nonnegative random variable under the stated draw, and Assumption 1 gives @@ -130,15 +137,35 @@ P\bigl(V(\alpha) \geq t\bigr) \;\leq\; \frac{E[V(\alpha)]}{t} $$ **Step 4 (combination).** By Step 2, the event -$\{\sum_i w_i Y_i \geq \tau + t\}$ implies $\{V(\alpha) \geq t\}$, so +$\{\sum_i w_i Y_i \geq B_u(\alpha) + t\}$ implies $\{V(\alpha) \geq t\}$, so $$ -P\!\left(\sum_i w_i Y_i \geq \tau + t\right) \;\leq\; \frac{\alpha}{t}, +P\!\left(\sum_i w_i Y_i \geq B_u(\alpha) + t\right) \;\leq\; \frac{\alpha}{t}, $$ and the choice $t = \sqrt{\alpha}$ gives the body statement: -miscoverage budget $\sqrt{\alpha}$ exceeded with probability at most -$\sqrt{\alpha}$. $\blacksquare$ +the endpoint budget $B_u(\alpha)$ is exceeded by more than $\sqrt{\alpha}$ with +probability at most $\sqrt{\alpha}$. $\blacksquare$ + +**Policy-term decomposition.** The optimizer constrains the $\gamma$-blended PD, +not $B_u(\alpha)$ itself. Since +$\sum_i w_i \tilde p_i(\alpha,\gamma) = \sum_i w_i \hat p_i + \gamma\,\Gamma_{\mathrm{CP}}(\alpha)$ +and $B_u(\alpha) = \sum_i w_i \hat p_i + \Gamma_{\mathrm{CP}}(\alpha)$, with +$\Gamma_{\mathrm{CP}}(\alpha) = \sum_i w_i\bigl(u_i(\alpha) - \hat p_i\bigr)$, + +$$ +B_u(\alpha) = \sum_i w_i \tilde p_i(\alpha,\gamma) + (1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha) + \;\leq\; \tau + (1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha), +$$ + +with equality when the cap binds. The promoted policy binds at $\tau = 0.175$ +with $\gamma = 0.45$ and $\Gamma_{\mathrm{CP}}(0.01) = 0.187987$, so +$B_u(0.01) = 0.278393$ and the deterministic bound reads +$\sum_i w_i Y_i \leq 0.278393 + V(0.01) = 0.307268$ against a realized +$\sum_i w_i Y_i = 0.032875$. The slack $(1-\gamma)\,\Gamma_{\mathrm{CP}}(\alpha)$ is +the part of the conformal premium the optimizer does not internalize at +$\gamma < 1$; setting $\gamma = 1$ recovers the tight endpoint cap +$B_u(\alpha) \leq \tau$. These steps leave a clean split. Theorem 1 proves the operational bound under Assumption 1. Proposition A.1 below shows that Markov is the sharp diff --git a/reports/crpto/figures/crpto_fig14_robust_region_heatmap.pdf b/reports/crpto/figures/crpto_fig14_robust_region_heatmap.pdf index 338c797..816f9df 100644 Binary files a/reports/crpto/figures/crpto_fig14_robust_region_heatmap.pdf and b/reports/crpto/figures/crpto_fig14_robust_region_heatmap.pdf differ diff --git a/reports/crpto/figures/crpto_fig14_robust_region_heatmap.png b/reports/crpto/figures/crpto_fig14_robust_region_heatmap.png index 6bfdabc..23c63e3 100644 Binary files a/reports/crpto/figures/crpto_fig14_robust_region_heatmap.png and b/reports/crpto/figures/crpto_fig14_robust_region_heatmap.png differ diff --git a/scripts/build_crpto_journal_package.py b/scripts/build_crpto_journal_package.py index 4d56850..76386d3 100644 --- a/scripts/build_crpto_journal_package.py +++ b/scripts/build_crpto_journal_package.py @@ -16,6 +16,7 @@ import matplotlib.pyplot as plt import numpy as np import pandas as pd +from matplotlib.patches import Rectangle from src.optimization.tail_satisficing_objective import ( entropic_oce, @@ -829,31 +830,45 @@ def _plot_robust_region_heatmap(shortlist: pd.DataFrame, promotion: dict[str, An if not pivot.empty: col = int(np.argmin(np.abs(pivot.columns.to_numpy(dtype=float) - champion_gamma))) row = int(np.argmin(np.abs(pivot.index.to_numpy(dtype=float) - champion_tau))) + # Outline the champion cell and move the star into its corner so the + # centered return value stays legible. White reads against the dark + # (highest-return) cell where the champion sits. + ax.add_patch( + Rectangle( + (col - 0.5, row - 0.5), + 1.0, + 1.0, + fill=False, + edgecolor="white", + linewidth=2.6, + zorder=4, + ) + ) ax.scatter( - col, - row, + col - 0.32, + row - 0.32, marker="*", - s=360, + s=170, color="white", edgecolor="#111111", - linewidth=1.4, - zorder=4, + linewidth=1.1, + zorder=5, ) ax.annotate( "economic\nchampion", (col, row), - xytext=(10, -24), + xytext=(14, -26), textcoords="offset points", color="#263238", fontsize=8.5, fontweight="bold", bbox={ - "boxstyle": "round,pad=0.18", + "boxstyle": "round,pad=0.2", "facecolor": "white", - "edgecolor": "none", - "alpha": 0.82, + "edgecolor": "#555555", + "alpha": 0.92, }, - zorder=5, + zorder=6, ) cbar = fig.colorbar(im, ax=ax) cbar.set_label("Realized return")