Skip to content

v1.0 fixes: consistent risk reporting, input handling, solvers, benchmarks and Docker - #11

Merged
Kiguli merged 15 commits into
masterfrom
v1.0-fixes
Oct 6, 2026
Merged

Kiguli merged 15 commits into
masterfrom
v1.0-fixes

Conversation

@Kiguli

@Kiguli Kiguli commented Oct 6, 2026 •

Copy link
Copy Markdown
Owner

Fixes found while documenting the tool for v1.0, so that the tool and the benchmark files match the paper. Nothing here changes the published results: all 82 tests pass, and the benchmarks give the same complexity, risk bounds and degeneracy as before (details below).

Merging deploys to the live site and rebuilds the Docker image (existing workflows).

Risk reporting

  • The MOSEK licence-upload path and the normal path now share one solve function (src/solve_form.py). The MOSEK path certified each run at β/(n_τ·n_ρ) but reported 1 − β/(n_τ·n_ρ). It also reported the cost including the τ and ρ terms, and left out n_attempts.
  • Degenerate runs: the results table shows the lower bound as N/A and the graphs leave it out, as the paper says.
  • Hidden parameters: the selected formulation decides which of τ and ρ apply. Before, a robust program still applied a τ left in the hidden form fields.
  • Confidence parameter: a missing or out-of-range β is now an error. It used to be treated as β = 0, which gave an upper bound of 1.

Input handling

  • Plain numbers: numbers in matrices (JSON, MAT, paste boxes) no longer give HTTP 500.
  • SymPy-style expressions: names such as sin, exp, sqrt, pi and ^ now work; math.sin still works and every other name is still rejected.
  • Empty fields: optional program fields may be left empty ([], "", null).
  • Readable errors: input errors come back as {"error": ...} and are shown as a message.
  • .MAT download: it works when a run in a sweep failed.

Solvers

  • The τ‖x − x̄‖ term is only added when τ > 0. It used to add a second-order cone to plain LPs, so LP-only solvers failed and SCS and CLARABEL were less accurate.
  • Norms: the regularization is a vector p-norm, so any p ≥ 1 works.
  • Silent shape errors fixed: a 1-D b(δ) or h, integer SDP keys, and a column x_ref in SDP no longer change the constraints or solution.
  • Q and empty inputs: Q is checked with a tolerance, an empty SDP Q is allowed, and empty deltas raises ValueError.
  • Solver table: corrected against CVXPY's metadata, and installed solvers missing from the table get their types from CVXPY.
  • quantify_risk: it validates its inputs and enables float64 only inside the function.

Web interface

  • Slow solves on the hosted app: after 30 s, a note says problems of this size are better solved locally and links to the install guide.
  • Display: tooltips for Risk Bounds, Total Constraints and Optimization Time, full confidence digits, and correct cell help.
  • Total Constraints now includes the hard constraints.

Benchmarks

  • run.py --solver: every run.py takes --solver (default MOSEK), and prints the actual confidence level.
  • Program files reproduce run.py. Inventory and portfolio had a nonzero rho although the penalty is built into c.
  • Portfolio ρ: the cost vector now uses ρ = 1/((1−0.95)·1255) = 0.015936 instead of 0.04, left over from N = 500. The published solution is also optimal for 0.015936, so only the objective value changes, from 0.053356 to 0.051337.
  • READMEs match metrics.json and the actual files.
  • Extra packages: scripts that need packages outside requirements.txt say which ones.
  • Leftover folder: SDP_LPV_stability_3d_numeric is removed. It duplicated the main LPV numeric files, which its test now uses.

Docker, README and screenshots

  • Docker: the image is built for linux/arm64 as well as amd64, GUNICORN_CMD_ARGS works, and the duplicate docker.yml workflow is removed.
  • README: corrections, take_screenshots.py works again, and the screenshots are regenerated.

Verification

  • Tests: 82 pass, the original 52 plus 30 new regression tests. They also pass in CI.
  • Paper examples: the three paper examples give identical numbers. Pasting their JSON from the paper PDF into the app reproduces them.
  • Browser checks: the N/A display, the slow-solve note and the β check were each checked in a real browser.
  • Docker: the image built on this branch contains linux/amd64 and linux/arm64.
  • Benchmarks: all 12 were run with CLARABEL before and after the changes (MOSEK's licence was unavailable). 10 give identical k, risk bounds and degeneracy. On the ellipsoid, CLARABEL went from a wrong k = 557 to the published k = 5 with [0, 0.0518]. Growth Bound is still running.

Kiguli and others added 15 commits October 5, 2026 21:30
The relaxation penalty of these two benchmarks is already built into
the cost vector c (shared-slack formulation), and run.py solves with
rho = 0. The program JSONs carried rho = 100.0 and 0.04, so uploading
them in the web app added a second penalty and gave a different
result (inventory cost 43012.6 instead of 45099.67). With rho = 0 the
web app selects Robust and reproduces run.py and results/metrics.json.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Each run.py hard-coded solver='MOSEK', so readers without a MOSEK
licence could not rerun the benchmarks. All 12 scripts now accept
--solver (default MOSEK, which produced the published results), e.g.

    python benchmarks/LP_half_width_2d/run.py --solver CLARABEL

The "Solving ... with" / "Solved with" / "failed" messages print the
solver actually used, and the Usage line in each docstring shows the
option. The existing fallbacks (SCS for the SDPs, CLARABEL for the
robot navigation QP) are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
matplotlib, scikit-learn, yfinance and mat73 are deliberately not in
requirements.txt. Each benchmark script that imports one of them now
says so in a comment below the shebang, with the pip command:

- matplotlib: every plot.py, plot_data_only.py and animate.py
- scikit-learn: SDP_covariance_wine_15d and SDP_ellipsoid_enclosure_15d
  generate.py
- yfinance: LP_portfolio_cvar_13d/generate.py
- mat73: QP_radiation_therapy_50d/generate.py

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Expected Results and the risk bounds quoted in Results now match
  results/metrics.json (e.g. half width 0.2073 -> 0.2085, Iris
  0.1430 -> 0.1441, LPV 0.1853 -> 0.1858). Also corrected: growth
  bound cost and the sign of M* and b*, radiation therapy k = 5 with
  degeneracy (was k = 6, none), the portfolio weights and CVaR, and
  the LPV matrix P.
- File listings match the files in each folder: the per-matrix CSVs
  (A_d.csv, c.csv, F_0.csv, ...) no longer exist, program_symbolic.json
  was listed twice (once as "MATLAB"), and program_numeric.json,
  scenarios_numeric.csv and a few scripts and figures were missing.
- Manual method uses the real option names (Robust, Regularization,
  Relaxation) and points to the fields of program_symbolic.json
  instead of the removed CSV files. Inventory and portfolio use Robust
  with rho = 0 because their penalty is already in c.
- The one-shot method mentions the numeric program and scenarios.
- Usage documents the new run.py --solver option.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
In parameters.txt, confidence = 0.999999 is the confidence level
1 - beta (run.py and plot.py compute beta = 1 - confidence), whereas
the program JSON's "confidence" field holds beta itself (1e-06). Add a
comment line above each entry. Values and parsing are unchanged; every
reader of parameters.txt skips lines starting with '#'.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The MOSEK licence-upload path had its own copy of the solve loop, which
had drifted from the app's: it certified each run at beta/(n_tau*n_rho)
but reported 1 - beta/(n_tau*n_rho), reported the full objective
(including the tau and rho terms) as the cost, and had no n_attempts.
Both paths now call src/solve_form.solve_form, so they report the same
results. While moving the loop:

- Hidden fields are ignored: the selected formulation decides which of
  tau and rho apply, so e.g. a robust program no longer applies or
  sweeps a tau that is still in the form.
- A missing or out-of-range confidence parameter is an error instead of
  being treated as beta = 0, and p must be at least 1.
- Input errors are returned as {"error": ...} with HTTP 400.
- Total Constraints counts the hard constraints as well as the N
  scenario constraints.
- Download .MAT stores a failed run's missing results as NaN instead of
  failing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…vers

- Leave the tau*||x - x_ref|| term out when tau = 0. With p = 2 it added
  a second-order cone, so LP-only solvers failed on plain LPs.
- Use the vector p-norm of x - x_ref, so any p >= 1 works (only 1, 2,
  inf and fro worked for LP/QP before); results are unchanged for those.
- Reshape b(delta) and h to columns: a 1-D array was silently broadcast
  to an (m, m) matrix and changed the constraints.
- SDP: accept integer matrix keys (key 0 was read as the last variable),
  reshape a column x_ref (it was broadcast to a matrix), and allow an
  empty Q.
- Check Q for symmetry and semidefiniteness with a small tolerance, so
  rank-deficient matrices are accepted.
- Empty deltas raise the documented ValueError instead of IndexError.
- Correct the solver capability table against CVXPY's metadata (ECOS,
  GUROBI, CPLEX and SCIP cannot solve SDPs; CBC and SCIPY are LP-only;
  QPALM is not a CVXPY solver) and derive the types of unlisted solvers
  from CVXPY instead of offering all three.
- get_norm_types no longer lists 'nuc'; load_file reads JSON without an
  invalid orient, accepts comma-separated .txt files, and its docstring
  says it returns None on errors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Plain numbers (from JSON paste boxes, program files or MAT files) are
  accepted; they crashed the solve with HTTP 500 before.
- sin, cos, exp, log, sqrt, abs, pi, e and similar names work without
  the math. prefix, and ^ means power, as in SymPy. ^ is replaced by **
  before parsing so it keeps the right precedence. math.sin still works,
  and every other name is still rejected.
- A scalar scenario can be indexed as delta[0].
- Constant matrix collections (E) evaluate entries like other matrices,
  so "1/2" works there too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Raise ValueError for N < 1, k outside 0..N, or beta outside (0, 1);
  k > N used to return (0, 1) and N = 0 divided by zero.
- Enable JAX float64 only inside quantify_risk (jax.experimental
  .enable_x64) instead of for the whole Python process.
- Return plain floats (epsU was the integer 1 when k = N).
- Remove the unused quantify_conf stub.

The computed bounds are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- When degeneracy is detected, the results table shows the lower risk
  bound as N/A and the graphs leave it out, as described in the paper.
- On the hosted app, a solve that runs for more than 30 s shows a note
  that problems of this size are better solved locally, with a link to
  the local install guide.
- The confidence parameter must be between 0 and 1 before solving.
- Server input errors are shown as a message instead of a raw response.
- The confidence row shows enough digits (0.999999, not 1.0000), and the
  Risk Bounds, Total Constraints and Optimization Time rows have tooltips.
- The cell help describes the accepted expressions (sin(), exp(), ^, ...)
  instead of promising SymPy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…orkflow

- Build the published image for linux/arm64 as well as linux/amd64.
- The default worker count is set with GUNICORN_CMD_ARGS, so
  -e GUNICORN_CMD_ARGS="--workers 4" now works (the command line used
  to override it).
- Remove docker.yml: docker-publish.yml already builds on every push to
  master and on release tags, and both pushed :latest.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README: Python 3.10 or newer, MOSEK is installed by requirements.txt
  but needs a licence (and is the recommended solver), the dialog button
  is "I have a local license", and p is any number >= 1, inf or fro.
- take_screenshots.py works again (repository-relative paths, current
  program file and option names) and writes screenshots/scrn_*.png,
  which are regenerated from the current interface.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Delete benchmarks/SDP_LPV_stability_3d_numeric: its files duplicate
  SDP_LPV_stability_3d's program_numeric.json and scenarios_numeric.csv
  (and its "symbolic" program had no F_d). The numeric-mode test now
  uses the main LPV files.
- run.py scripts print the actual confidence level (99.9999%) instead
  of a hard-coded "99%".
- README: drop the c.csv files that no longer exist from the benchmark
  layout, and mention --solver and the extra packages for plotting.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The program files still embedded rho = 0.04 (from an earlier N = 500
setup) in the cost vector, while parameters.txt, the README and the
paper use 0.015936 for N = 1255. The published solution (weights,
alpha, k = 3, risk bounds [0, 0.0203]) is also optimal with
rho = 0.015936, so the results are unchanged; only the objective value
in metrics.json and the README, which includes rho * sum(zeta), becomes
0.051337 (was 0.053356).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Kiguli
Kiguli merged commit 4a0a8aa into master Oct 6, 2026
2 checks passed
@Kiguli
Kiguli deleted the v1.0-fixes branch October 6, 2026 03:16

This branch was successfully deployed

1 active deployment
github-pages — 4a0a8aa4 Deployed Oct 6, 2026 by Kiguli via deploy #17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant