msprt

Anytime-valid A/B test analyzer that holds the false-positive rate under 1.5% while you peek at the dashboard continuously, where naive fixed-horizon testing leaks to 23% at 15 looks. mSPRT with confidence sequences, CUPED variance reduction (50% on the demo, SE 0.223 to 0.135), a peeking guard, and a reproducible A/A simulation.

Always-valid sequential A/B testing engine: mSPRT confidence sequences make peeking safe by construction, CUPED cuts variance up to 49%, SRM gates bad data. Built-in adversarial peeking harness proves the claim: naive daily peeking hit 27.5% false positives in 2,000 simulations; this engine held 1.7%. All numbers reproducible from committed seeds.

Streamlit web app that turns raw A/B test CSVs into a ship/no-ship decision: two-proportion test, always-valid mSPRT sequential inference, power analysis, and a sample-size planner.

Progressive delivery for prompts and models: sticky canary routing + an always-valid mSPRT sequential test driving automatic promote/rollback. Peeks every request without inflating the false-rollback rate (measured: 0% vs 11.7% for a naive peeked test).