Problem
If a prompt change renames a case, the old failing case vanishes and the new one has no baseline. Pass-rate looks fine while coverage silently drops: the diff only intersects prevPass keys.
Pattern
Follow app/api/runs/route.ts (diff logic) and the run page prev-results fetch. Return added[]/removed[] case lists in the POST response and render them on the run page; removed-passing cases optionally fail CI.
Acceptance
Response fields added/removed. Test posts run 2 with one renamed case and asserts both lists are populated.
Problem
If a prompt change renames a case, the old failing case vanishes and the new one has no baseline. Pass-rate looks fine while coverage silently drops: the diff only intersects
prevPasskeys.Pattern
Follow
app/api/runs/route.ts(diff logic) and the run page prev-results fetch. Returnadded[]/removed[]case lists in the POST response and render them on the run page; removed-passing cases optionally fail CI.Acceptance
Response fields
added/removed. Test posts run 2 with one renamed case and asserts both lists are populated.