You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
benchmark-memory outgrew its 60-minute budget: raise it or shrink the suite? #627
Owner decision needed (CLAUDE.md rule 7). Found while fixing the stuck weekly benchmarks in #626.
ci/check_benchmark_memory_workflow.py enforces build-and-measuretimeout-minutes == 60 as a hard budget (with negative controls for 61 and 45). The suite no longer fits:
run
date
run external memory suite step
33315579361
08-30
43m11s, passed
33820641684
09-04
48m52s, passed
35512774807
09-20
cut off at 52m56s by the 60 min job timeout (job 1h0m22s)
The Bun row (commit 2f2791f, 09-03) made it five allocators. Builds add about 7 minutes (native libs 6m08s, suite 54s). The 09-06 and 09-13 runs were cut off the same way.
#626 fixes the separate cause of the 09-27, 10-04 and 10-25 cancellations, a cron collision with benchmark-latency in the shared concurrency group, and leaves the timeout at 60.
Options:
Raise the budget to about 90 minutes, which gives roughly 25% headroom over the measured trend, and update the checker's bounds.
Keep 60 and cut the suite's blocks or cells (for example, fewer blocks per allocator) to fit.
Split the suite across two jobs.
Also worth doing in any case: the suite prints nothing while it runs, so a hang can't be told apart from slowness. A per-block progress line would fix that.
Activity