Repository navigation
apple_incremental_game: size the case pool by SFORGE_CPUS - #41
Closed
findalexli wants to merge 7 commits into
Closed
findalexli wants to merge 7 commits into
findalexli wants to merge 7 commits into
Conversation
findalexli
added a commit
to findalexli/EdgeBench
that referenced
this pull request
Nov 4, 2026
Contributor
Author
|
Closing for now. At the official 4 CPUs the real solvers we tested still pass all 50 cases, so we have not shown a solver scored wrongly there. We can reopen if a regrade shows real failures. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Task finding: #44. Part of Known issue 3. The pool mechanism is the one in #40, fixed for
vibrating_path_graph_coloringby #36.What and why
The judge runs 8 cases at once on the official 4 CPUs, under a 30 s wall-clock limit per case. SCHEMA says a parallel grader "sizes the pool with
SFORGE_CPUS, notnproc". The pool is nowmin(8, SFORGE_CPUS, cases), and 4 applies only when the variable is absent.timeout_secgoes from 600 to 1800: at 4 CPUs the longest grade took about 391 s, but at 600 a smaller--judge-cpu-limitlost a real file's score.Decision for maintainers
This doesn't meet the bar the way #36 did. No real solver we hold loses a case at official settings. Deadline files gain 3.65-10.86 of 100 from extra search time, which is a shift, not a repair, and there is no mapping. Anchors stay as they are, set on the old pool.
Real files do lose cases when CPU is short, and a control using a fixed 20 s of CPU per case gets 2 of 50 on the old judge and 50 of 50 on the new. Your Regression summary's "historical 1 (no time check)" and Known issue 3's submission (3/50 in v1, 45-50/50 now) may be such cases. Could you regrade them on both judges, as you did on #36? If they lose cases on the old judge only, the fix removes a real failure. If not, leaving this under Known issue 3 is a reasonable answer.
Old vs new judge
sforge eval, true 4 CPUs, one judge at a time, both judges on one worker per pair, judge order reversed in the second pair, 18 pairs on three workers. Old key506688ea9aec, new71e8a001fed1. Every grade is 50 of 50 OK. Longestruntime_seconds202.6 old, 375.3 new.solution.pysha2567a30c585eb8628cb741436cc9713841d2083a430ebe934de3d9f0e536d6119c47bd860ddc56c7b84305784027bd4fa94e918edc4d5cdc56c7b843057at a smaller--judge-cpu-limit, not an official setting, each row on its own worker: limit 2 old 30.88 and 19.85 (48 and 47 of 50 OK), new 73.28 and 73.30 (50 of 50, 721 s); limit 1 new 72.61 and 72.89 (50 of 50, 1443 s).Control tables, load conditions and archives: https://github.com/findalexli/EdgeBench/tree/8a4d1576af560cbd4074da3524d8df098dd97752.
Checklist for task changes
versionintask.tomlis bumped (any change that can move scores).NOTES.md: the change and its score effect are under Changes. No Known issues entry is fully fixed; issue 3 remains.