Skip to content

apple_incremental_game: size the case pool by SFORGE_CPUS - #41

Closed
findalexli wants to merge 7 commits into
ByteDance-Seed:communityfrom
findalexli:fix/apple-worker-pool-cpus
Closed

findalexli wants to merge 7 commits into
ByteDance-Seed:communityfrom
findalexli:fix/apple-worker-pool-cpus

Conversation

@findalexli

@findalexli findalexli commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Task finding: #44. Part of Known issue 3. The pool mechanism is the one in #40, fixed for vibrating_path_graph_coloring by #36.

What and why

The judge runs 8 cases at once on the official 4 CPUs, under a 30 s wall-clock limit per case. SCHEMA says a parallel grader "sizes the pool with SFORGE_CPUS, not nproc". The pool is now min(8, SFORGE_CPUS, cases), and 4 applies only when the variable is absent. timeout_sec goes from 600 to 1800: at 4 CPUs the longest grade took about 391 s, but at 600 a smaller --judge-cpu-limit lost a real file's score.

Decision for maintainers

This doesn't meet the bar the way #36 did. No real solver we hold loses a case at official settings. Deadline files gain 3.65-10.86 of 100 from extra search time, which is a shift, not a repair, and there is no mapping. Anchors stay as they are, set on the old pool.

Real files do lose cases when CPU is short, and a control using a fixed 20 s of CPU per case gets 2 of 50 on the old judge and 50 of 50 on the new. Your Regression summary's "historical 1 (no time check)" and Known issue 3's submission (3/50 in v1, 45-50/50 now) may be such cases. Could you regrade them on both judges, as you did on #36? If they lose cases on the old judge only, the fix removes a real failure. If not, leaving this under Known issue 3 is a reasonable answer.

Old vs new judge

sforge eval, true 4 CPUs, one judge at a time, both judges on one worker per pair, judge order reversed in the second pair, 18 pairs on three workers. Old key 506688ea9aec, new 71e8a001fed1. Every grade is 50 of 50 OK. Longest runtime_seconds 202.6 old, 375.3 new.

solution.py sha256 Pair 1: old, new Change Pair 2: old, new Change
7a30c585eb86 37.89, 48.75 +10.86 41.80, 51.27 +9.47
28cb741436cc 49.09, 59.23 +10.13 54.85, 63.91 +9.05
9713841d2083 31.75, 38.75 +7.00 33.70, 42.29 +8.59
a430ebe934de 51.99, 58.90 +6.91 51.36, 58.53 +7.17
3d9f0e536d61 67.25, 72.84 +5.59 69.65, 74.01 +4.36
19c47bd860dd 67.48, 72.32 +4.84 69.97, 73.93 +3.96
c56c7b843057 68.33, 71.99 +3.65 68.29, 72.49 +4.20
84027bd4fa94 19.88, 19.88 0.00 19.88, 19.88 0.00
e918edc4d5cd 19.58, 19.58 0.00 19.58, 19.58 0.00

c56c7b843057 at a smaller --judge-cpu-limit, not an official setting, each row on its own worker: limit 2 old 30.88 and 19.85 (48 and 47 of 50 OK), new 73.28 and 73.30 (50 of 50, 721 s); limit 1 new 72.61 and 72.89 (50 of 50, 1443 s).

Control tables, load conditions and archives: https://github.com/findalexli/EdgeBench/tree/8a4d1576af560cbd4074da3524d8df098dd97752.

Checklist for task changes

  • version in task.toml is bumped (any change that can move scores).
  • NOTES.md: the change and its score effect are under Changes. No Known issues entry is fully fixed; issue 3 remains.
  • Old vs new judge on the same submissions, on the same host, results above.
  • New blobs: none.

@findalexli

Copy link
Copy Markdown
Contributor Author

Closing for now. At the official 4 CPUs the real solvers we tested still pass all 50 cases, so we have not shown a solver scored wrongly there. We can reopen if a regrade shows real failures.

@findalexli findalexli closed this Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant