apple_incremental_game: size the case pool by SFORGE_CPUS

edgebench/tasks/apple_incremental_game/NOTES.md

@@ -2,7 +2,7 @@
 
 AtCoder Heuristic Contest 058 "Apple Incremental Game", in ALE-Bench style. The agent writes
 `solution.py` (stdin -> stdout): 500 turns, each either strengthening one of the 4×10 machines or
-doing nothing. The judge runs it on 50 fixed cases (8 at a time, 30 s wall clock each), scores each
+doing nothing. The judge runs it on 50 fixed cases (`min(8, SFORGE_CPUS)` at a time, 30 s wall clock each), scores each
 output with `tools/bin/tester` (`round(1e5 · log2(S))`, S = apples after T turns), and sums the
 per-case scores of the `OK` cases (failed cases count 0). Raw score = that sum (maximize). The
 rescale is the v1 one (`piecewise_max`, baseline 45000000).
@@ -13,7 +13,8 @@ rescale is the v1 one (`piecewise_max`, baseline 45000000).
   authors' repository (not public), identical by sha256 and mode to the published v1 images. The
   tools are Python scripts.
 - `environment/workspace/solution.py` (starter) and `judge/eval_apple_incremental_game.py`:
-  byte-identical to v1's inline scripts and to the published v1 images. `judge/grade.py` is new and
+  byte-identical to v1's inline scripts and to the published v1 images (the eval script until 1.1.0).
+  `judge/grade.py` is new and
   writes `result.json` with v1's scoring.
 - `instruction.md`: v1's prompt, verbatim. Base: `bases/python310`.
 
@@ -47,9 +48,22 @@ timeouts depending on host load, on both images. Historical 3 uses unseeded rand
 extra pairs of runs, both images give overlapping ranges with means 0.05 M apart. The other
 time-limited searches differ by under 1 point on 0..100.
 
+## Changes
+
+### 1.1.0
+
+The judge ran 8 cases at once on the official 4 CPUs, against SCHEMA's rule that a parallel grader
+sizes its pool with `SFORGE_CPUS`, so a deadline search got about half the CPU of 4 at once. The pool
+is now `min(8, SFORGE_CPUS, cases)`; `SFORGE_CPUS` must be a positive integer, and 4 applies only when
+it is absent. Deterministic solutions that finish on both judges keep their scores. Seven deadline-based
+files gained 3.65-10.86 of 100 across fourteen pairs at true 4 CPUs, with no case lost on either judge.
+There is no mapping from old scores, so regrade before comparing with v1 entries. `timeout_sec` goes
+from 600 to 1800 for a smaller `--judge-cpu-limit`: one file took 1443 s at limit 1, and at 4 CPUs the
+longest grade took about 391 s. Known issue 3 stays. Source, Deviations and Regression summary describe 1.0.0.
+
 ## Known issues
 
-Kept as in v1, so scores stay comparable. Fixing any of them changes scores and bumps `version`.
+Kept as in v1. Fixing any of them changes scores and bumps `version`.
 
 1. **The submission can forge its score through the tester's error message.** The eval script
    takes a score from any tester output line that looks like `Score = …` and ignores the tester's

edgebench/tasks/apple_incremental_game/judge/Dockerfile

@@ -15,7 +15,7 @@
 #   9 rm -rf .git && git init && git config user.* && add && commit -m init
 #                                                                -> kept, author/dates pinned to v1's commit
 #  10 heredoc > /tmp/eval_apple_incremental_game.py; chmod +x    -> COPY judge/eval_apple_incremental_game.py /judge/
-#                                                                   (byte-identical, moved out of /tmp)
+#                                                                   (v1's, except the pool size, moved out of /tmp)
 #   v1 eval_cmd + parser score_sum                               -> judge/grade.py (task.toml [judge].cmd)
 
 # Base = bases/python310 (sforge build passes it as BASE_IMAGE).

edgebench/tasks/apple_incremental_game/judge/eval_apple_incremental_game.py

@@ -42,7 +42,7 @@ def run_case(args):
     os.chdir(WORK_DIR)
     case_files = sorted(f for f in glob.glob(os.path.join(CASES_DIR, "*")) if os.path.isfile(f))
     results = {}
-    with ProcessPoolExecutor(max_workers=min(8, len(case_files))) as pool:
+    with ProcessPoolExecutor(max_workers=min(8, int(os.environ.get("SFORGE_CPUS", 4)), len(case_files))) as pool:
         futures = {pool.submit(run_case, (i, cf)): i for i, cf in enumerate(case_files)}
         for fut in as_completed(futures):
             cid, status, score = fut.result()

edgebench/tasks/apple_incremental_game/judge/grade.py

@@ -1,11 +1,11 @@
 #!/usr/bin/env python3
 """Grader for apple_incremental_game (Verified judge contract, see SCHEMA.md).
 
-Reproduces v1 exactly:
+Keeps v1's command, parser and score:
   v1 eval_cmd : cd /home/workspace/apple_incremental_game && python3 /tmp/eval_apple_incremental_game.py
   v1 parser   : score_sum  (sforge/harness/log_parsers/score_sum.py)
   v1 score    : first `TOTAL_SCORE <n>` in the eval output (sforge/harness/grading.py:extract_score)
-/judge/eval_apple_incremental_game.py is byte-identical to the v1 /tmp/eval_apple_incremental_game.py.
+/judge/eval_apple_incremental_game.py is the v1 /tmp/eval_apple_incremental_game.py except for its pool size (1.1.0).
 This wrapper only runs it and translates its stdout into /opt/sforge/result.json.
 stdout = logs only.
 """

edgebench/tasks/apple_incremental_game/task.toml

@@ -4,7 +4,7 @@ name = "Apple Incremental Game"
 category = "Combinatorial Optimization"
 language = "Python"
 description = "Decide each turn whether to invest in machines or collect output in an incremental production game, balancing short-term gains against long-horizon compounding."
-version = "1.0.0"
+version = "1.1.0"
 status = "migrated"
 
 [environment]
@@ -22,12 +22,12 @@ exclude = []
 
 [judge]
 cmd = "python3 /judge/grade.py"
-timeout_sec = 600
+timeout_sec = 1800
 cpus = 4
 memory = "8g"
 # Official leaderboard limits (v1 kept them only in the experiment YAMLs).
-# eval_apple_incremental_game.py runs the 50 cases 8 at a time (ProcessPoolExecutor,
-# max_workers=8), each `python3 solution.py` under a 30 s wall-clock timeout. Scores of
+# eval_apple_incremental_game.py runs the 50 cases min(8, SFORGE_CPUS) at a time,
+# each `python3 solution.py` under a 30 s wall-clock timeout. Scores of
 # time-limited (anytime) solutions depend on host CPU speed and load; see NOTES.md.
 
 [score]