feat: add ContinuousCasting domain (CuttingOptimization + CuttingOptimizationOnline)

[zzy2007zzy2007]

新增 ContinuousCasting 域：CuttingOptimization + CuttingOptimizationOnline

背景

题材取自 2021 全国大学生数学建模竞赛 D 题的连铸切割工程背景；本贡献将其重新形式化为两个确定性、纯标准库、可验证的 benchmark（ContinuousCasting 域，下辖两个任务）。

1. CuttingOptimization（离线/静态版）

• 目标：把一根连续浇铸的钢坯（含固定 0.8 m 零废段）切成成品段，先最小化报废总长度，再让每块成品贴近客户目标值。
• 求解：一维划分 DP（verification/ref_solver.py）= 理论最优；baseline 为朴素等分。
• 分数（固定 8 实例）：baseline 72.5、全知参考 DP 88.7（各实例 95.8/88.8/91.8/75.2/92.4/87.0/89.4/89.3）。
• agent（统一 low 推理）：openevolve 88.7（= 参考解）、ShinkaEvolve 88.4、AB-MCTS 72.5（= baseline）。
• 单测 35 个。
• 诚实说明：本任务最优可达——强 agent 能推导出精确划分 DP 打到 ~88.7 = 天花板，难度在"推导 DP"而非长程搜索。

2. CuttingOptimizationOnline（在线闭环版）

• 目标：与离线版相同，但换成本质不同的在线信息不对称——agent 只在报废段距切割线 reveal_lead（最终配置 10 m）以内时"得知"该段异常，每个切割决策点只拿到当下可见状态，看不到未来；最后按完整隐藏报废表评分（任何与报废段重叠的切块整块报废）。
• 参照分数（固定 8 实例）：baseline（恒定目标贪心）52.4；全知参考 DP ≈ 76.4（0.02 m 网格近似最优，作为 agent 的经验上限参照）。
• agent 分数（统一 low 推理、各 15 代、每框架 3 次）：openevolve 69.89 ± 0.36、shinkaevolve 70.45 ± 0.64、abmcts 70.95 ± 1.19（全部 9 次 70.43 ± 0.83）。
• 单测 23 个。
• 说明：默认 reveal_lead=10（制造"未知带"、体现信息不对称）。"在线 agent 低于全知最优"属经验观察（3 框架 × 3 次均低于 76.4），非严格下界证明；如需坐实可另加"仅用可见信息的在线 oracle"参照层。

完整性 / 防作弊

• ref_solver.py / generator.py 不进沙箱并被 validator 禁引用（ref_solver、generator、anomaly_seed token）。
• 在线版：固定实例（内嵌完整隐藏报废表）不进沙箱（copy_files.txt 去掉 verification/data/instances），评测器从宿主源目录加载——候选无法读取隐藏报废表作弊。
• 候选子进程剥离所有 FRONTIER_* 及任务前缀环境变量（封宿主侧信道）。
• 运行时生成 + 确定性探针 + EVOLVE-BLOCK 固定区字节比对 + 按名硬编码/绝对路径静态检查。
• 沙箱入口（frontier_eval/evaluator.py）有测试覆盖；verification/multiseed_stat.py 供多轮 mean±std。

文件

• benchmarks/ContinuousCasting/：两个 task 全套（baseline / verification / frontier_eval / Docker / README / Task.md）。
• 仓库级任务索引新增 ContinuousCasting 域两行：TASK_DETAILS.md / TASK_DETAILS_zh-CN.md。

[github-actions]

🤖 AI Code Review (gemini-3-flash-preview)

🇬🇧 English Analysis

1. Executive Summary

• Core Purpose: This PR introduces a new engineering benchmark task, CuttingOptimization, under the ContinuousCasting domain. It is a deterministic 1-D optimization problem inspired by real-world industrial steel production (CUMCM 2021 Problem D), requiring agents to minimize scrap and match customer targets when cutting steel billets with internal defects.
• Modified File Structure & Modifications:
  • TASK_DETAILS.md & TASK_DETAILS_zh-CN.md: Added entries for CuttingOptimization and CuttingOptimizationOnline to the task registry.
  • benchmarks/ContinuousCasting/CuttingOptimization/baseline/solver.py: Provided a naive "equal-split" baseline solver with EVOLVE-BLOCK markers for agent modification.
  • benchmarks/ContinuousCasting/CuttingOptimization/verification/: Implemented a comprehensive verification suite including generator.py (instance generation), simulator.py (scoring logic), validator.py (integrity/anti-cheat checks), and ref_solver.py (a reference DP solution).
  • benchmarks/ContinuousCasting/CuttingOptimization/frontier_eval/: Added framework-specific metadata (evaluator.py, constraints.txt, agent_files.txt) to integrate the task into the frontier_eval system.
  • benchmarks/ContinuousCasting/CuttingOptimization/Task.md & README.md: Provided detailed bilingual documentation on rules, scoring, and execution.

2. AI Content Analysis

• Estimated AI Component: 25%
• Reasoning & Evidence: The code structure follows a highly standardized and modular pattern typical of high-quality engineering, but certain boilerplate sections (e.g., the argparse setup in run_eval.py and the docstring styles in solver.py) exhibit AI-assisted characteristics. However, the domain-specific logic (1-D partition DP, defect isolation logic in simulator.py) and the specific integration with the frontier_eval framework (e.g., EVOLVE-BLOCK markers) indicate significant human architectural design and domain expertise.

3. Engineering & Economic Assessment

• Engineering Reality Check: This is a high-fidelity engineering problem. Unlike "toy" optimization tasks, it accounts for industrial constraints: minimum/maximum cut lengths, mandatory scrap segments (0.8m), and a lexicographic objective function (minimizing scrap is strictly prioritized over target matching). The inclusion of a validator.py that checks for determinism and environment stripping demonstrates a production-grade approach to benchmarking.
• Economic Value: High. It provides a rigorous testbed for evaluating LLMs on industrial process optimization. Solving such problems efficiently directly translates to reduced material waste and improved yield in manufacturing sectors.

4. Quality Assurance

• Verification & Testing:
  • frontier_eval Integration: Yes.
  • task_name: CuttingOptimization (and the registry mentions CuttingOptimizationOnline).
  • Execution & Dependencies: The README.md clearly documents execution commands for both local Python environments and Docker. It explicitly states a "standard library only" dependency policy, which simplifies environment setup.
• Documentation Quality: Excellent. The documentation is bilingual (EN/ZH), includes a clear file layout, detailed scoring rubrics, and reference scores for baselines and agents. No significant grammatical errors or formatting inconsistencies were detected.
• Organizational Structure: The structure is logical and modular. Separating verification logic from baseline code and providing a dedicated data/instances directory ensures scalability and clarity.

5. Security & Privacy Check

• Sensitive Files: Clean. The .gitignore correctly excludes __pycache__ and *.pyc. No .env or API keys were found.
• Absolute Paths: None detected. The validator.py specifically includes a check to forbid absolute paths in submitted code, which is a strong security feature.

---

🇨🇳 中文分析

1. 摘要

• 核心目的: 此 PR 在 ContinuousCasting（连铸）领域下引入了一个新的工程基准任务 CuttingOptimization。这是一个受实际工业钢材生产（2021年国赛 D 题）启发的确定性一维优化问题，要求 Agent 在切割带有内部缺陷的钢坯时，最小化报废量并贴合客户目标值。
• 修改的文件结构与变更摘要:
  • TASK_DETAILS.md & TASK_DETAILS_zh-CN.md: 在任务注册表中增加了 CuttingOptimization 和 CuttingOptimizationOnline 条目。
  • benchmarks/ContinuousCasting/CuttingOptimization/baseline/solver.py: 提供了一个朴素的“等分”基线求解器，并带有供 Agent 修改的 EVOLVE-BLOCK 标记。
  • benchmarks/ContinuousCasting/CuttingOptimization/verification/: 实现了一套完整的验证套件，包括 generator.py（实例生成）、simulator.py（计分逻辑）、validator.py（完整性/防作弊检查）和 ref_solver.py（参考 DP 解）。
  • benchmarks/ContinuousCasting/CuttingOptimization/frontier_eval/: 添加了框架特定的元数据，将任务集成到 frontier_eval 系统中。
  • benchmarks/ContinuousCasting/CuttingOptimization/Task.md & README.md: 提供了关于规则、评分和执行的详细中英文文档。

2. AI 成分分析

• 预估 AI 含量: 25%
• 判断依据与证据: 代码结构遵循高度标准化和模块化的模式，这是高质量工程的典型特征。某些模板化部分（如 run_eval.py 中的 argparse 设置和 solver.py 中的 docstring 风格）表现出 AI 辅助的特征。然而，特定领域的逻辑（一维划分 DP、simulator.py 中的缺陷隔离逻辑）以及与 frontier_eval 框架的特定集成（如 EVOLVE-BLOCK 标记）表明了显著的人工架构设计和领域专业知识。

3. 工程与经济评估

• 工程现实检验: 这是一个高保真度的工程问题。与“玩具”优化任务不同，它考虑了工业约束：最小/最大切割长度、强制报废段（0.8m）以及字典序目标函数（最小化报废严格优先于目标贴合）。包含检查确定性和环境剥离的 validator.py 展示了生产级的基准测试方法。
• 经济价值: 高。它为评估 LLM 在工业流程优化方面的能力提供了一个严谨的测试平台。高效解决此类问题可直接转化为制造业中材料浪费的减少和产量的提高。

4. 质量保证

• 验证与测试:
  • frontier_eval 集成: 是。
  • task_name: CuttingOptimization（注册表中还提到了 CuttingOptimizationOnline）。
  • 运行与依赖: README.md 清晰地记录了本地 Python 环境和 Docker 的运行命令。它明确规定了“仅限标准库”的依赖策略，简化了环境配置。
• 文档质量: 优秀。文档采用中英双语，包含清晰的文件布局、详细的评分细则以及基线和 Agent 的参考分数。未检测到明显的语法错误或格式不一致。
• 组织结构: 结构逻辑清晰且模块化。将 verification 逻辑与 baseline 代码分离，并提供专门的 data/instances 目录，确保了可扩展性和清晰度。

5. 安全与隐私检查

• 敏感文件: 未发现异常。.gitignore 正确排除了 __pycache__ 和 *.pyc。未发现 .env 或 API 密钥。
• 绝对路径: 未检测到。validator.py 专门包含了一项检查，禁止在提交的代码中使用绝对路径，这是一个很强的安全特性。

[y-ji24]

Review — ContinuousCasting

Thank you for the contribution. Overall, ContinuousCasting is a solid and fairly mature benchmark. The simulator, baselines, reference solutions, tests, and Frontier-Eng integration are generally well implemented. I was able to reproduce the baseline scores, and all 58 unit tests pass.

Before merging, I found the following issues that should be addressed.

1. Major — Online hidden defects are accessible in the local evaluation path

The online task relies on information asymmetry: the agent should only see defects within the configured reveal window. However, I constructed a candidate that reads ../verification/data/instances/*.json, computes the clairvoyant DP solution, and replays the cuts.

Running it through the real evaluator produced:

• score = 76.371
• valid = 1.0

This essentially reaches the clairvoyant ceiling of 76.4 while passing the static checks and deterministic probe.

The Docker/sandboxed path correctly excludes the instance files, but the documented local evaluation path does not. Therefore, the hidden-defect guarantee is currently environment-dependent.

Recommendation: Make isolated evaluation mandatory for authoritative scores, or explicitly state that the secrecy guarantee only applies to the sandboxed/Docker path.

2. Major — Runtime-generated instances are not enforced by the default evaluation path

Runtime-generated instances are the main defense against memorizing the fixed public instances, but generation is currently opt-in through environment variables and is not enforced by the task configuration.

The default evaluation therefore uses the 8 public instances. These can be memorized; for the online task, instances can also be fingerprinted through observable properties such as total_length. For the offline task, optimized plans can similarly be embedded using instance characteristics.

Recommendation: Make fresh generated instances mandatory for official evaluation and treat the fixed instances as calibration/smoke-test cases.

3. Major — The documented reference-score command does not reproduce the reference score

The documented verification/evaluate.py ref_solver.py path returns 0.0 because the reference solver is rejected during candidate preflight. The reported reference scores themselves are correct when evaluated through direct simulation.

Recommendation: Provide a dedicated reference-scoring path, or document the exact direct-scoring procedure used to obtain the reported scores.

4. Major — Time-budget semantics are inconsistent with the framework timeout

The online task's 60-second budget is per decision, while the framework-level timeout is 300 seconds for the entire evaluation. An instance may require roughly 10–35 decisions, so a legitimate candidate can exceed the framework timeout.

The offline task can also exceed the 300-second cap once all instances and deterministic probes are included.

Recommendation: Reconcile the task-level and framework-level time budgets and document the expected worst-case evaluation time.

5. Minor — Online EVOLVE-BLOCK protection is not actually enforced

The online evaluator calls check_candidate(prog) without the baseline program, so the fixed-region comparison described by the validator is not applied in this path.

The practical impact is limited, but the documented defense does not fully match the implementation.

Recommendation: Apply the same fixed-region check to the online evaluator, or update the documentation accordingly.

6. Minor — Offline scoring loses tie-break precision

The offline evaluator rounds utilization to two decimal places. Given the 1e-4 penalty weight, some secondary-objective differences can disappear after rounding.

Recommendation: Preserve sufficient precision if the secondary objective is intended to affect the final ranking.

7. Minor — Documentation inconsistencies

A few smaller issues should also be cleaned up:

• Use consistent wording for the reference DP: it is a grid-quantized empirical ceiling rather than a strict theoretical optimum.
• Remove duplicated paragraphs/comments in the online README.
• Report the reference score consistently as approximately 88.71/88.7 rather than mixing 88.7 and 88.72.
• Clearly distinguish fixed-instance calibration results from fresh-instance evaluation results.

Documentation / Collaboration

Please also keep the PR description, task introduction, README-facing explanations, and other repository-facing documentation in English.

Overall

The core benchmark is in good shape, and these issues do not require a major redesign. The main concern is that several parts of the evaluation protocol do not yet fully match the benchmark's stated threat model and reproducibility requirements.

I recommend addressing the two evaluation-integrity issues first, followed by the reference-score and timeout inconsistencies, and then the validator and documentation issues.

Recommendation: REQUEST CHANGES.