Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions TASK_DETAILS.md
Original file line number Diff line number Diff line change
Expand Up @@ -351,5 +351,10 @@ We welcome new engineering problem ideas — even without complete verification
<td><code>DiffSimThermalControl</code></td>
<td>Process optimization in additive manufacturing via differentiable simulation</td>
</tr>
<tr>
<td><b>WastewaterTreatment</b></td>
<td><code>BSM1AerationControl</code></td>
<td>Feedback control of activated-sludge aeration and internal recycle across dry, rain, and storm operation</td>
</tr>
</tbody>
</table>
5 changes: 5 additions & 0 deletions TASK_DETAILS_zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -351,5 +351,10 @@ Frontier-Eng 目前已覆盖以下领域的任务。每个任务均配有可运
<td><code>DiffSimThermalControl</code></td>
<td>基于可微仿真的增材制造工艺优化</td>
</tr>
<tr>
<td><b>WastewaterTreatment</b></td>
<td><code>BSM1AerationControl</code></td>
<td>在旱天、降雨和暴雨工况下反馈控制活性污泥曝气与内回流</td>
</tr>
</tbody>
</table>
53 changes: 53 additions & 0 deletions benchmarks/WastewaterTreatment/BSM1AerationControl/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,53 @@
# BSM1 Aeration Control

Design a deterministic feedback controller for an activated-sludge plant derived from the
IWA Benchmark Simulation Model No. 1 (BSM1). The controller sets oxygen-transfer coefficients
in the three aerobic reactors and the internal recycle flow. It must balance effluent quality,
energy, compliance, and actuator smoothness across dry, rain, and storm scenarios.

Edit only the EVOLVE-BLOCK in `scripts/init.py`, preserving:

```python
def reset_controller(scenario: dict) -> None: ...
def control(observation: dict) -> dict: ...
```

## Setup

The task is offline and CPU-only:

```bash
python -m pip install -r verification/requirements.txt
```

No raw IWA influent files are redistributed. The evaluator generates deterministic trajectories
from published BSM1 averages and weather-event descriptions. A complete baseline run takes about
45 seconds on a laptop.

## Direct evaluation

```bash
python verification/evaluator.py scripts/init.py --metrics-out metrics.json --artifacts-out artifacts.json
```

## Regression tests

```bash
python -m unittest discover -s verification -p "test_*.py" -v
```

## Unified evaluation

From the repository root:

```bash
python -m frontier_eval task=unified task.benchmark=WastewaterTreatment/BSM1AerationControl algorithm=openevolve algorithm.iterations=0
```

The ranking metric is `combined_score` (higher is better). `metrics.json` contains the aggregate
score and validity fields; `artifacts.json` contains scenario-level engineering metrics and daily
effluent samples. Candidate code runs in a separate, bounded JSON-lines worker process. This is
process isolation, not an operating-system security sandbox.

See `Task.md` for the exact interface and scoring model, and
`references/design_notes.md` for provenance, validation, and modelling limitations.
44 changes: 44 additions & 0 deletions benchmarks/WastewaterTreatment/BSM1AerationControl/README_zh-CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# BSM1 曝气控制

为基于 IWA 一号基准仿真模型(BSM1)的活性污泥处理厂设计确定性反馈控制器。控制器设置三个好氧反应池的氧传质系数与内回流量,并在旱天、降雨和暴雨场景下权衡出水质量、能耗、达标情况和执行器平滑性。

只修改 `scripts/init.py` 中的 EVOLVE-BLOCK,并保持以下接口:

```python
def reset_controller(scenario: dict) -> None: ...
def control(observation: dict) -> dict: ...
```

## 环境安装

本任务离线运行且只需要 CPU:

```bash
python -m pip install -r verification/requirements.txt
```

仓库不再分发 IWA 原始进水文件。评测器依据已发表的 BSM1 平均值与天气事件说明生成确定性轨迹。在普通笔记本上完整基准评测约需 45 秒。

## 直接评测

```bash
python verification/evaluator.py scripts/init.py --metrics-out metrics.json --artifacts-out artifacts.json
```

## 回归测试

```bash
python -m unittest discover -s verification -p "test_*.py" -v
```

## 统一评测

在仓库根目录运行:

```bash
python -m frontier_eval task=unified task.benchmark=WastewaterTreatment/BSM1AerationControl algorithm=openevolve algorithm.iterations=0
```

排名指标为 `combined_score`,越高越好。`metrics.json` 给出聚合分数与有效性;`artifacts.json` 给出逐场景工程指标和每日出水样本。候选代码运行在带时限的独立 JSON-lines 工作进程中;这是进程隔离,不是操作系统级安全沙箱。

完整接口和评分模型见 `Task_zh-CN.md`,来源、验证和模型限制见 `references/design_notes.md`。
92 changes: 92 additions & 0 deletions benchmarks/WastewaterTreatment/BSM1AerationControl/Task.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,92 @@
# Task: BSM1 Aeration Control

## Engineering setting

Activated-sludge plants must remove carbon and nitrogen while limiting blower and pumping energy.
Influent flow and composition vary diurnally and become more difficult during rain and storm
events. This task exposes feedback control rather than a one-shot parameter fit: every 15 minutes,
the policy observes noisy process measurements and selects aeration and internal recycle settings.

## Process model and scenarios

The evaluator implements the standard BSM1 layout: five completely mixed ASM1 reactors (two
anoxic, three aerobic) followed by a ten-layer Takacs secondary settler. Fixed return-activated
sludge and waste-sludge flows are 18,446 and 385 m3/day. The plant is simulated for 14 days and
the final seven days are scored.

Three deterministic scenarios share the same diurnal base load:

- `dry`: diurnal and weekend load variation;
- `rain`: a sustained dilution-water event from day 8.35 to day 10.44;
- `storm`: two shorter hydraulic pulses centred near days 8.87 and 11.18.

The rain and storm series are deterministic IWA-derived engineering trajectories, not byte-for-byte
copies of the official BSM1 dynamic influent files. Sensor noise uses fixed, scenario-specific seeds.

## Observation

`control(observation)` receives only current or past information:

- `scenario_id`, `weather`, `time_day`, and `step_minutes`;
- influent flow and ammonium;
- dissolved oxygen in reactors 3, 4, and 5;
- nitrate in reactor 2;
- effluent ammonium and total nitrogen;
- `previous_action`.

The dissolved-oxygen, nitrate, and ammonium measurements contain deterministic sensor noise. The
policy does not receive future influent, process state arrays, random seeds, or evaluator internals.

## Action interface and hard constraints

Return exactly these four finite numeric fields:

```python
{
"kla3_per_day": float, # [0, 360]
"kla4_per_day": float, # [0, 360]
"kla5_per_day": float, # [0, 360]
"internal_recycle_m3_per_day": float, # [0, 92230]
}
```

At one 15-minute step, each KLa may change by at most 120/day and internal recycle by at most
30,000 m3/day. Missing/extra fields, booleans, non-finite values, range errors, slew errors,
exceptions, protocol errors, or timeouts make the complete candidate invalid with score zero.
`reset_controller(scenario)` must reset all candidate-owned state before each scenario.

## Metrics and score

For every scored step, the evaluator computes the official BSM1 effluent quality index (EQI),
aeration energy, pumping energy, mixing energy, actuator switching, and normalized exceedance of
five standard limits: NH4-N 4, total nitrogen 18, COD 100, TSS 30, and BOD5 10 g/m3.

Each objective is mapped to a dimensionless utility:

```text
Uq = exp(-EQI / 6000)
Ua = exp(-aeration_energy / 5000)
Up = exp(-pumping_energy / 1200)
Um = exp(-mixing_energy / 600)
Us = exp(-4 * switching_index)

base = 100 * (0.45 Uq + 0.20 Ua + 0.10 Up + 0.10 Um + 0.15 Us)
scenario_score = base * exp(-6 * violation_index)
```

The aggregate rewards typical and worst-case behavior:

```text
combined_score = 0.75 * mean(scenario_scores) + 0.25 * min(scenario_scores)
```

The score is absolute and contains no frozen candidate baseline. A fixed published BSM1 operating
point is supplied only as editable starter code and as a reproducible comparison in the result log.

## Candidate process boundary

The evaluator copies the candidate to a temporary directory and imports it in a persistent worker.
Communication is JSON-only; import, each call, cumulative response time, and response size are
bounded. Candidate stdout is discarded and credentials are removed from its environment. This
prevents ordinary Python-level mutation of the parent evaluator; it is not a sandbox against
hostile native code or unrestricted filesystem/network access.
58 changes: 58 additions & 0 deletions benchmarks/WastewaterTreatment/BSM1AerationControl/Task_zh-CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
# 任务:BSM1 曝气控制

## 工程背景

活性污泥处理厂既要去除有机物和氮,又要控制鼓风与泵送能耗。进水流量和组分具有日周期变化,降雨与暴雨会进一步增加水力冲击。本任务是反馈控制而非一次性参数拟合:策略每 15 分钟读取带噪过程测量,并设置曝气与内回流。

## 工艺模型与场景

评测器实现标准 BSM1 布局:五个完全混合 ASM1 反应池(前两个缺氧、后三个好氧)以及十层 Takacs 二沉池。固定污泥回流和排泥流量分别为 18,446 与 385 m3/day。每个场景仿真 14 天,最后 7 天计分。

三个确定性场景共享同一日周期基础负荷:

- `dry`:日周期和周末负荷变化;
- `rain`:第 8.35 至 10.44 天的持续稀释水事件;
- `storm`:中心约在第 8.87 与 11.18 天的两次短时水力脉冲。

降雨与暴雨序列是依据 IWA 资料构造的确定性工程轨迹,并非官方 BSM1 动态进水文件的逐字节副本。传感器噪声使用固定且按场景区分的种子。

## 观测

`control(observation)` 只接收当前或历史信息:场景、天气、时间、步长;进水流量与氨氮;3—5 号池溶解氧;2 号池硝酸盐;出水氨氮与总氮;以及 `previous_action`。溶解氧、硝酸盐和氨氮测量含确定性噪声。策略不能获得未来进水、完整过程状态、随机种子或评测器内部信息。

## 动作接口与硬约束

必须恰好返回四个有限数值字段:

```python
{
"kla3_per_day": float, # [0, 360]
"kla4_per_day": float, # [0, 360]
"kla5_per_day": float, # [0, 360]
"internal_recycle_m3_per_day": float, # [0, 92230]
}
```

相邻 15 分钟内,每个 KLa 的最大变化为 120/day,内回流最大变化为 30,000 m3/day。字段缺失或多余、布尔值、非有限值、越界、变化率超限、异常、协议错误或超时都会使整个候选无效并得零分。`reset_controller(scenario)` 必须在每个场景前重置策略自身状态。

## 指标与评分

计分期的每一步都会计算 BSM1 出水质量指数(EQI)、曝气/泵送/搅拌能耗、执行器切换,以及五项标准限值的归一化超标量:NH4-N 4、总氮 18、COD 100、TSS 30、BOD5 10 g/m3。

```text
Uq = exp(-EQI / 6000)
Ua = exp(-曝气能耗 / 5000)
Up = exp(-泵送能耗 / 1200)
Um = exp(-搅拌能耗 / 600)
Us = exp(-4 * switching_index)

base = 100 * (0.45 Uq + 0.20 Ua + 0.10 Up + 0.10 Um + 0.15 Us)
场景分 = base * exp(-6 * violation_index)
combined_score = 0.75 * 场景均分 + 0.25 * 最差场景分
```

该分数是绝对工程分,不依赖冻结的候选基线。公开的 BSM1 固定工况仅作为可编辑起始程序和结果日志中的可复现实验对照。

## 候选进程边界

评测器把候选复制到临时目录并在持久工作进程中导入。通信仅使用 JSON;导入、单次调用、累计响应时间和响应大小均受限。候选标准输出会被丢弃,环境中的凭据会被移除。该边界可阻止普通 Python 代码修改父评测器,但不能替代针对恶意原生代码、文件系统或网络访问的操作系统沙箱。
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
Direct command:
python verification/evaluator.py scripts/init.py --metrics-out metrics.json --artifacts-out artifacts.json

Environment:
Windows 11, Python 3.12.13, CPU-only

Result:
=== BSM1 Aeration Control ===
scenario=dry score=53.457 eqi=5488.86 ae=3341.39 violations=0.011330
scenario=rain score=44.609 eqi=7535.45 ae=3341.39 violations=0.025575
scenario=storm score=48.513 eqi=6780.93 ae=3341.39 violations=0.017011
---
completed_scenarios: 3/3
diagnostic_score: 47.7973
combined_score: 47.7973

Regression:
python -m unittest discover -s verification -p "test_*.py" -v
Ran 12 tests in 3.266s
OK

Unified evaluation:
python -m frontier_eval task=unified task.benchmark=WastewaterTreatment/BSM1AerationControl algorithm=openevolve algorithm.iterations=0
combined_score=47.7973, valid=1.0000, completed_scenarios=3.0000, benchmark_returncode=0.0000
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
"""Reference fixed-operating-point policy for BSM1AerationControl."""

from __future__ import annotations


def reset_controller(scenario: dict) -> None:
pass


def control(observation: dict) -> dict:
return {
"kla3_per_day": 240.0,
"kla4_per_day": 240.0,
"kla5_per_day": 84.0,
"internal_recycle_m3_per_day": 55338.0,
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
README.md
README_zh-CN.md
Task.md
Task_zh-CN.md
scripts/init.py
references/design_notes.md
frontier_eval/constraints.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
metrics.json
artifacts.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
scripts/init.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
BSM1 aeration-control constraints:
1) Edit only `scripts/init.py` inside the EVOLVE-BLOCK markers.
2) Keep `reset_controller(scenario) -> None` and `control(observation) -> dict` working.
3) Return exactly the four finite numeric fields documented in Task.md.
4) Keep each KLa in [0, 360]/day and internal recycle in [0, 92230] m3/day.
5) Respect per-step slew limits: 120/day for each KLa and 30000 m3/day for recycle.
6) Reset all candidate-owned state before each dry, rain, or storm scenario.
7) Keep the controller deterministic, self-contained, and within the 0.10-second call limit.
8) Do not read, write, import, or modify evaluator, model, result, or environment-secret files.
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
README.md
README_zh-CN.md
Task.md
Task_zh-CN.md
scripts/init.py
verification/evaluator.py
verification/bsm1_model.py
verification/policy_runtime.py
verification/policy_worker.py
verification/requirements.txt
references/config.json
references/BSD-3-Clause-bsm2-python.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
{python} verification/evaluator.py {candidate} --metrics-out metrics.json --artifacts-out artifacts.json
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
scripts/init.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
README.md
README_zh-CN.md
Task.md
Task_zh-CN.md
verification/evaluator.py
verification/bsm1_model.py
verification/policy_runtime.py
verification/policy_worker.py
verification/test_bsm1_model.py
verification/test_evaluator.py
verification/test_policy_runtime.py
verification/requirements.txt
references/config.json
references/design_notes.md
references/BSD-3-Clause-bsm2-python.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
BSD 3-Clause License

Copyright (c) 2025, FAU-EVT

Redistribution and use in source and binary forms, with or without
modification, are permitted provided that the following conditions are met:

1. Redistributions of source code must retain the above copyright notice, this
list of conditions and the following disclaimer.

2. Redistributions in binary form must reproduce the above copyright notice,
this list of conditions and the following disclaimer in the documentation
and/or other materials provided with the distribution.

3. Neither the name of the copyright holder nor the names of its
contributors may be used to endorse or promote products derived from
this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS"
AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
Loading
Loading