Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 13 additions & 0 deletions .github/workflows/verification.yml
Original file line number Diff line number Diff line change
Expand Up @@ -216,14 +216,27 @@ jobs:
mimo --config examples/cli_agent/mimo-harness.json report --run examples/cli_agent/.mimo/runs/recovered-repair
mimo export --run examples/cli_agent/.mimo/runs/recovered-repair --output cli-repair-evidence.zip
mimo gate --bundle cli-repair-evidence.zip
- name: Prepare portable user-kit dependencies and rehearse clean installation
run: |
python -m pip download --only-binary=:all: --platform any --implementation py --abi none --dest .mimo/kit-wheels pip==25.0.1 pytest==8.4.2 iniconfig==2.1.0 packaging==25.0 pluggy==1.6.0 pygments==2.19.2 exceptiongroup==1.3.0 tomli==2.2.1 colorama==0.4.6 typing_extensions==4.15.0
python -c "import pathlib,shutil; shutil.copy2(next(pathlib.Path('.mimo/dist').glob('*.whl')), '.mimo/kit-wheels')"
python tests/ci_user_kit.py --wheelhouse .mimo/kit-wheels --output .mimo/user-kit-rehearsal
- uses: actions/upload-artifact@v4
if: always()
with:
name: cli-pytest-${{ matrix.os }}
path: |
cli-pytest-results.xml
.mimo/dist/*
.mimo/user-kit-rehearsal/kit/
.mimo/user-kit-rehearsal/rehearsal.json
include-hidden-files: true
- uses: actions/upload-artifact@v4
with:
name: independent-user-kit-${{ matrix.os }}
path: .mimo/user-kit-rehearsal/kit/
include-hidden-files: true
if-no-files-found: error

unit-and-smoke:
runs-on: ubuntu-latest
Expand Down
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -258,6 +258,8 @@ mimo readiness

`readiness` returns `3` while external acceptance is pending. Use `trial-init`/`trial-run` for an explicitly budgeted live model trial and `walkthrough-init` for each independent new user. Blank templates and deterministic controls do not count as those observations. See the [acceptance guide](docs/PRODUCT_ACCEPTANCE_CN.md) and [support/upgrade boundaries](docs/SUPPORT_AND_UPGRADES.md).

v0.12 adds [`walkthrough-pack`](docs/INDEPENDENT_USER_KIT_CN.md): distribute one exact wheel, offline Python dependencies and three blank user-study slots. A cross-platform bootstrap creates a private environment, verifies the installed engine and preserves feedback when resuming interrupted setup. Actual user observations remain separate.

### Continue an interrupted run

```bash
Expand Down
2 changes: 2 additions & 0 deletions README_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -215,6 +215,8 @@ v0.11 已增加原生 Node.js 与通用 JUnit 的逐用例验证,可通过 `ms

真实模型试跑、新用户反馈和最终验收分别使用 `trial-init`/`trial-run`、`walkthrough-init` 和 `readiness`。执行 `mimo readiness` 可查看缺项;返回 `3` 表示尚未齐备,不会把测试模板当作真实试用。详见[中文产品验收指南](docs/PRODUCT_ACCEPTANCE_CN.md)和[支持范围与升级说明](docs/SUPPORT_AND_UPGRADES.md)。

v0.12 新增 [`walkthrough-pack` 独立用户试用包](docs/INDEPENDENT_USER_KIT_CN.md):三位使用者共用同一份 wheel 和离线 Python 依赖,用一条命令创建独立环境;安装中断后可以恢复,已填写的反馈会保留。自动安装不算真人验收。

### 6.3 初始化并检查项目

先使用明确的离线模式。下面命令会创建 `mimo-harness.json` 和 `.mimo/models.json`,不会调用模型 API。
Expand Down
2 changes: 1 addition & 1 deletion docs/ACCEPTANCE_MATRIX.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ The two actual-Codex tests require `MIMO_CODEX_BINARY` and the exact pinned exec
- Signed/remote attestation, richer evidence migration and organization-level enforced workflow policies are outside the current unsigned bundle format. The opt-in PR template still requires maintainer-configured required checks/review protection.
- Generic CLI Agent repair, the pinned Codex event adapter and two real Python defect controls have automated gates. Live paid-provider/model repair validation still requires credentials and an explicit spending allowance. Opaque external Agent tokens/costs remain unavailable; invocation/time caps are not billing caps.
- Broader language/framework and defect diversity beyond the documented native Node/JUnit subset; Keil/firmware/hardware behavior remain separately scoped.
- Maintainer-selected license and at least three independent new-user walkthroughs. Wheel/sdist checks, upgrade/support notes, trial/feedback generation and source-bound CI collection are implemented; generated templates are not completed external evidence.
- Maintainer-selected license and at least three independent new-user walkthroughs. Wheel/sdist checks, upgrade/support notes, exact-wheel user kits, offline install/setup recovery, trial/feedback generation and source-bound CI collection are implemented; generated templates and automated installer rehearsals are not completed external evidence.
- One final clean checkout/install run across the supported matrix, all failure-injection scenarios, evidence audit and scope review after the last functional change.

Passing an intermediate batch is not the final completeness sign-off. Unavailable environments, skipped tests and external/manual gates must remain visible in the release record.
Expand Down
40 changes: 40 additions & 0 deletions docs/INDEPENDENT_USER_KIT_CN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# 把同一份可验证安装包交给三位新用户

v0.12 beta 提供 `walkthrough-pack`,把当前引擎对应的 wheel、固定依赖、跨平台安装脚本、中文指南和三份空白反馈放在同一个目录。三位使用者安装完全相同的引擎文件,避免各自重建 wheel 带来身份差异。生成材料或自动演练不计为真人试用。

## 维护者准备

在准备交付的干净 checkout 中构建并安装 wheel:

```bash
python -m pip install build==1.3.0
python -m build --outdir .mimo/dist
python -m pip install --force-reinstall --no-deps .mimo/dist/mimo_codeharness-0.12.0-py3-none-any.whl
python -m pip download --only-binary=:all: --platform any --implementation py --abi none --dest .mimo/kit-wheels pip==25.0.1 pytest==8.4.2 iniconfig==2.1.0 packaging==25.0 pluggy==1.6.0 pygments==2.19.2 exceptiongroup==1.3.0 tomli==2.2.1 colorama==0.4.6 typing_extensions==4.15.0
python -c "import shutil; shutil.copy2('.mimo/dist/mimo_codeharness-0.12.0-py3-none-any.whl', '.mimo/kit-wheels')"
mimo walkthrough-pack --wheelhouse .mimo/kit-wheels --output .mimo/user-kit
```

准备依赖需要联网;输出目录必须是新目录。wheel 必须与当前 CLI 的版本和引擎字节一致;缺依赖、重复版本、平台专用 wheel 和不同引擎都会被拒绝。安装时使用这些确切 wheel,不从在线索引解析新依赖。

CI 的 Linux/Windows `cli-install` 任务从真实构建产物生成试用包,并演练新环境安装、拒绝覆盖、半成品虚拟环境恢复、保留反馈、真实 boltons 对照验收、手动恢复和迁移后的 ZIP 门禁。选择一份可信 CI 的 `independent-user-kit-ubuntu-latest` 或 `independent-user-kit-windows-latest` artifact 下载;解压后即可看到 bootstrap.py 和 GUIDE_CN.md。原样分发同一份包给三位使用者,不要混合不同任务中重建的 wheel。技术演练记录 rehearsal.json 另保留在 `cli-pytest-*` artifact 中。

## 新用户操作

使用者只需预装 Python 3.10+ 和 Git,在试用包目录运行:

```text
python bootstrap.py --participant user-01 --output my-study
```

另外两人选择 user-02、user-03。这些名称只是材料分配槽位,真实匿名 tester_id 由使用者自己填写。

安装脚本先校验传输文件,在新目录创建 `.venv`,仅离线安装包内依赖并核对实际安装的引擎;不修改系统 Python,不需要激活环境。随后进入 my-study,按照 GUIDE_CN.md 使用 `python mimo.py ...` 完成流程。Git 下载固定上游源码需要访问 GitHub,离线安装不等于整个真实案例都不需要网络。

安装失败或中断后,使用相同参数加 `--resume`。每次尝试保留独立安装日志,技术安装状态写入 setup.json。已有 feedback.json 和报告不会被自动填写或覆盖;无法恢复时保留原目录并使用新输出目录。

试用者提交自己填写的 feedback.json、自己运行生成的 evidence.zip 及必要日志。反馈仍要经过 [readiness](PRODUCT_ACCEPTANCE_CN.md) 检查和人工独立性复核。包中哈希和 CI JSON 均未签名,用于检查版本/传输一致性;请从可信维护者和 CI 取得材料。

汇总验收的维护者也应安装这份包中的同一 wheel,再运行 readiness。仅版本号相同、分别重建的代码文件未必具有相同的引擎身份。

本流程不调用模型、不设置模型额度,也不替维护者选项目许可证。[真实模型和最终发布验收](PRODUCT_ACCEPTANCE_CN.md) 仍单独进行。
2 changes: 2 additions & 0 deletions docs/PRODUCT_ACCEPTANCE_CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,8 @@ mimo trial-run --plan .mimo/live-trial-plan.json --output .mimo/live-trial-001

## 2. 至少三位独立新用户试用

推荐先按[独立用户试用包指南](INDEPENDENT_USER_KIT_CN.md)准备同一份 wheel 和离线依赖,让三位使用者获得一致的引擎。下面的 `walkthrough-init` 仍可用于已经安装好相同版本的环境。

为每位未参与实现的使用者生成一份独立材料:

```bash
Expand Down
2 changes: 1 addition & 1 deletion docs/PRODUCT_PLAN.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ Priorities are ordered by user dependency, with a concrete exit condition:
| P2b · adapter implemented; live trial pending | Ready-to-use vendor adapter, then live model validation | Pinned Codex CLI adapter and config generator; actual-binary/local-server CI. A real model still requires available credentials, an explicit spending allowance and a separately recorded outcome |
| P3a · implemented | Two independent real Python defect cases | boltons and more-itertools pinned source/fix commits; reproduced target assertions, reference acceptance, regression rejection, fresh recovery and portable evidence via `case-init`/`case-check` |
| P3b · implemented | Native Node and generic JUnit case checks | Actual Node 22/24, typed JUnit contract, ms/boltons reference and regression controls, fixed acceptance, fresh recovery and exact portable proof; no zero-exit-only PASS |
| P4 · workflow prepared; real users pending | New-user workflow and actionable diagnosis | Chinese guide and blank feedback generation, duplicate/stale/unfinished evidence checks. At least three actual independent new users still need to complete the workflow and resolve observed issues |
| P4 · workflow prepared; real users pending | New-user workflow and actionable diagnosis | Exact-engine user kit, offline dependency installation, interrupted setup recovery, Chinese guides and three blank feedback slots; duplicate/stale/unfinished evidence checks. At least three actual independent new users still need to complete the workflow and resolve observed issues |
| P5 · automation/docs prepared; final sign-off pending | Release and final completeness sign-off | Wheel/sdist, support/upgrade notes, exact-source CI record and readiness gate. Maintainer license choice, live trial, independent users and final manual review remain external gates |

P2–P4 may be scoped into smaller batches, but a capability is not advertised as certified before its exit condition is met. Existing `mimo verify` already accepts patches produced by external tools without needing a tool-specific Agent integration.
Expand Down
5 changes: 3 additions & 2 deletions docs/SUPPORT_AND_UPGRADES.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# Supported scope and upgrades

v0.11 remains a beta. Automated release evidence and a stable release sign-off are separate. The package has no mandatory third-party Python runtime dependencies; configured projects/runners need their own explicitly prepared dependencies.
v0.12 remains a beta. Automated release evidence and a stable release sign-off are separate. The package has no mandatory third-party Python runtime dependencies; configured projects/runners need their own explicitly prepared dependencies.

| Component | Automated coverage | Boundary |
|---|---|---|
Expand All @@ -12,6 +12,7 @@ v0.11 remains a beta. Automated release evidence and a stable release sign-off a
| Linux Docker | Actual non-root isolated baseline/candidate/Agent tests; Node/JUnit included | Prebuilt image, exact resolved image ID, explicit network/resources; no local fallback |
| Codex | CLI 0.163.0 native-binary protocol on Linux/Windows | Explicit model/key; live model/tool quality and billing require observed trials |
| Portable evidence | Relocation, strict gate, HTML/JSON/JUnit/log linkage; exact Node/JUnit identity | Unsigned integrity evidence, not hostile-party authentication |
| Independent user kit | One exact wheel, portable offline Python dependencies; actual Python 3.12 installation/recovery on Linux/Windows | Python/Git must already be installed; fetching pinned upstream cases needs GitHub; automated rehearsal is not a human trial |
| Embedded/other languages | Existing scans and explicit commands | No claim of full Keil firmware build, hardware validity or certified Go/Rust/Jest/Vitest acceptance |

## Install and upgrade
Expand All @@ -22,7 +23,7 @@ Keep the previous checkout/tag and all run directories before upgrading. Do not

Recovery requires the exact source/test/configuration, task, engine, executable/environment and Docker identity that produced the original run. A version upgrade, changed reporter or changed Node binary can invalidate recovery. Restore the matching old environment to resume an old run, or start a new run; do not edit fingerprints/accounting to force reuse. Previously completed portable bundles remain available for inspection, subject to their evidence format and verifier compatibility.

v0.11 does not migrate cached tests or silently reuse old PASS. Resume recreates baseline/candidate checks; completed proposals are reused only when inputs match. Native Node executable bytes are included in local recovery identity; Docker runtime bytes are bound through image identity.
The current version does not migrate cached tests or silently reuse old PASS. Resume recreates baseline/candidate checks; completed proposals are reused only when inputs match. Native Node executable bytes are included in local recovery identity; Docker runtime bytes are bound through image identity.

## Final release record

Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"

[project]
name = "mimo-codeharness"
version = "0.11.0"
version = "0.12.0"
description = "Evidence-backed repository and code-agent patch verification harness"
readme = "README.md"
requires-python = ">=3.10"
Expand Down
2 changes: 1 addition & 1 deletion src/mimo_harness/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Installable command-line interface for MiMo-CodeHarness."""

__version__ = "0.11.0"
__version__ = "0.12.0"

5 changes: 5 additions & 0 deletions src/mimo_harness/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -1017,6 +1017,11 @@ def build_parser() -> argparse.ArgumentParser:
walkthrough = sub.add_parser("walkthrough-init", help="Create an independent-user guide and blank feedback record.")
walkthrough.add_argument("--output", required=True)
walkthrough.set_defaults(handler=command_walkthrough_init)
from mimo_harness.user_kit import command_walkthrough_pack
pack = sub.add_parser("walkthrough-pack", help="Package one exact wheel and offline dependencies for three independent user walkthroughs.")
pack.add_argument("--wheelhouse", required=True, help="Directory containing the matching harness wheel and all portable pytest dependencies")
pack.add_argument("--output", required=True, help="New kit directory; existing files are preserved")
pack.set_defaults(handler=command_walkthrough_pack)
readiness = sub.add_parser("readiness", help="Check source-bound product evidence; missing external gates remain pending.")
readiness.add_argument("--repository", default=".")
readiness.add_argument("--trial", help="Completed trial.json")
Expand Down
8 changes: 6 additions & 2 deletions src/mimo_harness/manifests.py
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ def environment_identity(execution: dict, checks, salt: str, runtime_home: Path)
"environment_sha256": digest([salt, env])}


def engine_identity() -> str:
def engine_inventory() -> dict:
# A wheel lives in site-packages beside unrelated packages; a checkout may
# contain legacy scripts that are not distributed. Hash the same executable
# components in both layouts, using module names instead of install paths.
Expand All @@ -191,7 +191,11 @@ def engine_identity() -> str:
for name in ("run_mimo_codeharness_v02", "scoring_v03", "verification_v04"):
spec = importlib.util.find_spec(name)
code[name + ".py"] = file_digest(Path(spec.origin)) if spec and spec.origin else None
return digest(code)
return code


def engine_identity() -> str:
return digest(engine_inventory())


class RecoverySource:
Expand Down
2 changes: 1 addition & 1 deletion src/mimo_harness/product_acceptance.py
Original file line number Diff line number Diff line change
Expand Up @@ -133,7 +133,7 @@ def command_walkthrough_init(args):
3. 执行 `mimo --config .mimo/user-case/mimo-harness.json doctor`。
4. 执行 `mimo case-check --directory .mimo/user-case --name first`。阅读参考 PASS、故意回归 FAIL 和恢复重跑的证据;整个对照验收应成功。
5. 再用相同名称运行一次,确认拒绝覆盖;记录提示是否足以解决问题。换一个新名称可重新验收。
6. 将 `results/first/reference.zip` 复制到其他目录,执行 `mimo gate --bundle ZIP路径 --json`;解压后打开 report.html。
6. 将 `.mimo/user-case/results/first/reference.zip` 复制到其他目录,执行 `mimo gate --bundle ZIP路径 --json`;解压后打开 report.html。
7. 填写匿名 tester_id、观察时间、环境、各步骤是否完成、遇到的问题。只在实际独立试用后设置 independence_confirmed=true。issues 中每个问题记录 description 和 resolved。
8. evidence_bundle 填写你自己运行产生的参考 ZIP 路径(相对 feedback.json 所在目录,或绝对路径)。不同试用者需要各自的运行证据。

Expand Down
Loading
Loading