Skip to content

docs(research): link retained benchmark trajectories #8235

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
huangruiteng merged 1 commit into from
Nov 7, 2026
Merged

docs(research): link retained benchmark trajectories #8235

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions apps/presentation/dashboard/src/data/fixtures/edgebench-ad-20261023-history.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@
"Native and local measurement scopes remain distinct. Cancelled or invalid runs do not qualify a scientific comparison."
],
"redactions": [],
"trajectory_archive": {
"url": "https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/resolve/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/ad_placement_optimization.zip",
"sha256": "45ce26e44a59587b5a086d3aabde9bba9899b184abd6a21029f75c8b6695a44b",
"manifest_url": "https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/blob/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/manifest.json",
"redactions_url": "https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/blob/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/redactions.json"
},
"boundary": "Display-state events only. No authority or credential store, raw session transcript, hidden evaluator, workspace or model weights are included."
},
"source_counts": {
Expand Down
6 changes: 6 additions & 0 deletions .../presentation/dashboard/src/data/fixtures/edgebench-grid-context-on-20261103-history.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,12 @@
"These runs include operational interventions and are not a qualified causal comparison. Local and native measurements retain separate scopes."
],
"redactions": [],
"trajectory_archive": {
"url": "https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/resolve/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/grid_turing_robot.zip",
"sha256": "ea147c9ae7173e3f1ce9ca6e0bbda5e0103e2be035b8e6d0658dd35a2d282c8d",
"manifest_url": "https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/blob/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/manifest.json",
"redactions_url": "https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/blob/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/redactions.json"
},
"boundary": "Display-state events only. No execution/authority store, credentials, raw session transcript, hidden evaluator, workspace or model weights are included."
},
"source_counts": {
Expand Down
6 changes: 6 additions & 0 deletions apps/presentation/dashboard/src/data/fixtures/edgebench-schema-20261025-history.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,12 @@
"Native and local measurement scopes remain distinct. Cancelled or invalid runs do not qualify a scientific comparison."
],
"redactions": [],
"trajectory_archive": {
"url": "https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/resolve/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/schemathesis_config_modernization.zip",
"sha256": "e892d3b9569d84c7fff8ce80c2a0ae9f9a801d5f745eb980bfc918835327b2ea",
"manifest_url": "https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/blob/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/manifest.json",
"redactions_url": "https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/blob/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/redactions.json"
},
"boundary": "Display-state events only. No authority or credential store, raw session transcript, hidden evaluator, workspace or model weights are included."
},
"source_counts": {
Expand Down
30 changes: 30 additions & 0 deletions examples/personal-workspace-browser/research-history.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -92,6 +92,36 @@ the same preview's `--fixture` option; their original Goal IDs stay intact.
These display captures do not include their complete session trajectories or
qualify installed App adoption, paid analysis or a scientific comparison.

## Captured session archives

The [fixed capture set](https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/tree/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories) links the three real display
fixtures above to all four retained solver-session files:

| Display fixture | Captured session archive |
| --- | --- |
| Grid context on | [Grid ZIP](https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/resolve/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/grid_turing_robot.zip) |
| Ad placement | [Ad ZIP](https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/resolve/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/ad_placement_optimization.zip) |
| Schemathesis | [Schema ZIP](https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/resolve/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/schemathesis_config_modernization.zip) |

Each fixture's `provenance.trajectory_archive` pins the archive and its SHA-256.
The [manifest](https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/blob/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/manifest.json) binds native session IDs,
original/published/compressed hashes, record counts and capture variants; the
[redaction audit](https://huggingface.co/datasets/huangrt01/loopx-edgebench-research-checkpoints-20261015/blob/a97e88f951deb6827d7051a59927860e9e4c4476/20261107-real-benchmark-trajectories/redactions.json) identifies every
removed opaque encrypted field. All plaintext values and unchanged JSONL lines
were compared with the source, and all seven published files were read back by
SHA-256. The 382 Explore records remain byte-identical.

The Schema receiver's alternate runtime capture is a strict prefix missing
56 tail records; the longer worker-session file is retained. Earlier Schema
session history keeps its original identity. Complete captured files do not
establish whole-run coverage or active duration. Invalid/cancelled statuses,
unknown adoption/cost and unqualified scientific comparisons remain explicit.
The earlier checkpoint release below remains available at its original version.

中文：三例现有真实展示样例现在关联到同一固定 HF 版本的完整已捕获会话，保留
原文、身份、取消/无效状态及未知覆盖。Schema 较短副本缺少的 56 条尾部记录已
由较长原文件保留；清单和逐字段脱敏审计可查，不据此推断科学收益或完整运行覆盖。

## Earlier research checkpoints

### Saved analysis reference
Expand Down
Loading