Skip to content

docs(benchmark): cross-task five-arm causal attribution & replan overhead study - #7959

Merged
huangruiteng merged 1 commit into
mainfrom
codex/edgebench-causal-attribution-20261103
Nov 3, 2026
Merged

huangruiteng merged 1 commit into
mainfrom
codex/edgebench-causal-attribution-20261103

Conversation

@loopx-agent

Copy link
Copy Markdown
Collaborator

Summary

Cross-task five-arm empirical causal attribution and trajectory efficiency analysis on EdgeBench Sol 6.1-xhigh across Portfolio Risk Calibration, Graph Clustering Calibration, and Ad Layout Cut Optimization.

Key Evidence & Contributions

  1. Five-Arm Profile Performance Matrix: Compares Official, Native-Goal, Explore Off (Heartbeat-Resume), Explore On (Heartbeat-Explore), and Stride (evidence_conditioned_v1) on Sol 6.1-xhigh.
  2. Feedback Grounding Ratchet: Quantifies the essential stabilizing role of native feedback vs blind catastrophic collapse (falling to 0.00 across 3 arms in Portfolio).
  3. True Causal Inflection Point vs Score Slope: Identifies that the Graph 18h longitudinal peak at agent-10 (0.7240) was driven 100% by probability calibration (-28.4% ECE) and latency speedup (-23.4%) rather than classification F1 (flat at 0.7090 -> 0.7088).
  4. Ineffective Replan Overhead Quantification: Measures 159.87M input tokens (46.0% of run tokens) spent across 7 fruitless post-peak rounds in Graph while hitting the mathematical ceiling of 0.73903.
  5. Reproducible Analysis Module & Test Suite: benchmark/edgebench/causal_attribution.py with deterministic verification and comprehensive pytest coverage (benchmark/tests/test_causal_attribution.py).

Verification

uv run --extra test python -m pytest benchmark/tests/test_causal_attribution.py
uv run python -m benchmark.edgebench.causal_attribution --verify benchmark/edgebench/studies/cross-task-causal-attribution-20261103/data

…head study

Add EdgeBench Sol 6.1-xhigh cross-task five-arm causal attribution analysis,
reproducible data archive, and verification test suite:
- Profile matrix across Official, Native-Goal, Explore Off/On, and Stride
- Causal inflection points vs score slopes in Graph, Ad, and Portfolio
- Replan token waste quantification and theoretical component ceilings
- Comprehensive bilingual technical report and cryptographic index bindings

Signed-off-by: LoopX Agent <337587101+loopx-agent@users.noreply.github.com>
@huangruiteng
huangruiteng merged commit 3539bc0 into main Nov 3, 2026
9 of 30 checks passed
@huangruiteng
huangruiteng deleted the codex/edgebench-causal-attribution-20261103 branch November 3, 2026 04:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants