Skip to content

Document a worked Codex compaction usage audit #8306

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
loopx-agent wants to merge 1 commit into
base: main
Choose a base branch
from
+45 −0
Open

Document a worked Codex compaction usage audit #8306

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions docs/reference/trajectory-assessment.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1136,6 +1136,51 @@ Registration retains the source hash, and changed or missing sources mark the
saved capture historical. Re-registering a source with this additional
projection can create a new pack identity; old packs are never rewritten.

#### Audit one captured session

Suppose a session records two ordinary requests and one compaction request.
The three response IDs in this synthetic capture give these counts.

| Captured requests | Input | Cached input (subset) | Output |
| --- | ---: | ---: | ---: |
| Two ordinary requests | 140 | 80 | 15 |
| One compaction request | 20 | 10 | 7 |
| All three requests | 160 | 90 | 22 |

If **Session counters** show 140 input and 15 output, **Request coverage** can
verify `ordinary_requests` as the counter basis and show the full 160 input
and 22 output separately. If the counters already show 160 and 22, coverage
can verify `all_requests`. The compaction row is already part of those totals.
The checkpoint's copy of the compaction record shares its response ID. Omit
the copy in the first capture and the session counters still show 140 and 15.
**Request coverage** reports unavailable.
[Reconciliation fixture and readback test](../../tests/capabilities/test_trajectory_assessment_store.py)

Use the same Goal and runtime as registration. Set `PACK_ID` to the pack ID
returned at registration and `OWNER_GOAL` to its owning Goal.

```sh
loopx --format json benchmark trajectory-packs --goal-id "$OWNER_GOAL" |
jq -e --arg pack "$PACK_ID" '
.packs[] | select(.pack_id == $pack) | .solver_usage |
{artifact_sha256, applicability, projection: (.projection |
{session_id, start, end, input_tokens, cached_input_tokens,
output_tokens, request_coverage})}'
```

Keep the pack ID and source hash with the review. Use **Download selected
usage** to capture the relevant Turn line ranges, then check the request and
checkpoint in those ranges of the original JSONL. Replaying model behavior
also requires the original request inputs and runtime.

以一条合成会话为例，两次普通请求合计 140 个输入 token、15 个输出 token，
一次压缩用去 20 个输入和 7 个输出。若会话计数只覆盖普通请求，查看「请求覆盖」
里的全量 160 个输入、22 个输出。若会话计数已覆盖全部请求，压缩用量就是其中
一部分。压缩记录与检查点副本共用一个响应 ID。缺少关联记录时仍可读到会话计数，
请求级核对会显示不可用。复核时记录 pack ID 和源文件哈希，用「下载所选用量」
保存相关轮次的行号，再到同一份 JSONL 中核对。回放模型行为还需要原始请求输入
和运行环境。

### LoopX packet share of model input

Research → Sources and usage also exposes **LoopX input share**. For the same
Expand Down
Loading