Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -165,3 +165,17 @@
.research-turn-cost-list dl { grid-template-columns: repeat(2, minmax(0, 1fr)); }
.research-turn-costs-sort select { min-height: 44px; }
}

.research-usage-range { border: 1px solid var(--pw-line); border-radius: 6px; padding: 12px; min-width: 0; }
.research-usage-range legend { font-size: var(--text-body); }
.research-usage-range-controls { display: flex; flex-wrap: wrap; align-items: end; gap: 12px; }
.research-usage-range-controls label { display: grid; gap: 4px; width: auto; font-size: 12px; }
.research-usage-range input { width: 96px; min-height: 36px; padding: 4px 8px; border: 1px solid var(--pw-line); border-radius: 6px; background: var(--pw-card); color: var(--pw-text); font: inherit; }
.research-usage-range-summary { font-size: 12px; }
.research-usage-range-summary dl { display: grid; grid-template-columns: repeat(5, minmax(0, 1fr)); gap: 12px; }
.research-usage-range-summary dt, .research-usage-range-summary p { color: var(--pw-muted); }
.research-usage-range-summary dd { margin: 4px 0; font-variant-numeric: tabular-nums; overflow-wrap: anywhere; }
@media (max-width: 600px) {
.research-usage-range input { min-height: 44px; }
.research-usage-range-summary dl { grid-template-columns: repeat(2, minmax(0, 1fr)); }
}
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,7 @@ export function ResearchProgress({pack: p, packs, selectedId, zh, onSelect, onCi
</> : null}
</div> : null}
</div>
{usage && onUsageTurn ? <ResearchTurnCosts key={p.pack_id} usage={usage} zh={zh} onRead={onUsageTurn}/> : null}
{usage && onUsageTurn ? <ResearchTurnCosts key={`${p.pack_id}:${measured?.artifact_sha256}`} usage={usage} source={{pack_id: p.pack_id, goal_id: p.scope.goal_id, artifact_sha256: measured!.artifact_sha256, applicability: measured!.applicability}} zh={zh} onRead={onUsageTurn}/> : null}
{usage && measured ? <details className="research-usage-detail"><summary>{zh ? "计数明细与来源" : "Counter details and source"}</summary>
<dl className="research-usage-basis">
<dt>{zh ? "捕获时间(UTC)" : "Capture window (UTC)"}</dt><dd><time dateTime={usage.start}>{stamp(usage.start)}</time> — <time dateTime={usage.end}>{stamp(usage.end)}</time></dd>
Expand Down
Original file line number Diff line number Diff line change
@@ -1,24 +1,30 @@
import {useId, useState} from "react";
import type {TrajectorySolverUsage} from "../../../../../../loopx/control_plane/capabilities/trajectory_assessment_types.js";
import {downloadReport} from "./report-download";
import {exportResearchUsage, researchUsageRows, selectResearchUsage} from "./research-usage-selection";
import {ContextHelp} from "./context-help";

/** Presentation of existing Turn counters; never estimates missing usage or attributes it to control work. */
export function ResearchTurnCosts({usage, zh, onRead}: {usage: TrajectorySolverUsage; zh: boolean;
export function ResearchTurnCosts({usage, source, zh, onRead}: {usage: TrajectorySolverUsage; zh: boolean;
source: Parameters<typeof exportResearchUsage>[3];
onRead: (turnId: string, origin: HTMLButtonElement) => void}) {
const id = useId();
const [sort, setSort] = useState("input");
const [page, setPage] = useState(0);
const rows: Array<Omit<TrajectorySolverUsage["turns"][number], "usage"> & {usage: TrajectorySolverUsage["turns"][number]["usage"] | null; ordinal: number; incomplete: boolean}> = usage.turns.map((turn, index) => ({...turn, ordinal: index + 1, incomplete: false}));
const tail = usage.incomplete_turn;
if (tail) rows.push({...tail, end: tail.observed_through,
elapsed_seconds: Math.max(0, (Date.parse(tail.observed_through) - Date.parse(tail.start)) / 1000),
ordinal: rows.length + 1, incomplete: true, usage: tail.usage});
const ordered = [...rows].sort((a, b) => {
const rows = researchUsageRows(usage);
const [first, setFirst] = useState("1");
const [last, setLast] = useState(String(rows.length));
const [downloadFailed, setDownloadFailed] = useState(false);
const valid = Number.isInteger(Number(first)) && Number.isInteger(Number(last))
&& Number(first) >= 1 && Number(last) >= Number(first) && Number(last) <= rows.length;
const selection = valid ? selectResearchUsage(usage, Number(first), Number(last)) : null;
const selectedRows = selection?.turns ?? [];
const ordered = [...selectedRows].sort((a, b) => {
const value = (row: typeof a) => sort === "elapsed" ? row.elapsed_seconds
: sort === "time" ? row.ordinal : row.usage?.input_tokens ?? -1;
return (sort === "time" ? value(a) - value(b) : value(b) - value(a)) || a.ordinal - b.ordinal;
});
const size = 8, pages = Math.max(1, Math.ceil(rows.length / size)), current = Math.min(page, pages - 1);
const size = 8, pages = Math.max(1, Math.ceil(selectedRows.length / size)), current = Math.min(page, pages - 1);
const number = (n: number | undefined | null) => n == null ? "—" : n.toLocaleString(zh ? "zh-CN" : "en");
return <details className="research-turn-costs">
<summary>{zh ? "定位耗时与输入最多的轮次" : "Find time and input hotspots"}</summary>
Expand All @@ -35,7 +41,37 @@ export function ResearchTurnCosts({usage, zh, onRead}: {usage: TrajectorySolverU
<p>{zh ? "仅比较当前捕获内的轮次。耗时含工具、等待与编排,不是可节省时间;轮次输入不等于 LoopX 控制输入。金额未采集。" : "Compares Turns within this capture only. Elapsed time includes tools, waiting and orchestration, not avoidable time. Turn input is not LoopX control input. Dollar cost is unmeasured."}</p>
</ContextHelp>
</div>
{rows.length ? <ol className="research-turn-cost-list">
{rows.length ? <fieldset className="research-usage-range">
<legend>{zh ? "分析轮次范围" : "Analyze a Turn range"}</legend>
<div className="research-usage-range-controls">
<label>{zh ? "起始轮次" : "First Turn"}<input type="number" min="1" max={rows.length} step="1" value={first}
aria-invalid={!valid} onChange={event => {setFirst(event.target.value); setPage(0); setDownloadFailed(false);}}/></label>
<label>{zh ? "结束轮次" : "Last Turn"}<input type="number" min="1" max={rows.length} step="1" value={last}
aria-invalid={!valid} onChange={event => {setLast(event.target.value); setPage(0); setDownloadFailed(false);}}/></label>
<button type="button" onClick={() => {setFirst("1"); setLast(String(rows.length)); setPage(0); setDownloadFailed(false);}}>{zh ? "全部轮次" : "All Turns"}</button>
<button type="button" disabled={!valid} onClick={() => {
try {downloadReport("selected-turn-usage.json", source.artifact_sha256,
exportResearchUsage(usage, Number(first), Number(last), source, new Date().toISOString())); setDownloadFailed(false);}
catch {setDownloadFailed(true);}
}}>{zh ? "下载所选用量" : "Download selected usage"}</button>
</div>
{!valid ? <p role="alert">{zh ? `请选择 1 至 ${rows.length} 轮之间的有效范围。` : `Choose a valid range between Turns 1 and ${rows.length}.`}</p> : null}
{selection ? <div className="research-usage-range-summary" role="status">
<p>{zh ? `已选 ${selection.selected_turns} / ${rows.length} 轮 · 完成 ${selection.completed_turns} 轮 · 无计数 ${selection.turns_without_counters} 轮`
: `${selection.selected_turns} / ${rows.length} Turns selected · ${selection.completed_turns} completed · ${selection.turns_without_counters} without counters`}
{source.applicability === "historical" ? (zh ? " · 历史捕获" : " · Historical capture") : ""}</p>
<dl>
<div><dt>{zh ? "已观测输入" : "Observed input"}</dt><dd>{number(selection.observed_input_tokens)}</dd></div>
<div><dt>{zh ? "其中缓存" : "Cached subset"}</dt><dd>{number(selection.observed_cached_input_tokens)}</dd></div>
<div><dt>{zh ? "未缓存" : "Uncached"}</dt><dd>{number(selection.observed_uncached_input_tokens)}</dd></div>
<div><dt>{zh ? "已观测输出" : "Observed output"}</dt><dd>{number(selection.observed_output_tokens)}</dd></div>
<div><dt>{zh ? "轮次耗时合计" : "Summed Turn time"}</dt><dd>{(selection.observed_turn_seconds / 60).toLocaleString(zh ? "zh-CN" : "en", {maximumFractionDigits: 1})} {zh ? "分钟" : "min"}</dd></div>
</dl>
<p>{zh ? "仅合计已观测计数;无计数不记零,未完成轮不代表最终用量。耗时不含轮次间空档。" : "Sums observed counters only. Missing counters are not zero; unfinished Turns are not final usage. Time excludes gaps between Turns."}</p>
</div> : null}
{downloadFailed ? <p role="alert">{zh ? "无法开始下载,请重试。" : "Could not start the download. Try again."}</p> : null}
</fieldset> : null}
{selectedRows.length ? <ol className="research-turn-cost-list">
{ordered.slice(current * size, (current + 1) * size).map(turn => <li key={turn.turn_id}>
<div><strong>{zh ? `第 ${turn.ordinal} 轮` : `Turn ${turn.ordinal}`}</strong>
{turn.incomplete ? <span className="research-usage-scope">{zh ? " · 未完成(仅已观测)" : " · Incomplete (observed only)"}</span> : null}
Expand All @@ -51,7 +87,7 @@ export function ResearchTurnCosts({usage, zh, onRead}: {usage: TrajectorySolverU
<button type="button" aria-label={zh ? `第 ${turn.ordinal} 轮:查看计数证据` : `Turn ${turn.ordinal}: read counter evidence`}
onClick={event => onRead(turn.turn_id, event.currentTarget)}>{zh ? "计数证据" : "Counter evidence"}</button>
</li>)}
</ol> : <p>{zh ? "此捕获没有可读轮次。" : "No captured Turns are available."}</p>}
</ol> : rows.length ? null : <p>{zh ? "此捕获没有可读轮次。" : "No captured Turns are available."}</p>}
{pages > 1 ? <nav aria-label={zh ? "轮次分页" : "Turn pages"}>
<button type="button" disabled={current === 0} onClick={() => setPage(current - 1)}>{zh ? "上一页" : "Previous"}</button>
<span aria-live="polite">{current + 1} / {pages}</span>
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,60 @@
import assert from 'node:assert/strict';
import {test} from 'node:test';
import {selectResearchUsage, exportResearchUsage} from './research-usage-selection.ts';

const turn = (n, input = 100) => ({turn_id: `turn-${n}`, start: `2026-01-01T00:${String(n).padStart(2, '0')}:00Z`,
end: `2026-01-01T00:${String(n).padStart(2, '0')}:10Z`, first_line: n * 3, last_line: n * 3 + 2, elapsed_seconds: 10,
usage: {input_tokens: input, cached_input_tokens: input / 2, uncached_input_tokens: input / 2, output_tokens: 10}});
const usage = {session_id: 'session', method: 'codex_observed_session_counters_v1',
start: '2026-01-01T00:00:00Z', end: '2026-01-01T00:13:00Z', turns: Array.from({length: 12}, (_, i) => turn(i + 1)),
incomplete_turn: {turn_id: 'tail', start: '2026-01-01T00:13:00Z', observed_through: '2026-01-01T00:13:05Z',
first_line: 40, last_line: 41, counter_observations: 0, usage: null}, limitations: ['Only recorded counters.']};

test('inclusive chronological selection sums independent counters and excludes inter-Turn gaps', () => {
const result = selectResearchUsage(usage, 2, 10);
assert.equal(result.selected_turns, 9);
assert.equal(result.observed_input_tokens, 900);
assert.equal(result.observed_cached_input_tokens, 450);
assert.equal(result.observed_uncached_input_tokens, 450);
assert.equal(result.observed_output_tokens, 90);
assert.equal(result.observed_turn_seconds, 90);
assert.deepEqual(result.turns.map(t => t.ordinal), [2,3,4,5,6,7,8,9,10]);
});

test('unknown, partial and measured zero remain distinct', () => {
const tail = selectResearchUsage(usage, 13, 13);
assert.equal(tail.observed_input_tokens, null);
assert.equal(tail.completed_turns, 0);
assert.equal(tail.turns_without_counters, 1);
assert.equal(tail.observed_turn_seconds, 5);
const partial = selectResearchUsage(usage, 12, 13);
assert.equal(partial.observed_input_tokens, 100);
assert.equal(partial.turns_without_counters, 1);
assert.equal(partial.completed_turns, 1);
const zero = {...usage, incomplete_turn: {...usage.incomplete_turn, counter_observations: 1,
usage: {input_tokens: 0, cached_input_tokens: 0, uncached_input_tokens: 0, output_tokens: 0}}};
assert.equal(selectResearchUsage(zero, 13, 13).observed_input_tokens, 0);
assert.equal(selectResearchUsage(zero, 13, 13).turns_without_counters, 0);
});

test('rejects empty, reversed, fractional and out-of-capture selections', () => {
for (const [first, last] of [[0,1],[2,1],[1,14],[1.5,3],[1,NaN],[1,Infinity]]) {
assert.throws(() => selectResearchUsage(usage, first, last), /valid captured Turn range/);
}
assert.throws(() => selectResearchUsage({...usage, turns: [], incomplete_turn: null}, 1, 1));
});

test('portable selection retains provenance and source lines beyond paging without exporting raw source', () => {
const source = {pack_id: 'pack', goal_id: 'goal', artifact_sha256: 'a'.repeat(64), applicability: 'historical'};
const before = JSON.stringify(usage);
const result = JSON.parse(exportResearchUsage(usage, 1, 13, source, '2026-01-02T00:00:00Z'));
assert.equal(result.selection.turns.length, 13);
assert.equal(result.selection.turns[11].first_line, 36);
assert.equal(result.selection.turns[12].usage, null);
assert.equal(result.artifact_sha256, source.artifact_sha256);
assert.equal(result.applicability, 'historical');
assert.equal(result.session_id, 'session');
assert.equal(result.selection.observed_input_tokens, 1200);
assert(result.limitations.includes(usage.limitations[0]));
assert.equal(JSON.stringify(usage), before);
});
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
import type {TrajectorySolverUsage} from "../../../../../../loopx/control_plane/capabilities/trajectory_assessment_types.js";

/** Presentation arithmetic over one already-qualified capture, not a ledger. */
export function researchUsageRows(usage: TrajectorySolverUsage) {
const rows: Array<Omit<TrajectorySolverUsage["turns"][number], "usage"> & {
usage: TrajectorySolverUsage["turns"][number]["usage"] | null; ordinal: number; incomplete: boolean;
}> = usage.turns.map((turn, index) => ({...turn, ordinal: index + 1, incomplete: false}));
const tail = usage.incomplete_turn;
if (tail) rows.push({...tail, end: tail.observed_through,
elapsed_seconds: Math.max(0, (Date.parse(tail.observed_through) - Date.parse(tail.start)) / 1000),
ordinal: rows.length + 1, incomplete: true, usage: tail.usage});
return rows;
}

export function selectResearchUsage(usage: TrajectorySolverUsage, first: number, last: number) {
const all = researchUsageRows(usage);
if (!Number.isInteger(first) || !Number.isInteger(last) || first < 1 || last < first || last > all.length) {
throw new Error("Select a valid captured Turn range");
}
const turns = all.slice(first - 1, last);
const observed = turns.filter(turn => turn.usage !== null);
const sum = (key: keyof NonNullable<typeof turns[number]["usage"]>) => observed.length
? observed.reduce((total, turn) => total + turn.usage![key], 0) : null;
return {
first_turn: first, last_turn: last, captured_turns: all.length,
selected_turns: turns.length, completed_turns: turns.filter(turn => !turn.incomplete).length,
turns_without_counters: turns.length - observed.length,
observed_input_tokens: sum("input_tokens"), observed_cached_input_tokens: sum("cached_input_tokens"),
observed_uncached_input_tokens: sum("uncached_input_tokens"), observed_output_tokens: sum("output_tokens"),
// Sum observed intervals; gaps between Turns are not execution time.
observed_turn_seconds: turns.reduce((total, turn) => total + turn.elapsed_seconds, 0),
start: turns[0].start, observed_through: turns.at(-1)!.end, turns,
};
}

export function exportResearchUsage(usage: TrajectorySolverUsage, first: number, last: number,
source: {pack_id: string; goal_id: string; artifact_sha256: string; applicability: "current" | "historical"},
exportedAt: string) {
return JSON.stringify({
...source, exported_at: exportedAt, session_id: usage.session_id, method: usage.method,
capture_start: usage.start, capture_end: usage.end,
selection: selectResearchUsage(usage, first, last),
limitations: [...usage.limitations,
"Observed counters from one capture, not complete study or monetary cost. Missing counters are null, not zero.",
"Elapsed intervals include tools and waiting. This selection does not establish avoidable cost or causal benefit.",
"Do not add overlapping captures or separate request-reconciliation counters to these totals."],
}, null, 2) + "\n";
}
33 changes: 33 additions & 0 deletions docs/reference/trajectory-assessment.md
Original file line number Diff line number Diff line change
Expand Up @@ -1004,6 +1004,39 @@ TypeScript reducer retain ownership, with Python restricted to scoped file
reading and transport. There is no new telemetry, accounting store or scoring
policy.

### Inspect and save a Turn range

In Research → Sources and usage → **Find time and input hotspots**, enter the
first and last chronological Turn numbers to inspect a contiguous portion of the
selected capture. Sorting changes only the display order; it does not change
range membership. **All Turns** restores the capture. The summary reports
selected/completed Turns, missing counter coverage, observed input (including
its cached subset), output and summed observed Turn intervals. Gaps between
Turns are excluded from that elapsed sum. Unknown counters stay unknown; an
unfinished Turn remains incomplete even when some counters are available.

**Download selected usage** saves every selected Turn, including rows beyond the
current page, as JSON. The file retains the Goal, pack, session, source SHA-256,
method, capture window, historical/current applicability, original source line
ranges and limitations. It contains captured counters rather than raw messages
or a source filesystem location. A failed download can be retried without
changing the selection or Goal state. Changing the pack/source digest resets
the range so a previous selection cannot silently refer to different evidence.

This local reading operation needs no model call, new activation, server write
or CLI/Lark configuration. Close the disclosure to leave the workflow. The file
is a portable reading snapshot, not an import, billing or acceptance receipt.
Do not combine overlapping captures, treat partial observations as final costs,
or infer monetary savings or comparable outcomes from the selected sums.

在研究页的「来源与用量」→「定位耗时与输入最多的轮次」中,输入起止轮次,
查看同一捕获内连续范围的已观测用量。排序只改变显示顺序;「全部轮次」恢复全量。
汇总保留完成状态与无计数覆盖,缺失不记零,未完成轮不当作最终用量;耗时合计不含
轮次间空档。「下载所选用量」导出跨分页的完整所选计数、来源摘要、行号与适用性,
不导出原始消息或本机路径。下载失败可原地重试,切换捕获或来源摘要会重置范围。
该操作不调用模型、不写入 Goal,也不增加 CLI/Lark 配置;关闭详情即可退出。
导出是阅读快照,不是账单、导入或验收凭据,不能据此推断金额、节省或因果收益。

### Reconcile request and compaction coverage

New usage captures also inspect persisted Codex `token_usage_record` and
Expand Down
Loading
Loading