diff --git a/CHANGELOG.md b/CHANGELOG.md index 75d215f39..2efe2c9b7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -42,8 +42,8 @@ All notable changes to this project will be documented in this file. See [standa - A `manifest/roles.yaml` that exists but does not parse now fails the pull for that scope instead of warning and syncing with no role filter at all, for a member with no role as much as for one with a role. The same applies to `init` and `push`, which each fell back to a guess at the namespaces when any error came out of the loader. The legacy role migration skips with a warning instead of failing, so every command, `pull` included, still loads the config and can fetch the fixed manifest; until it can run, the member holds no role rather than every role, so hooks, MCP servers and env variables scoped by `roles:` reach them no more than skills do. For a member with no active project that fallback meant an unfiltered sync, so a broken manifest delivered every namespace it was written to gate. Only an absent manifest still means "this team does not use roles"; an unreadable or empty file is an error, as it now is for `manifest/projects.yaml` too. `push` stops at its scan for such a manifest (exit 2) even with `--role `, since the scan needs it to tell which namespaces are the member's. - `teamai members list` and `teamai projects members` read the roster registered before the reports switch, so a team upgrading past the orphan-branch split no longer sees "No team members registered" while its `members/` still lives on the default branch. The default-branch copy becomes a read-only inherited root, the way learnings' already was: listed in union with the `teamai-reports` copy, with the branch copy winning when the same file exists on both; nothing is copied or deleted, and a cold `members list` still does not publish the reports branch. Member registration merges against the inherited copy too, so a re-init keeps the original `registeredAt` and projects. Fixes [#735](https://github.com/Tencent/teamai-cli/issues/735). - Cache GC now rejects partial integers such as `12abc`, decimals, zero and unsafe integers for `--max-bytes` and `--stale-days` before deleting anything. An invalid `TEAMAI_CACHE_MAX_BYTES` value falls back to the default 5 GB limit instead of using a numeric prefix. -- Each scope reports only the dashboard sessions recorded in it. Every scope read one machine-wide event log and picked its sessions out by the `cwd` they started in, so a user-scope pull reported every project's sessions to the user team, a project never reported its Copilot sessions (Copilot sends no `cwd`), and a session started under a symlinked path (or `/tmp` for `/private/tmp` on macOS) never matched its project. Each event now records the data directory of the scope `hook-dispatch` resolved, and a report keeps only its own scope's. Events recorded by an earlier release go to the project whose root holds their `cwd`, never to the user scope. The usage guide explains how to remove by hand a skill an earlier release reported into `stats/.yaml` from another project (for [#785](https://github.com/Tencent/teamai-cli/issues/785)). -- A session whose events belong to two scopes is reported to both teams with the part recorded in each. The snapshots of what was already reported were machine-wide, so after a `cd` into another project mid-session, the second scope to report compared its part with the first scope's totals and sent nothing. Each scope now keeps its own (`/dashboard/reported-*.json`, and `~/.teamai/dashboard/user-reported-*.json` for the user scope), seeded the first time from the shared `~/.teamai/dashboard/reported-*.json`, so the first report after the upgrade sends nothing already reported. The shared file is no longer written; after a rollback an earlier release writes it again, and only a scope not yet seeded reads it (for [#786](https://github.com/Tencent/teamai-cli/issues/786)). +- Each scope reports only the dashboard sessions recorded in it. Every scope read one machine-wide event log and picked its sessions out by the `cwd` they started in, so a user-scope pull reported every project's sessions to the user team, a project never reported its Copilot sessions (teamai does not record Copilot's `cwd`), and a session started under a symlinked path (or `/tmp` for `/private/tmp` on macOS) never matched its project. Each event now records a key (a hash, so no path is stored) of the data directory of the scope `hook-dispatch` resolved, and a report keeps only its own scope's. A session in a directory that resolves to no project (a subdirectory of a non-git project, a git submodule or nested clone) is the user scope's, as for skill usage. A session is reported once, whole, by the scope it started in, even after a `cd` into another project: its Stop carries the whole transcript's totals, so reporting its later part elsewhere would count them twice. A session ends with its session end or process exit (a process exit recorded right after its session end is the same session), so a later session that reuses its ID (Copilot's fallback ID is the parent process ID) is attributed and counted on its own, even when an earlier release reported that ID for another scope. Events recorded by an earlier release go to the scope their `cwd` resolves to now (a nested clone under a project is not the project's); events with no `cwd`, or one removed since, are reported by no one. The usage guide explains how to remove by hand a skill an earlier release reported into `stats/.yaml` from another project (for [#785](https://github.com/Tencent/teamai-cli/issues/785)). +- Each scope keeps its own snapshots of the dashboard sessions it already reported. They were machine-wide and keyed by session ID, so a session ID reused in another scope (Copilot's fallback ID is the parent process ID) was taken as already reported there and sent nothing. Each scope now keeps its own (`/dashboard/reported-*.json`, and `~/.teamai/dashboard/user-reported-*.json` for the user scope), seeded the first time from the shared `~/.teamai/dashboard/reported-*.json`, so the first report after the upgrade sends nothing already reported. The shared file is no longer written; after a rollback an earlier release writes it again, and only a scope not yet seeded reads it (for [#786](https://github.com/Tencent/teamai-cli/issues/786)). - Usage reporting scopes sessions by path on Windows too. The project/user scope filter compared an event's `cwd` against `projectRoot` with a hard-coded `/` separator, so on Windows only a session started in the project root itself matched: every session started in a subdirectory was dropped from the project team's report and counted in the user scope's instead, which is the isolation the usage guide promises. Windows paths are also compared case-insensitively, so a drive letter or a directory name spelled with different case in the two sources no longer leaks a project session into the user scope. POSIX paths keep their own rules: case-sensitive, and a `\` in a filename stays part of the name. - `teamai tags subscribe` and `teamai tags unsubscribe` now invalidate the pull revision cache, as `teamai skill exclude` already does, so the next `teamai pull` applies the new subscriptions instead of reporting "Already synced" when the team repo has not changed. - `teamai pull` now deletes a tombstoned agent under all three render extensions, so the Codex `.toml` and Kiro `.json` copies of a removed agent no longer survive on other machines. The cleanup also runs when the team repo rev is unchanged, so an upgrade reaches machines that already pulled the tombstone with an older CLI. `teamai remove agents ` also honours `enabledAgents` and no longer deletes from excluded tools. Fixes [#576](https://github.com/Tencent/teamai-cli/issues/576). diff --git a/docs/designs/data-directory-layout.md b/docs/designs/data-directory-layout.md index a6463906b..57e489fc2 100644 --- a/docs/designs/data-directory-layout.md +++ b/docs/designs/data-directory-layout.md @@ -348,25 +348,143 @@ is never read, and its pending deltas are not pushed. The dashboard stays an A2 singleton: `teamai dashboard`, `session save` and the contribute check read across scopes; `stats --by-repo` reads only the current scope's events, as the rest of `stats` does (#795). Each event instead -carries `dataHome`, the `getDataHome()` of the scope the hook resolved (#785), and a -scope's report keeps only its own. An event written before that field existed is -attributed by its `cwd`, realpath'd, to the project whose root holds it, never to -the user scope. Inside git an event also carries `projectAnchor`, the repo's -main checkout, which all of its worktrees share (#809). `stats --by-repo`, +carries `dataHomeKey`, a hash of the realpath'd `getDataHome()` of the scope the +hook resolved (#785; a hash, so a Copilot event still stores no path), and a +scope's report keeps the sessions whose first keyed event is its own, whole: a +Stop carries the whole transcript's totals, so a session that moved scope (a `cd` +mid-session) is reported once, where it started. A tool's own session ID is one +session whatever ends it records: `claude --resume` continues it, in a new +process, and its Stop carries the whole transcript. A fallback ID (`pid-…`; +Copilot's is the parent PID) names one run up to its `session_end` or +`process_exit`, so a later run that reuses it is decided on its own; a second end with nothing +recorded since the first (the dashboard monitor's `process_exit` after +`SessionEnd`) belongs to the run it closed. A `session_start` on a fallback ID +(`pid-…`) whose `monitorPid` differs from its open run's begins a new run even +though nothing ended that one (a crash with no dashboard running), and that one +counts as the run closed before it; a tool's own +ID is not split this way, since Claude fires SessionStart again on resume, in a +new process, and its Stop carries the whole transcript. The monitor's `process_exit` also +records `processExitAfter`, the last event it observed, and closes only that +run: an exit appended after the next run of the same ID began does not end it, +and one whose run compaction dropped is ignored. A dashboard started before +that field existed writes none. A dead process records nothing more, so such +an exit followed by more events of its fallback ID before the next start did +not end the open run: it belongs to the run closed before it, however late it +was appended. A tool's own session ID is +reported and snapshotted under the ID itself, as before, so a session resumed +after compaction dropped its events still reads what its scope reported. The +scope that first reports it also appends the ID and its own data home key to +`~/.teamai/dashboard/session-owners.jsonl` (never a path, #666), and a session +recorded there is that scope's wherever it is resumed later, whatever the log +still holds, so another scope never reports its transcript again; the first +line for an ID wins, and the file grows by one line per such session, like the +snapshots. An earlier release kept only per-scope snapshots, so the file is +first written from them: a tool's own ID is the scope's whose snapshots show it +reported it with the greatest total (prompts, then tokens). They show it when the +shared snapshots (all three) hold none of it, or the scope is past their total: +that release copied the shared file into every scope it ran in, so a copy, even +the only one, shows nothing, and a tie names no owner. When several scopes +reported it (a session that release split per event), the owner's line also +carries the credit of their parts, applied once as its baseline: a part whose +daily entry shows it ended in a Stop holds the transcript's cumulative total, so +the greatest such part counts once, while a part with no Stop counted its own +prompts, which add; interruptions, rejections and tokens, from Stops, take the +greatest, and corrections, counted per prompt, add. Whether a part with no +Stop came before another's cumulative Stop, which already counts it, is read +from the session's transcript when it has one (Claude): it keeps every prompt +in order with the directory it was typed in, so the Stop covers the first +prompts and only the part's prompts after those add. With no transcript to +place them they all add, which undercounts once but never sends a prompt again. The scopes read are the +user scope, every partition, and a project whose data home is in its workspace +that a session still in the log leads to; each report also records the IDs of +its own snapshots that have no owner yet and show it reported them (absent from +the shared snapshot, or past its total there). A session none of these reach +(a project whose data home is in its workspace, with no event left in the log) +is found by its transcript: hooks record `transcriptPath` on UserPromptSubmit, +Stop and SessionEnd (not SessionStart, whose path on a resume from another +project names a file that never exists; never Copilot's), and a Claude +transcript keeps its first `cwd` when resumed elsewhere, as a Codex rollout +keeps its `session_meta` and Copilot's own session log, found by the session ID +without storing its path, its `session.start` context. So a tool's own session +with no owner is the scope's that directory resolves to, when that scope's +snapshots already hold it; else it is decided as before (a fork under a new ID, +a tool whose transcript records no start). A session main split across scopes +per event, whose events are still in the log with each part's `dataHome`, is +credited once with every part reported: for each scope, the shortest prefix of +its events whose metrics reach its snapshot, and the owner's entry is raised, +counter by counter, to at least the metrics of their union (a part may have +reported more time, tokens or costs with no more prompts), so parts counted before +any Stop carried the transcript's total are neither lost nor sent twice. +A Codex session (any Codex variant: `codex`, `codex-internal`, `tcodex`) is kept +per rollout, with or without a token record, and when +its tokens come from a thread-level counter that already spans rollouts (then +no rollout holds tokens of its own, nor does the prior rollout an entry from +before leaves); a rollout's prompts are its Stop's count or +else its submits. A +rollout's counters restart, and compaction drops its events, so its prompt-token +entry holds each rollout's reported prompts, tokens, interruptions, rejections, +corrections, active time and request costs under a hash of the rollout's path, +written with any delta, and whether it failed (an error, an interruption or a +correction). The session's daily request costs sum its rollouts in the log. A +rollout compaction has dropped keeps those totals in the session's prompt-token, +intervention and daily sums (cache tokens from its tokens), and a failed one +keeps the session unsuccessful, so a later rollout is reported in full and does +not turn it into a success. An entry +written before rollouts were kept is one total: an earlier release rewrote every +session in the log on each report, so it covers the rollouts begun by the time +its file was last written, or, earlier, when that report wrote the team stats +file in this scope's reports checkout (after reading the log, before its push; +the snapshot came after the push). That is read before this report writes +anything; a seed keeps the +shared file's time, and `teamai stats`, which only reads, writes no seed). Those still in +the log consume it in order, as far as each had got by that time, what is left is the dropped rollouts', kept as one +prior rollout, and a rollout begun later is new. +Compaction also keeps a session whose tool process is still running, so a run +an exit from a dashboard before `processExitAfter` marked stopped keeps its +start, and its ID. A +fallback run is reported and snapshotted as `@`, +which does not change when compaction drops earlier runs. A snapshot entry keyed +by a bare fallback ID (written before) is the sum of the runs of that ID in the +log at the earlier release's last report, and compaction keeps or drops the +runs of an ID together. So the next time the scope reports, those runs consume +the entry in log order, each taking up to its own totals of what is left; once +the prompt-token entry is used up, the later runs were not reported and count +as new sessions (the interventions and daily snapshots follow the prompt-token +one, since their counts say nothing when they run out). A run keeps its own +success and correction flags, since the sum's are no single run's, so an +adopted run changes no status total. The first run always +takes a share, as the entry means that release reported it. The entry is then +removed, so no later run of that ID reads it. Only an earlier release wrote bare +entries, and a seeded one may be another scope's, so a run whose first event +carries `dataHomeKey` (recorded by this release), and every later run of its ID, +takes none. A session written before that field existed is attributed by its first +`cwd`: to the scope `resolveConfigForDir` resolves that directory to now, the +dispatcher's rule, so a nested clone under a project is not the project's; no +`cwd`, or one removed since, is no scope's. Inside git an event also carries `projectAnchor`, the repo's +main checkout, which all of its worktrees share (#809); a Copilot event, with no +`cwd`, carries none. `stats --by-repo`, `session save` and the dashboard's Repository filter key a session by the last anchor it recorded, else by its `cwd`, and the dashboard gives an event to the project rooted at its anchor, so a worktree counts as its repo, also after it -is removed. The snapshots of what was already reported are per scope too -(#786), because a session can record events in two scopes (a `cd` mid-session): +is removed. The snapshots of what was already reported are per scope +too (#786), because a session ID can recur in another scope (Copilot's fallback +ID is the parent PID): `/dashboard/reported-*.json`, and `~/.teamai/dashboard/user-reported-*.json` -for the user scope. The first time a scope needs one it copies the shared +for the user scope. The first time a scope needs one it seeds it from the shared `~/.teamai/dashboard/reported-*.json`, so nothing reported before the upgrade is -sent again; after that it reads only its own. The shared file is no longer +sent again; after that it reads only its own. That file summed every scope's +runs of an ID, so the runs of the whole log consume it in log order, whichever +scope each belongs to, and the seed keeps the shares of the scope's own runs, +under their run IDs; none goes to a run recorded +with a `dataHome` path: that release already kept per-scope snapshots, so a +shared entry under its ID is another scope's. An unmatched fallback entry is +dropped, so a later reuse of the PID cannot inherit it; a tool's own session ID +is copied whole, as before, so a session resumed after compaction dropped its +events is not sent again. The shared file is no longer written, except by an earlier release after a rollback, so every scope seeds from what the machine had reported by then, never from another scope's later report. -The seed holds a session's whole total, so a session still running at the upgrade -that later records in a second scope reports nothing there until that scope's -part exceeds it. +The seed holds a session's whole total, so a session still running at the +upgrade goes on from the reported total, as before. **`anchor` on save.** Previously only migration wrote a partition's `anchor` reverse-lookup file, so freshly-init'd partitions had none. `saveLocalConfigForScope` diff --git a/docs/usage-guide.md b/docs/usage-guide.md index 9e7fb1f37..9cf535959 100644 --- a/docs/usage-guide.md +++ b/docs/usage-guide.md @@ -390,23 +390,6 @@ branch, exactly where they are. Nothing is copied, deleted or migrated: that directory is still read, so every existing learning keeps coming back from `teamai recall`. New learnings go to `teamai-learnings`. -**Removing a skill another project reported into your `stats/`.** Before skill -usage was kept per scope, whichever project pulled next reported every -project's skills, so `stats/.yaml` on `teamai-reports` can count a skill -that belongs to an unrelated repo. Those events recorded no directory, so -teamai cannot attribute them and never rewrites the file. Remove the entry by -hand, from a clone of your own so teamai's `reports-wt/` checkout is untouched: - -```bash -git clone --branch teamai-reports --single-branch teamai-reports -cd teamai-reports -# delete the skill's entry under `skills:` in stats/.yaml -git commit -am "stats: remove reported from another project" -git push origin teamai-reports -``` - -The next report reads the branch first, so the entry does not come back. - **Minimum Git permissions with a protected default branch.** A member needs to: @@ -1772,15 +1755,23 @@ reported snapshots. Skill usage is recorded per scope, in the data directory of the project teamai is set up for where the session ran (or the user scope), so each target reports only its own; a directory without teamai records none. Dashboard sessions stay in one machine-wide `~/.teamai/dashboard/events.jsonl`, -but each event records the data home of its scope, so a scope reports only the -sessions recorded in it: a user-scope pull no longer reports a project's -sessions, and a project reports its Copilot sessions and sessions started under -a symlinked path. Events recorded by an earlier release carry no data home: a -project reports those whose directory lies under its root, the user scope -reports none of them. Each scope also keeps its own snapshot of what it already -reported, so a session that moved into another project mid-session reaches both -teams with the part recorded in each; the first report after upgrading starts -from the snapshot every scope used to share, so nothing is reported twice. A +but each event records a key of its scope's data home (a hash, not the path), +so a scope reports only the sessions recorded in it: a user-scope pull no +longer reports a project's sessions, and a project reports its Copilot sessions +and sessions started under a symlinked path. A session is reported once, whole, +by the scope it started in, even if it later moves into another project: its +Stop carries the whole transcript's totals, so a second scope would count them +again. Events recorded by an earlier release carry no key: the scope their +directory resolves to now reports them (a nested clone under a project resolves +to its own project or the user scope, not the enclosing project); events with +no directory, or one removed since, are reported by no one. Each scope also +keeps its own snapshot of what it already reported, and a session under a +reused fallback ID (Copilot's PID-based ID when it sends none) counts as new, +whichever scope reported the earlier one, while a resumed session +(`claude --resume`) keeps its ID and stays one session, reported by the scope +that first reported it wherever it is resumed; the first +report after upgrading starts from the snapshot every scope used to share, so +nothing is reported twice. A target removes its usage events only after it confirms success; failed pushes preserve them, up to the newest 5,000 (see below). The affected sync locks remain held until reporting finishes, preventing another pull from racing the report. @@ -1815,6 +1806,23 @@ files; `pull` and `push` add those entries to an existing single-repo one, and the usage file's first pending file or rewrite adds them to an existing project-scope one. +**Removing a skill another project reported into your `stats/`.** Before skill +usage was kept per scope, whichever project pulled next reported every +project's skills, so `stats/.yaml` on `teamai-reports` can count a skill +that belongs to an unrelated repo. Those events recorded no directory, so +teamai cannot attribute them and never rewrites the file. Remove the entry by +hand, from a clone of your own so teamai's `reports-wt/` checkout is untouched: + +```bash +git clone --branch teamai-reports --single-branch teamai-reports +cd teamai-reports +# delete the skill's entry under `skills:` in stats/.yaml +git commit -am "stats: remove reported from another project" +git push origin teamai-reports +``` + +The next report reads the branch first, so the entry does not come back. + ### Git submodules If your team distributes skills as git submodules, opt in with `submodules: true` diff --git a/docs/usage-guide.zh-CN.md b/docs/usage-guide.zh-CN.md index 37731a845..84201047d 100644 --- a/docs/usage-guide.zh-CN.md +++ b/docs/usage-guide.zh-CN.md @@ -360,21 +360,6 @@ learnings 迁到独立分支之前团队已经写下的内容，原地留在默 不迁移：该目录仍会被读取，所有既有 learning 依然能从 `teamai recall` 中找回。新的 learning 写入 `teamai-learnings`。 -**删除其他项目上报进你 `stats/` 的 skill。** 在 skill 使用按 scope 记录之前，下一个 -执行 pull 的项目会上报所有项目的 skill，因此 `teamai-reports` 上的 `stats/.yaml` 可能 -统计了属于无关仓库的 skill。这些事件没有记录目录，teamai 无法归属，也不会改写该 -文件。请手动删除该条目，在单独的 clone 中操作，不要动 teamai 的 `reports-wt/` 检出： - -```bash -git clone --branch teamai-reports --single-branch teamai-reports -cd teamai-reports -# 删除 stats/.yaml 中 `skills:` 下该 skill 的条目 -git commit -am "stats: remove reported from another project" -git push origin teamai-reports -``` - -下一次上报会先读取该分支，所以条目不会再出现。 - **默认分支受保护时所需的最小 Git 权限。** 成员需要： @@ -1706,12 +1691,14 @@ Pull 对整批统计上报最多等待 5 秒，之后继续其他工作，上报 超时后推送成功，仍会更新本地已上报快照。skill 使用按 scope 记录：写入会话所在 目录对应的已配置 teamai 项目的数据目录（或 user scope），因此每个目标只上报 自己的使用；未配置 teamai 的目录不记录。Dashboard 会话仍写入整机共用的 -`~/.teamai/dashboard/events.jsonl`，但每条事件都记下所属 scope 的数据目录（data home），因此每个 scope +`~/.teamai/dashboard/events.jsonl`，但每条事件都记下所属 scope 数据目录（data home）的键（哈希值，不是路径），因此每个 scope 只上报在其中记录的会话：user scope 的 pull 不再上报项目的会话，项目会上报自己的 -Copilot 会话以及从软链接路径启动的会话。旧版本记录的事件没有数据目录：项目上报目录 -位于其根目录下的那些，user scope 一条也不上报。每个 scope 还各自保存已上报快照， -因此会话中途切换到另一个项目时，两个团队各自收到在其中记录的那部分；升级后的首次上报 -从原先各 scope 共用的快照开始，不会重复上报。目标确认成功后才清理自己的使用事件， +Copilot 会话以及从软链接路径启动的会话。每个会话只上报一次，整体归属其开始时所在的 +scope，即使之后切换到另一个项目：它的 Stop 带有整份 transcript 的累计值，第二个 scope +会重复计算。旧版本记录的事件没有该键：由其目录当前解析到的 scope 上报（项目下的嵌套 +clone 解析到它自己的项目或 user scope，而不是外层项目）；没有目录或目录已删除的事件不由任何 scope 上报。 +每个 scope 还各自保存已上报快照，且复用回退 ID 的新会话（Copilot 未提供会话 ID 时基于 PID 的 ID） +总算作新会话，无论先前那个由哪个 scope 上报；恢复的会话（`claude --resume`）保留原 ID，仍是同一个会话，无论在哪里恢复，都由最先上报它的 scope 上报；升级后的首次上报从原先各 scope 共用的快照开始，不会重复上报。目标确认成功后才清理自己的使用事件， 推送失败会保留事件（最多保留最新 5,000 条，见下文）。上报完成前继续持有相关同步锁， 避免另一次 Pull 与尚未完成的上报竞争。 @@ -1736,6 +1723,21 @@ pending 文件的权限不宽于使用文件（尚无使用文件时仅所有者 忽略该锁、改写的临时副本与 pending 文件；`pull` 与 `push` 会为已有的单仓库 `.gitignore` 补上这些条目， 已有的项目级 `.gitignore` 则在使用文件第一次写 pending 文件或改写时补上。 +**删除其他项目上报进你 `stats/` 的 skill。** 在 skill 使用按 scope 记录之前，下一个 +执行 pull 的项目会上报所有项目的 skill，因此 `teamai-reports` 上的 `stats/.yaml` 可能 +统计了属于无关仓库的 skill。这些事件没有记录目录，teamai 无法归属，也不会改写该 +文件。请手动删除该条目，在单独的 clone 中操作，不要动 teamai 的 `reports-wt/` 检出： + +```bash +git clone --branch teamai-reports --single-branch teamai-reports +cd teamai-reports +# 删除 stats/.yaml 中 `skills:` 下该 skill 的条目 +git commit -am "stats: remove reported from another project" +git push origin teamai-reports +``` + +下一次上报会先读取该分支，所以条目不会再出现。 + ### Git 子模块 若团队以 git submodule 形式分发 skill，在 `teamai.yaml` 中开启 `submodules: true`： diff --git a/src/__tests__/conversation-token-metrics.test.ts b/src/__tests__/conversation-token-metrics.test.ts index 0f34402c8..777858e60 100644 --- a/src/__tests__/conversation-token-metrics.test.ts +++ b/src/__tests__/conversation-token-metrics.test.ts @@ -7,7 +7,7 @@ import { aggregateSessionMetrics, rebuildSessions, } from '../dashboard-collector.js'; -import { computePromptTokenDelta, mergePromptTokenStats } from '../team-push.js'; +import { computePromptTokenDelta, droppedRollouts, mergePromptTokenStats } from '../team-push.js'; import { summarizeConversation, formatTokenCount } from '../digest.js'; import type { DashboardEvent, SessionMetrics, TokenUsage, UserStats } from '../types.js'; @@ -25,6 +25,8 @@ afterEach(() => { fs.rmSync(tmpDir, { recursive: true, force: true }); }); +const emptyTokens = (): TokenUsage => ({ input: 0, output: 0, cacheRead: 0, cacheCreation: 0 }); + // An assistant transcript line carrying token usage for one message id. function assistantLine(id: string, usage: Partial>, blockType = 'text'): string { return JSON.stringify({ @@ -368,6 +370,68 @@ describe('aggregateSessionMetrics', () => { .toEqual({ input: 0, output: 0, cacheRead: 0, cacheCreation: 0 }); }); + it('keeps a reported rollout\'s totals once compaction drops it, so a resumed rollout is reported', () => { + // A Codex build that writes a new rollout per resume restarts its counters. + const tokensA = { input: 500, output: 50, cacheRead: 1_000, cacheCreation: 0 }; + const tokensB = { input: 30, output: 3, cacheRead: 100, cacheCreation: 0 }; + const stop = (path: string, tokens: TokenUsage, prompts: number): DashboardEvent => ({ + type: 'stop', timestamp: new Date().toISOString(), sessionId: 's1', tool: 'codex', + transcriptPath: path, tokenScope: 'transcript', tokens, prompts, + }); + const first = computePromptTokenDelta(aggregateSessionMetrics([stop('/rollouts/a.jsonl', tokensA, 5)]), {}); + expect(first.delta).toEqual({ prompts: 5, tokens: tokensA }); + expect(JSON.stringify(first.nextReported)).not.toContain('/rollouts/'); + + // Compaction dropped rollout A; the resume wrote rollout B. + const resumed = computePromptTokenDelta(aggregateSessionMetrics([stop('/rollouts/b.jsonl', tokensB, 2)]), first.nextReported); + expect(resumed.delta).toEqual({ prompts: 2, tokens: tokensB }); + const again = computePromptTokenDelta(aggregateSessionMetrics([stop('/rollouts/b.jsonl', tokensB, 2)]), resumed.nextReported); + expect(again.delta).toEqual({ prompts: 0, tokens: emptyTokens() }); + }); + + it('an entry from before rollouts were kept covers only the rollouts that existed when it was written', () => { + const tokensA = { input: 500, output: 50, cacheRead: 1_000, cacheCreation: 0 }; + const tokensB = { input: 30, output: 3, cacheRead: 100, cacheCreation: 0 }; + const stop = (path: string, timestamp: string, tokens: TokenUsage, prompts: number): DashboardEvent => ({ + type: 'stop', timestamp, sessionId: 's1', tool: 'codex', transcriptPath: path, tokenScope: 'transcript', tokens, prompts, + }); + const writtenAt = Date.parse('2026-09-01T12:00:00Z'); + // An earlier release reported rollout A as a whole, with no rollouts. + const legacy = { s1: { prompts: 5, tokens: tokensA } }; + const a = stop('/rollouts/a.jsonl', '2026-09-01T11:00:00Z', tokensA, 5); + const b = stop('/rollouts/b.jsonl', '2026-09-01T13:00:00Z', tokensB, 2); + // A was compacted before this build's first report; B came after the entry was written. + const compacted = aggregateSessionMetrics([b]); + expect(computePromptTokenDelta(compacted, legacy, droppedRollouts(compacted, legacy, {}, {}, writtenAt)).delta) + .toEqual({ prompts: 2, tokens: tokensB }); + // A still in the log takes the entry; B is new. + const retained = aggregateSessionMetrics([a, b]); + expect(computePromptTokenDelta(retained, legacy, droppedRollouts(retained, legacy, {}, {}, writtenAt)).delta) + .toEqual({ prompts: 2, tokens: tokensB }); + }); + + it('keeps a Codex rollout\'s latest Stop, whatever order the Stops come in', () => { + // A background handler appended the older scan after the newer one. + const cost = (costMicros: number) => ({ '2026-09-01': { + pricedRequests: 1, costMicros, cacheReadTokens: 0, cacheEligibleInputTokens: 0, priceVersion: 'v1', + } }); + const stop = (timestamp: string, prompts: number, costMicros: number, toolReject: number): DashboardEvent => ({ + type: 'stop', timestamp, sessionId: 's1', tool: 'codex', transcriptPath: '/rollouts/a.jsonl', + prompts, requestDaily: cost(costMicros), interventions: { interrupt: 0, toolReject }, + }); + const rollout = aggregateSessionMetrics([stop('2026-09-01T10:10:00Z', 3, 100, 2), stop('2026-09-01T10:05:00Z', 2, 40, 1)]) + .get('s1')?.segments?.['/rollouts/a.jsonl']; + expect(rollout).toMatchObject({ prompts: 3, toolReject: 2, requestDaily: cost(100) }); + }); + + it('counts a Codex session\'s prompts across its rollouts', () => { + const stop = (path: string, prompts: number): DashboardEvent => ({ + type: 'stop', timestamp: new Date().toISOString(), sessionId: 's1', tool: 'codex', + transcriptPath: path, tokenScope: 'transcript', tokens: emptyTokens(), prompts, + }); + expect(aggregateSessionMetrics([stop('/rollouts/a.jsonl', 5), stop('/rollouts/b.jsonl', 2)]).get('s1')?.prompts).toBe(7); + }); + it('does not add transcript-scoped Codex totals to a session-scoped snapshot', () => { const events: DashboardEvent[] = [ { diff --git a/src/__tests__/dashboard-collector.test.ts b/src/__tests__/dashboard-collector.test.ts index bb6e09aef..09a1107ab 100644 --- a/src/__tests__/dashboard-collector.test.ts +++ b/src/__tests__/dashboard-collector.test.ts @@ -2122,6 +2122,28 @@ describe('compactEvents', () => { const content = fs.readFileSync(eventsPath, 'utf-8'); expect(content.trim().split('\n')).toHaveLength(1); }); + + it('keeps a session whose tool process is still running, though an exit marked it stopped', async () => { + // A dashboard from before processExitAfter may close a live run with an + // exit it observed for the run before; dropping that run's start would + // give its next activity a new run ID. + const eventsPath = path.join(tmpDir, '.teamai', 'dashboard', 'events.jsonl'); + fs.mkdirSync(path.dirname(eventsPath), { recursive: true }); + const old = new Date(Date.now() - 3_600_000).toISOString(); + const at = (sessionId: string, type: string, extra: Record = {}) => + JSON.stringify({ type, timestamp: old, sessionId, tool: 'copilot', ...extra }); + const lines = [ + at('pid-live', 'session_start', { monitorPid: process.pid }), at('pid-live', 'process_exit'), + at('pid-dead', 'session_start', { monitorPid: 2 ** 22 + 12345 }), at('pid-dead', 'process_exit'), + ...Array.from({ length: 5_000 }, (_, i) => at(`filler-${i}`, 'session_end')), + ]; + fs.writeFileSync(eventsPath, lines.join('\n') + '\n'); + + await compactEvents(eventsPath); + + const kept = fs.readFileSync(eventsPath, 'utf-8').split('\n').filter(Boolean).map((line) => JSON.parse(line).sessionId); + expect(kept).toEqual(['pid-live', 'pid-live']); + }); }); // ─── countInterventions (CodeBuddy index.json) ───────── diff --git a/src/__tests__/dashboard-report-gate.test.ts b/src/__tests__/dashboard-report-gate.test.ts index 6c070bc4c..ecb7d3e39 100644 --- a/src/__tests__/dashboard-report-gate.test.ts +++ b/src/__tests__/dashboard-report-gate.test.ts @@ -3,7 +3,7 @@ import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; import { Readable } from 'node:stream'; -import { dashboardReport } from '../dashboard-collector.js'; +import { dashboardReport, dataHomeKey } from '../dashboard-collector.js'; import { _setLogFilePath, _resetState } from '../utils/logger.js'; // ─── legacy `dashboard-report` config gate ─── @@ -110,6 +110,8 @@ describe('legacy dashboard-report config gate', () => { expect(events).toHaveLength(1); expect(events[0].type).toBe('session_start'); expect(events[0].sessionId).toBe('sess-legacy-1'); + // Keyed to the scope it resolved, so that scope's report keeps it (#785). + expect(events[0].dataHomeKey).toBe(await dataHomeKey(userConfigDir)); }); it('records nothing when no cwd can be resolved from the payload', async () => { diff --git a/src/__tests__/dashboard-report-scope.test.ts b/src/__tests__/dashboard-report-scope.test.ts index 97a1286a4..b4ae939d6 100644 --- a/src/__tests__/dashboard-report-scope.test.ts +++ b/src/__tests__/dashboard-report-scope.test.ts @@ -32,6 +32,9 @@ vi.mock('../utils/reports-branch.js', () => ({ const { hookDispatchCli } = await import('../hook-dispatch-cli.js'); const { resolveProjectDataHome, saveLocalConfigForScope, resolveConfigForDir, loadLocalConfig } = await import('../config.js'); const { reportUsageToTeam } = await import('../team-push.js'); +const { getDataHome } = await import('../types.js'); +const { dataHomeKey } = await import('../dashboard-collector.js'); +const dataHomeKeyOf = (config: LocalConfig) => dataHomeKey(getDataHome(config)); let tmp: string; let originalHome: string | undefined; @@ -125,14 +128,29 @@ describe('each scope reports only the dashboard sessions recorded in it (#785)', expect(await reportedSessions(user)).toBe(1); }); - it('a Copilot session, which records no cwd, is reported by its project', async () => { + it.each([ + ['names its cwd', true], + ['sends no cwd, from a hook running in the project', false], + ])('a Copilot session, whose events record no cwd, is reported by its project: payload %s', async (_, withCwd) => { const { root, user, project } = await setup(); - await session('copilot', { session_id: 'copilot-p', cwd: root }); + if (!withCwd) process.chdir(root); + await session('copilot', withCwd ? { session_id: 'copilot-p', cwd: root } : { session_id: 'copilot-p' }); expect(await reportedSessions(user)).toBe(0); expect(await reportedSessions(project)).toBe(1); }); + it('a Copilot event records its scope without persisting a path (#666)', async () => { + const { root, project } = await setup(); + await session('copilot', { session_id: 'copilot-p', cwd: root }); + + const log = fs.readFileSync(path.join(teamaiHome(), 'dashboard', 'events.jsonl'), 'utf-8'); + const partition = getDataHome(project); + expect(log).not.toContain(root); + expect(log).not.toContain(partition); + expect(log).not.toContain(path.basename(partition)); + }); + it('records the repo of a session, and still no path for a Copilot one (#809)', async () => { const { root } = await setup(); await session('claude', { session_id: 'sid-a', cwd: root }); @@ -158,10 +176,17 @@ describe('each scope reports only the dashboard sessions recorded in it (#785)', expect(await reportedSessions(project)).toBe(1); }); - it('events recorded before sessions carried a data home go to their project, never to the user scope', async () => { + it('events recorded before sessions carried a data home go to the scope their cwd resolves to now', async () => { const { root, user, project } = await setup(); const link = path.join(tmp, 'link-p'); fs.symlinkSync(root, link, 'dir'); + fs.mkdirSync(path.join(root, 'src')); + fs.mkdirSync(path.join(tmp, 'elsewhere')); + // A nested clone under P resolves to no project, so it is the user scope's, not P's too. + fs.mkdirSync(path.join(root, 'nested')); + execFileSync('git', ['init', '-q'], { cwd: path.join(root, 'nested') }); + // A sibling whose name only starts with P's. + fs.mkdirSync(`${root}-ab`); const timestamp = new Date().toISOString(); const old = (sessionId: string, cwd: string | undefined) => [ { type: 'session_start', timestamp, sessionId, tool: 'claude', cwd }, @@ -173,10 +198,14 @@ describe('each scope reports only the dashboard sessions recorded in it (#785)', ...old('old-in-p', path.join(root, 'src')), ...old('old-via-link', link), ...old('old-elsewhere', path.join(tmp, 'elsewhere')), + ...old('old-nested', path.join(root, 'nested')), + ...old('old-sibling', `${root}-ab`), + // Nothing can tell whose these were, so no scope reports them. + ...old('old-gone', path.join(root, 'removed-worktree')), ...old('old-no-cwd', undefined), ].map((e) => JSON.stringify(e)).join('\n') + '\n'); - expect(await reportedSessions(user)).toBe(0); + expect(await reportedSessions(user)).toBe(3); expect(await reportedSessions(project)).toBe(2); }); }); @@ -188,11 +217,19 @@ describe('each scope keeps its own reported snapshot (#786)', () => { return stats && typeof stats === 'object' && 'prompts' in stats && typeof stats.prompts === 'number' ? stats.prompts : 0; } + /** The sessions a scope's intervention stats hold, 0 before its first push. */ + async function reportedInterventionSessions(config: LocalConfig): Promise { + const stats = await report(config); + const interventions = stats && typeof stats === 'object' && 'interventions' in stats ? stats.interventions : undefined; + return interventions && typeof interventions === 'object' && 'sessions' in interventions + && typeof interventions.sessions === 'number' ? interventions.sessions : 0; + } + const shared = (name: string) => path.join(teamaiHome(), 'dashboard', `reported-${name}.json`); const SNAPSHOTS = ['interventions', 'prompt-tokens', 'daily-sessions']; /** The shared snapshots as a release before #786 left them, with `prompts` reported per session. */ - function writeSharedSnapshots(prompts: Record, date: string): void { + function writeSharedSnapshots(prompts: Record, date: string, config?: LocalConfig): void { const entries = (value: (n: number) => unknown) => Object.fromEntries(Object.entries(prompts).map(([sid, n]) => [sid, value(n)])); const values: Record = { @@ -201,8 +238,11 @@ describe('each scope keeps its own reported snapshot (#786)', () => { // Longer than any fixture session, so no duration is left to report. 'daily-sessions': entries((n) => ({ date, prompts: n, durationMs: 3_600_000, succeeded: 1, corrected: 0 })), }; - fs.mkdirSync(path.join(teamaiHome(), 'dashboard'), { recursive: true }); - for (const name of SNAPSHOTS) fs.writeFileSync(shared(name), JSON.stringify(values[name])); + const dir = path.join(config ? getDataHome(config) : teamaiHome(), 'dashboard'); + // The user scope's own snapshot sits beside the shared one, prefixed. + const prefix = config?.scope === 'user' ? 'user-' : ''; + fs.mkdirSync(dir, { recursive: true }); + for (const name of SNAPSHOTS) fs.writeFileSync(path.join(dir, `${prefix}reported-${name}.json`), JSON.stringify(values[name])); } async function prompts(sessionId: string, cwd: string, count: number): Promise { @@ -211,21 +251,39 @@ describe('each scope keeps its own reported snapshot (#786)', () => { } } - it('a session split across the user scope and a project reaches both teams with its own counts', async () => { + /** + * Run `record` as an earlier release would have: before #785 its events carry + * no data home, and main since #795 wrote the data home path, `dataHome`. + */ + async function asEarlierRelease(record: () => Promise, dataHome?: string): Promise { + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + const before = fs.existsSync(log) ? fs.readFileSync(log, 'utf-8') : ''; + await record(); + const added = fs.readFileSync(log, 'utf-8').slice(before.length).split('\n').filter(Boolean).map((line) => { + const event = Object.fromEntries(Object.entries(JSON.parse(line)).filter(([field]) => field !== 'dataHomeKey')); + return `${JSON.stringify(dataHome === undefined ? event : { ...event, dataHome })}\n`; + }); + fs.writeFileSync(log, before + added.join('')); + } + + // A Stop carries the whole transcript's totals, so a session is reported once, + // whole, by the scope it started in; split per event, the later scope would + // count the earlier scope's part again. + it('a session that moves from the user scope into a project is reported once, by the user scope', async () => { const { root, user, project } = await setup(); const elsewhere = path.join(tmp, 'elsewhere'); fs.mkdirSync(elsewhere); - await hook('session-start', 'claude', { session_id: 'split', cwd: elsewhere, hook_event_name: 'SessionStart' }); - await prompts('split', elsewhere, 3); - await prompts('split', root, 2); // `cd` into the project mid-session - await hook('stop', 'claude', { session_id: 'split', cwd: root, hook_event_name: 'Stop' }); + await hook('session-start', 'claude', { session_id: 'moved', cwd: elsewhere, hook_event_name: 'SessionStart' }); + await prompts('moved', elsewhere, 3); + await prompts('moved', root, 2); // `cd` into the project mid-session + await hook('stop', 'claude', { session_id: 'moved', cwd: root, hook_event_name: 'Stop' }); - expect(await reportedPrompts(user)).toBe(3); - expect(await reportedPrompts(project)).toBe(2); + expect(await reportedPrompts(project)).toBe(0); + expect(await reportedPrompts(user)).toBe(5); }); - it('a session split across two projects reaches both teams with its own counts', async () => { - const { root, project } = await setup(); + /** A second git project, Q, with its own team. */ + async function setupQ(): Promise<{ rootQ: string; projectQ: LocalConfig }> { const rootQ = path.join(tmp, 'project-q'); fs.mkdirSync(rootQ); execFileSync('git', ['init', '-q'], { cwd: rootQ }); @@ -237,21 +295,948 @@ describe('each scope keeps its own reported snapshot (#786)', () => { }); const projectQ = await resolveConfigForDir(rootQ); if (!projectQ) throw new Error('fixture config Q did not resolve'); - await hook('session-start', 'claude', { session_id: 'split', cwd: root, hook_event_name: 'SessionStart' }); - await prompts('split', root, 3); - await prompts('split', rootQ, 2); - await hook('stop', 'claude', { session_id: 'split', cwd: rootQ, hook_event_name: 'Stop' }); + return { rootQ, projectQ }; + } + + it('a session that moves from one project into another is reported once, by the first', async () => { + const { root, project } = await setup(); + const { rootQ, projectQ } = await setupQ(); + await hook('session-start', 'claude', { session_id: 'moved', cwd: root, hook_event_name: 'SessionStart' }); + await prompts('moved', root, 3); + await prompts('moved', rootQ, 2); + await hook('stop', 'claude', { session_id: 'moved', cwd: rootQ, hook_event_name: 'Stop' }); + + expect(await reportedPrompts(projectQ)).toBe(0); + expect(await reportedPrompts(project)).toBe(5); + }); + + it('a session ID another scope already reported is a new session in this scope (Copilot PID fallback)', async () => { + const { root, user, project } = await setup(); + const elsewhere = path.join(tmp, 'elsewhere'); + fs.mkdirSync(elsewhere); + // No session ID: Copilot falls back to `pid-`, reused by the next session. + await session('copilot', { cwd: elsewhere }); + expect(await reportedSessions(user)).toBe(1); + // Compaction dropped the ended session; a later one in P gets the same ID. + fs.writeFileSync(path.join(teamaiHome(), 'dashboard', 'events.jsonl'), ''); + await session('copilot', { cwd: root }); + + expect(await reportedSessions(project)).toBe(1); + }); + + it('a session ID reused after its session ended goes to the scope that reuses it, while the log still holds the ended one', async () => { + const { root, user, project } = await setup(); + const elsewhere = path.join(tmp, 'elsewhere'); + fs.mkdirSync(elsewhere); + await session('copilot', { cwd: elsewhere }); + await hook('session-end', 'copilot', { cwd: elsewhere, hook_event_name: 'SessionEnd' }); + expect(await reportedSessions(user)).toBe(1); + // Below the compaction threshold the ended `pid-` session stays; the next one in P reuses its ID. + await session('copilot', { cwd: root }); + + expect(await reportedSessions(project)).toBe(1); + expect(await reportedSessions(user)).toBe(1); + }); + + it('two sessions that reuse one session ID in the same scope are two sessions, while the log holds both', async () => { + const { root, project } = await setup(); + await session('copilot', { cwd: root }); + await hook('session-end', 'copilot', { cwd: root, hook_event_name: 'SessionEnd' }); + expect(await reportedSessions(project)).toBe(1); + await session('copilot', { cwd: root }); + await hook('session-end', 'copilot', { cwd: root, hook_event_name: 'SessionEnd' }); + + expect(await reportedSessions(project)).toBe(2); + expect(await reportedPrompts(project)).toBe(2); + }); + + it('a session ID reused in the same scope after compaction dropped the run it reported is a new session', async () => { + const { root, project } = await setup(); + await session('copilot', { cwd: root }); + await hook('session-end', 'copilot', { cwd: root, hook_event_name: 'SessionEnd' }); + expect(await reportedSessions(project)).toBe(1); + // Compaction dropped the ended run before P reported again; the next run in P reuses its ID. + fs.writeFileSync(path.join(teamaiHome(), 'dashboard', 'events.jsonl'), ''); + await session('copilot', { cwd: root }); + + expect(await reportedSessions(project)).toBe(2); + expect(await reportedPrompts(project)).toBe(2); + }); + + it('a session that ends twice, SessionEnd then the dashboard monitor\'s process_exit, is one session', async () => { + const { root, project } = await setup(); + await session('copilot', { cwd: root }); + await hook('session-end', 'copilot', { cwd: root, hook_event_name: 'SessionEnd' }); + // The monitor read the session as still running while SessionEnd was appended. + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + const last = Object.fromEntries(Object.entries(JSON.parse(fs.readFileSync(log, 'utf-8').trim().split('\n').at(-1) ?? '{}'))); + fs.appendFileSync(log, `${JSON.stringify({ ...last, type: 'process_exit', timestamp: new Date(Date.now() + 1000).toISOString() })}\n`); + + expect(await reportedInterventionSessions(project)).toBe(1); + expect(await reportedSessions(project)).toBe(1); + }); + + it.each([ + ['after the project first reported', true], + ['before the project first reported', false], + ])('a session ID an earlier release reported for the user scope is a new session when this build records it in a project, %s', async (_, reportFirst) => { + const { root, project } = await setup(); + const elsewhere = path.join(tmp, 'elsewhere'); + fs.mkdirSync(elsewhere); + const pid = `pid-${process.ppid}`; + // The ended user-scope run stays in the log, below the compaction threshold. + await asEarlierRelease(async () => { + await session('copilot', { cwd: elsewhere }); + await hook('session-end', 'copilot', { cwd: elsewhere, hook_event_name: 'SessionEnd' }); + }); + writeSharedSnapshots({ [pid]: 1 }, new Date().toISOString().slice(0, 10)); + if (reportFirst) expect(await report(project)).toBeNull(); + await session('copilot', { cwd: root }); + + expect(await reportedSessions(project)).toBe(1); + expect(await reportedPrompts(project)).toBe(1); + }); + + it('a run in progress across the upgrade is not reported again for what the earlier release reported', async () => { + const { root, project } = await setup(); + const pid = `pid-${process.ppid}`; + await asEarlierRelease(async () => { + await hook('session-start', 'copilot', { cwd: root, hook_event_name: 'SessionStart' }); + await prompts(pid, root, 1); + }); + writeSharedSnapshots({ [pid]: 1 }, new Date().toISOString().slice(0, 10)); + await hook('prompt-submit', 'copilot', { cwd: root, hook_event_name: 'UserPromptSubmit', prompt: 'after the upgrade' }); + await hook('stop', 'copilot', { cwd: root, hook_event_name: 'Stop' }); + + expect(await reportedPrompts(project)).toBe(1); + expect(await reportedInterventionSessions(project)).toBe(0); + }); + + it.each([ + ['retained', false, false], + ['compacted', true, false], + ['seeded before reuse', false, true], + ])('a path-keyed project run does not inherit a shared user snapshot (%s)', async (_, compact, seedFirst) => { + const { root, project } = await setup(); + const pid = `pid-${process.ppid}`; + await asEarlierRelease(async () => { + await session('copilot', { cwd: tmp }); + await hook('session-end', 'copilot', { cwd: tmp, hook_event_name: 'SessionEnd' }); + }); + writeSharedSnapshots({ [pid]: 1 }, new Date().toISOString().slice(0, 10)); + if (seedFirst) expect(await report(project)).toBeNull(); + if (compact) fs.writeFileSync(path.join(teamaiHome(), 'dashboard', 'events.jsonl'), ''); + await asEarlierRelease(() => session('copilot', { cwd: root }), getDataHome(project)); + + expect(await reportedSessions(project)).toBe(1); + expect(await reportedInterventionSessions(project)).toBe(1); + expect(await reportedPrompts(project)).toBe(1); + }); + + it('a delayed monitor exit does not split the next invocation with the same ID', async () => { + const { root, project } = await setup(); + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + await session('copilot', { cwd: root }); + const observed = JSON.parse(fs.readFileSync(log, 'utf8').trim().split('\n').at(-1) ?? '{}'); + await hook('session-end', 'copilot', { cwd: root, hook_event_name: 'SessionEnd' }); + await hook('session-start', 'copilot', { cwd: root, hook_event_name: 'SessionStart' }); + fs.appendFileSync(log, JSON.stringify({ + type: 'process_exit', sessionId: observed.sessionId, tool: 'copilot', + timestamp: new Date().toISOString(), dataHomeKey: observed.dataHomeKey, + processExitAfter: observed.timestamp, + }) + '\n'); + await hook('prompt-submit', 'copilot', { cwd: root, hook_event_name: 'UserPromptSubmit', prompt: 'second run' }); + await hook('stop', 'copilot', { cwd: root, hook_event_name: 'Stop' }); - expect(await reportedPrompts(project)).toBe(3); - expect(await reportedPrompts(projectQ)).toBe(2); + expect(await reportedInterventionSessions(project)).toBe(2); + expect(await reportedSessions(project)).toBe(2); + expect(await reportedPrompts(project)).toBe(2); + }); + + it.each([ + ['a shared snapshot, before #785', false], + ['the scope\'s own snapshot, main since #795', true], + ])('two runs an earlier release reported under one bare ID are not sent again: %s', async (_, ownSnapshot) => { + const { root, project } = await setup(); + // No session ID: a fallback ID the second run reuses. That release summed + // both runs under it; Claude's fallback carries the cwd, Copilot's does not. + const [tool, payload] = ownSnapshot ? ['copilot', {}] : ['claude', { cwd: root }]; + const run = async () => { + await session(tool, { ...payload, cwd: root }); + await hook('session-end', tool, { ...payload, cwd: root, hook_event_name: 'SessionEnd' }); + }; + await asEarlierRelease(async () => { await run(); await run(); }, ownSnapshot ? getDataHome(project) : undefined); + const events = fs.readFileSync(path.join(teamaiHome(), 'dashboard', 'events.jsonl'), 'utf-8'); + const id: string = JSON.parse(events.split('\n')[0]).sessionId; + writeSharedSnapshots({ [id]: 2 }, new Date().toISOString().slice(0, 10), ownSnapshot ? project : undefined); + + expect(await report(project)).toBeNull(); + }); + + it('a session resumed in a project after it ended is still the one the user scope reported', async () => { + const { root, user, project } = await setup(); + const elsewhere = path.join(tmp, 'elsewhere'); + fs.mkdirSync(elsewhere); + await session('claude', { session_id: 'resumed', cwd: elsewhere }); + await hook('session-end', 'claude', { session_id: 'resumed', cwd: elsewhere, hook_event_name: 'SessionEnd' }); + expect(await reportedSessions(user)).toBe(1); + // `claude --resume` in P: the same session ID, whose Stop carries the whole transcript. + await session('claude', { session_id: 'resumed', cwd: root }); + + expect(await reportedSessions(project)).toBe(0); + expect(await reportedSessions(user)).toBe(1); + expect(await reportedPrompts(user)).toBe(2); + }); + + it('a session resumed in another project after compaction dropped its events stays its first project\'s', async () => { + const { root, project } = await setup(); + const { rootQ, projectQ } = await setupQ(); + await session('claude', { session_id: 'resumed', cwd: root }); + await hook('session-end', 'claude', { session_id: 'resumed', cwd: root, hook_event_name: 'SessionEnd' }); + expect(await reportedSessions(project)).toBe(1); + fs.writeFileSync(path.join(teamaiHome(), 'dashboard', 'events.jsonl'), ''); + // `claude --resume` in Q: its Stop carries P's transcript, which Q never reported. + await session('claude', { session_id: 'resumed', cwd: rootQ }); + + expect(await report(projectQ)).toBeNull(); + expect(await reportedSessions(project)).toBe(1); + expect(await reportedPrompts(project)).toBe(1); + }); + + it('a session main reported in a project, resumed in another after compaction, stays the first project\'s', async () => { + const { project } = await setup(); + const { rootQ, projectQ } = await setupQ(); + // Main reported `resumed` in P: only P's own snapshot holds it, and main kept no owners. + writeSharedSnapshots({ resumed: 1 }, new Date().toISOString().slice(0, 10), project); + // Compaction dropped its events; `claude --resume` in Q. + await session('claude', { session_id: 'resumed', cwd: rootQ }); + + expect(await report(projectQ)).toBeNull(); + expect(await report(project)).toBeNull(); + }); + + /** `claude --resume` of `sessionId` in `cwd`, whose Stop carries the transcript's `prompts`. */ + async function resume(sessionId: string, cwd: string, prompts: number): Promise { + await session('claude', { session_id: sessionId, cwd }); + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + fs.writeFileSync(log, fs.readFileSync(log, 'utf-8').split('\n').filter(Boolean).map((line) => { + const event = JSON.parse(line); + return JSON.stringify(event.sessionId === sessionId && event.type === 'stop' ? { ...event, prompts } : event); + }).join('\n') + '\n'); + } + + it('main split a session between two projects: the one holding the greater total owns it', async () => { + const { root, project } = await setup(); + const { projectQ } = await setupQ(); + const today = new Date().toISOString().slice(0, 10); + // Main reported 3 prompts to P before the session moved, then its cumulative 5 to Q. + writeSharedSnapshots({ moved: 3 }, today, project); + writeSharedSnapshots({ moved: 5 }, today, projectQ); + // Compaction dropped its events; resumed in P, its Stop carries 6. + await resume('moved', root, 6); + + expect(await report(project)).toBeNull(); + expect(await reportedPrompts(projectQ)).toBe(1); + }); + + it('equal totals copied from the shared snapshot name no owner: the scope it resumes in reports it', async () => { + const { root, user, project } = await setup(); + const today = new Date().toISOString().slice(0, 10); + // Main copied the shared entry into both scopes' snapshots: they tie. + writeSharedSnapshots({ copied: 1 }, today); + writeSharedSnapshots({ copied: 1 }, today, user); + writeSharedSnapshots({ copied: 1 }, today, project); + await resume('copied', root, 2); + + expect(await report(user)).toBeNull(); + expect(await reportedPrompts(project)).toBe(1); + }); + + it('a sole copy of a shared entry names no owner: a resume in the scope that reported it counts there', async () => { + const { user, project } = await setup(); + const elsewhere = path.join(tmp, 'elsewhere'); + fs.mkdirSync(elsewhere); + const today = new Date().toISOString().slice(0, 10); + // Only P pulled on main, so only P's snapshot holds a copy of the user-scope session. + writeSharedSnapshots({ 'user-session': 1 }, today); + writeSharedSnapshots({ 'user-session': 1 }, today, project); + await resume('user-session', elsewhere, 2); + + expect(await report(project)).toBeNull(); + expect(await reportedPrompts(user)).toBe(1); + }); + + it('a copy the shared intervention snapshot alone holds does not let a report claim it', async () => { + const { root, user, project } = await setup(); + const elsewhere = path.join(tmp, 'elsewhere'); + fs.mkdirSync(elsewhere); + const today = new Date().toISOString().slice(0, 10); + // A session reported with no prompts: only intervention entries, copied into P. + writeSharedSnapshots({ quiet: 0 }, today); + writeSharedSnapshots({ quiet: 0 }, today, project); + for (const dir of [path.join(teamaiHome(), 'dashboard'), path.join(getDataHome(project), 'dashboard')]) { + for (const name of ['prompt-tokens', 'daily-sessions']) fs.rmSync(path.join(dir, `reported-${name}.json`)); + } + // P reports another session first, and records owners from its snapshots. + await session('claude', { session_id: 'p-other', cwd: root }); + const reportedByP = await report(project); + await resume('quiet', elsewhere, 1); + + expect(await report(project)).toEqual(reportedByP); + expect(await reportedPrompts(user)).toBe(1); + }); + + it('main split a session before any Stop and compaction dropped it: the owner credits both parts', async () => { + const { root, project } = await setup(); + const { projectQ } = await setupQ(); + const today = new Date().toISOString().slice(0, 10); + // 3 prompts reported in P and 2 in Q, as submit counts: no Stop, so no daily entry. + writeSharedSnapshots({ split: 3 }, today, project); + writeSharedSnapshots({ split: 2 }, today, projectQ); + for (const config of [project, projectQ]) fs.rmSync(path.join(getDataHome(config), 'dashboard', 'reported-daily-sessions.json')); + // Resumed in P; its cumulative Stop carries 6. + await resume('split', root, 6); + + expect((await reportedPrompts(project)) + (await reportedPrompts(projectQ))).toBe(1); + }); + + it.each([ + // P's 3 prompts came before Q's cumulative Stop of 5, which counts them already. + ['before', ['p', 'p', 'p', 'q', 'q'], 3, 5, 1], + // Q's Stop counted 2; P's 3 prompts came after it. + ['after', ['q', 'q', 'p', 'p', 'p'], 3, 2, 1], + ])('a compacted split session whose part with no Stop came %s the other part\'s Stop is credited by its transcript', + async (_, order, loose, stop, expected) => { + const { root, project } = await setup(); + const { rootQ, projectQ } = await setupQ(); + const today = new Date().toISOString().slice(0, 10); + writeSharedSnapshots({ split: loose }, today, project); + fs.rmSync(path.join(getDataHome(project), 'dashboard', 'reported-daily-sessions.json')); + writeSharedSnapshots({ split: stop }, today, projectQ); + // The transcript holds every prompt in order, each with the directory it was typed in. + const transcript = path.join(tmp, 'split.jsonl'); + const cwdOf = (at: string) => (at === 'p' ? root : rootQ); + fs.writeFileSync(transcript, [...order, 'q'].map((at, i) => JSON.stringify({ + type: 'user', sessionId: 'split', uuid: `u${i}`, cwd: cwdOf(at), message: { role: 'user', content: `turn ${i}` }, + })).join('\n') + '\n'); + // Resumed in Q, one new prompt: the Stop's transcript total is 6. + await session('claude', { session_id: 'split', cwd: rootQ, transcript_path: transcript }); + + expect((await reportedPrompts(project)) + (await reportedPrompts(projectQ))).toBe(expected); + }); + + /** Codex events of one session, as the collector records them, keyed to `config`'s scope. */ + async function codexLog(config: LocalConfig) { + const key = await dataHomeKeyOf(config); + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + fs.mkdirSync(path.dirname(log), { recursive: true }); + const base = Date.now() - 3_600_000; + const event = (rollout: string, type: string, minute: number, extra: Record = {}) => JSON.stringify({ + type, timestamp: new Date(base + minute * 60_000).toISOString(), sessionId: 'codex-s', tool: 'codex', dataHomeKey: key, + transcriptPath: path.join(tmp, rollout), ...extra, + }); + const write = (lines: string[]) => fs.writeFileSync(log, lines.join('\n') + '\n'); + const stats = async () => { + const reported = await report(config); + const day = reported && typeof reported === 'object' && 'daily' in reported && reported.daily && typeof reported.daily === 'object' + ? Object.values(reported.daily)[0] : undefined; + const interventions = reported && typeof reported === 'object' && 'interventions' in reported ? reported.interventions : undefined; + const prompts = reported && typeof reported === 'object' && 'prompts' in reported ? reported.prompts : undefined; + return { day, interventions, prompts }; + }; + return { event, write, stats }; + } + + it('a Codex rollout with no token record is kept per rollout too', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + write([event('rollout-a.jsonl', 'stop', 0, { prompts: 5 })]); + await stats(); + write([event('rollout-b.jsonl', 'stop', 30, { prompts: 2 })]); + + expect((await stats()).prompts).toBe(7); + }); + + it('a tcodex rollout with no token record is kept per rollout too', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + write([event('rollout-a.jsonl', 'stop', 0, { tool: 'tcodex', prompts: 5 })]); + await stats(); + write([event('rollout-b.jsonl', 'stop', 30, { tool: 'tcodex', prompts: 2 })]); + + expect((await stats()).prompts).toBe(7); + }); + + it('a Codex rollout whose Stop records its cost as requestMetrics keeps that cost', async () => { + const { project } = await setup(); + const { event, write } = await codexLog(project); + // An older Stop: its cost as one request record, on the Stop's day. + write([event('rollout-a.jsonl', 'stop', 0, { + prompts: 1, requestMetrics: { pricedRequests: 1, costMicros: 40, cacheReadTokens: 0, cacheEligibleInputTokens: 0, priceVersion: 'v1' }, + })]); + const reported = await report(project); + const days = reported && typeof reported === 'object' && 'daily' in reported && reported.daily && typeof reported.daily === 'object' + ? Object.values(reported.daily) : []; + + expect(days.reduce((sum: number, d: unknown) => + sum + (d && typeof d === 'object' && 'costMicros' in d && typeof d.costMicros === 'number' ? d.costMicros : 0), 0)).toBe(40); + }); + + it('a dropped Codex rollout keeps its corrections', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + const withCorrection = (rollout: string, minute: number) => [ + event(rollout, 'stop', minute, { prompts: 1 }), + event(rollout, 'prompt_submit', minute + 0.1, { correction: true, promptSummary: 'no, not that' }), + event(rollout, 'stop', minute + 0.2, { prompts: 2 }), + ]; + write(withCorrection('rollout-a.jsonl', 0)); + await stats(); + write(withCorrection('rollout-b.jsonl', 30)); + + expect((await stats()).interventions).toMatchObject({ correction: 2 }); + }); + + it('a dropped Codex rollout keeps its duration and cache tokens', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + const tokens = (cacheRead: number) => ({ tokenScope: 'transcript', tokens: { input: 10, output: 1, cacheRead, cacheCreation: 0 } }); + // Rollout A: 8 active minutes and 100 cache-read tokens; B: 1 minute and 20. + write([event('rollout-a.jsonl', 'prompt_submit', 0), event('rollout-a.jsonl', 'prompt_submit', 4), + event('rollout-a.jsonl', 'stop', 8, { prompts: 2, ...tokens(100) })]); + await stats(); + write([event('rollout-b.jsonl', 'prompt_submit', 30), event('rollout-b.jsonl', 'stop', 31, { prompts: 1, ...tokens(20) })]); + const { day } = await stats(); + + expect(day).toMatchObject({ durationMs: 9 * 60_000, cacheReadTokens: 120 }); + }); + + it('a Codex session\'s request costs sum its rollouts still in the log', async () => { + const { project } = await setup(); + const { event, write } = await codexLog(project); + const day = new Date().toISOString().slice(0, 10); + const cost = (costMicros: number) => ({ + requestDaily: { [day]: { pricedRequests: 1, costMicros, cacheReadTokens: 0, cacheEligibleInputTokens: 0, priceVersion: 'v1' } }, + }); + write([event('rollout-a.jsonl', 'stop', 0, { prompts: 1, ...cost(100) })]); + await report(project); + // Rollout A is still in the log when rollout B adds its own cost. + write([event('rollout-a.jsonl', 'stop', 0, { prompts: 1, ...cost(100) }), event('rollout-b.jsonl', 'stop', 30, { prompts: 1, ...cost(20) })]); + const reported = await report(project); + const days = reported && typeof reported === 'object' && 'daily' in reported && reported.daily && typeof reported.daily === 'object' + ? Object.values(reported.daily) : []; + + expect(days.reduce((sum: number, d: unknown) => + sum + (d && typeof d === 'object' && 'costMicros' in d && typeof d.costMicros === 'number' ? d.costMicros : 0), 0)).toBe(120); + }); + + it('a dropped Codex rollout that failed keeps the session failed', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + // Rollout A was interrupted; rollout B, after compaction, finishes cleanly. + write([event('rollout-a.jsonl', 'stop', 0, { prompts: 1, interventions: { interrupt: 1, toolReject: 0 } })]); + await stats(); + write([event('rollout-b.jsonl', 'stop', 30, { prompts: 1, interventions: { interrupt: 0, toolReject: 0 } })]); + + expect((await stats()).day).toMatchObject({ sessionsSucceeded: 0 }); + }); + + it('a Codex session with a session-scoped token counter still keeps its other metrics per rollout', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + const day = new Date().toISOString().slice(0, 10); + // The thread-level token counter spans rollouts; prompts and costs restart with each. + const stop = (rollout: string, minute: number, prompts: number, costMicros: number, input: number) => event(rollout, 'stop', minute, { + prompts, tokenScope: 'session', tokens: { input, output: 0, cacheRead: 0, cacheCreation: 0 }, + requestDaily: { [day]: { pricedRequests: 1, costMicros, cacheReadTokens: 0, cacheEligibleInputTokens: 0, priceVersion: 'v1' } }, + }); + write([stop('rollout-a.jsonl', 0, 5, 100, 500)]); + await stats(); + write([stop('rollout-b.jsonl', 30, 2, 20, 530)]); + const after = await stats(); + const reported = await report(project); + const tokens = reported && typeof reported === 'object' && 'tokens' in reported ? reported.tokens : undefined; + // Costs go to their request's day, which need not be the session's. + const days = reported && typeof reported === 'object' && 'daily' in reported && reported.daily && typeof reported.daily === 'object' + ? Object.values(reported.daily) : []; + const costMicros = days.reduce((sum: number, d: unknown) => + sum + (d && typeof d === 'object' && 'costMicros' in d && typeof d.costMicros === 'number' ? d.costMicros : 0), 0); + + expect(after.prompts).toBe(7); + expect(costMicros).toBe(120); + expect(tokens).toMatchObject({ input: 530 }); + }); + + it('a dropped Codex rollout keeps the prompts its submits counted', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + // Codex Stops carry no prompt count: the submits of each rollout count its prompts. + write([event('rollout-a.jsonl', 'prompt_submit', 0), event('rollout-a.jsonl', 'prompt_submit', 1), + event('rollout-a.jsonl', 'prompt_submit', 2), event('rollout-a.jsonl', 'stop', 3)]); + await stats(); + write([event('rollout-b.jsonl', 'prompt_submit', 30), event('rollout-b.jsonl', 'stop', 31)]); + + expect((await stats()).prompts).toBe(4); + }); + + it('a whole entry an earlier release left adds no tokens to a counter that spans rollouts', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + // That release reported rollout A as one entry: 5 prompts, the counter at 500. + writeSharedSnapshots({ 'codex-s': 5 }, new Date().toISOString().slice(0, 10), project); + const file = path.join(getDataHome(project), 'dashboard', 'reported-prompt-tokens.json'); + fs.writeFileSync(file, JSON.stringify({ 'codex-s': { prompts: 5, tokens: { input: 500, output: 0, cacheRead: 0, cacheCreation: 0 } } })); + const past = new Date(Date.now() - 2 * 3_600_000); + for (const name of SNAPSHOTS) fs.utimesSync(path.join(getDataHome(project), 'dashboard', `reported-${name}.json`), past, past); + // A was compacted; rollout B, begun later, shows the thread-level counter at 530. + write([event('rollout-b.jsonl', 'stop', 30, { + prompts: 2, tokenScope: 'session', tokens: { input: 530, output: 0, cacheRead: 0, cacheCreation: 0 }, + })]); + await stats(); + const reported = await report(project); + const tokens = reported && typeof reported === 'object' && 'tokens' in reported ? reported.tokens : undefined; + + expect(tokens).toMatchObject({ input: 30 }); + }); + + it('a whole daily entry from before request costs were kept per day keeps its cost for a later rollout', async () => { + const { project } = await setup(); + const { event, write } = await codexLog(project); + const today = new Date().toISOString().slice(0, 10); + // An earlier release reported rollout A: its daily entry holds the cost as session fields. + writeSharedSnapshots({ 'codex-s': 5 }, today, project); + const dir = path.join(getDataHome(project), 'dashboard'); + fs.writeFileSync(path.join(dir, 'reported-daily-sessions.json'), JSON.stringify({ 'codex-s': { + date: today, prompts: 5, durationMs: 0, succeeded: 1, corrected: 0, + pricedRequests: 1, costMicros: 100, cacheReadTokens: 0, cacheEligibleInputTokens: 0, priceVersion: 'v1', + } })); + const past = new Date(Date.now() - 2 * 3_600_000); + for (const name of SNAPSHOTS) fs.utimesSync(path.join(dir, `reported-${name}.json`), past, past); + // A was compacted; rollout B, begun later, costs $20 on the same day. + write([event('rollout-b.jsonl', 'stop', 30, { + prompts: 2, requestDaily: { [today]: { pricedRequests: 1, costMicros: 20, cacheReadTokens: 0, cacheEligibleInputTokens: 0, priceVersion: 'v1' } }, + })]); + const reported = await report(project); + const days = reported && typeof reported === 'object' && 'daily' in reported && reported.daily && typeof reported.daily === 'object' + ? Object.values(reported.daily) : []; + + expect(days.reduce((sum: number, d: unknown) => + sum + (d && typeof d === 'object' && 'costMicros' in d && typeof d.costMicros === 'number' ? d.costMicros : 0), 0)).toBe(20); + }); + + it('a whole entry from before covers a rollout still running only as far as it had got', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + const today = new Date().toISOString().slice(0, 10); + // That release reported 6 prompts: rollout A's 5 and rollout B's first. + writeSharedSnapshots({ 'codex-s': 6 }, today, project); + const dir = path.join(getDataHome(project), 'dashboard'); + const writtenAt = new Date(Date.now() - 20 * 60_000); + for (const name of SNAPSHOTS) fs.utimesSync(path.join(dir, `reported-${name}.json`), writtenAt, writtenAt); + // A was compacted; B had 1 prompt by then, and has 3 now. + write([event('rollout-b.jsonl', 'stop', 30, { prompts: 1 }), event('rollout-b.jsonl', 'stop', 50, { prompts: 3 })]); + + expect((await stats()).prompts).toBe(2); + }); + + it('a whole entry from before covers what its report read, not what came in during its push', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + const today = new Date().toISOString().slice(0, 10); + // That release read the log (A 5, B 1), wrote the team stats file, pushed, then its snapshot. + writeSharedSnapshots({ 'codex-s': 6 }, today, project); + const dir = path.join(getDataHome(project), 'dashboard'); + const teamStats = path.join(path.dirname(project.repo.localPath), 'reports-wt', 'stats', 'tester.yaml'); + fs.mkdirSync(path.dirname(teamStats), { recursive: true }); + fs.writeFileSync(teamStats, YAML.stringify({ username: 'tester', skills: {} })); + const read = new Date(Date.now() - 25 * 60_000); + const pushed = new Date(Date.now() - 15 * 60_000); + fs.utimesSync(teamStats, read, read); + for (const name of SNAPSHOTS) fs.utimesSync(path.join(dir, `reported-${name}.json`), pushed, pushed); + // A was compacted; B reached 2 during that push and has 3 now. + write([event('rollout-b.jsonl', 'stop', 30, { prompts: 1 }), event('rollout-b.jsonl', 'stop', 40, { prompts: 2 }), + event('rollout-b.jsonl', 'stop', 55, { prompts: 3 })]); + + expect((await stats()).prompts).toBe(2); + }); + + it('reading the baselines without persisting them, as teamai stats does, does not move the time they cover', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + // An earlier release reported rollout A into the shared snapshot, then compaction dropped it. + writeSharedSnapshots({ 'codex-s': 5 }, new Date().toISOString().slice(0, 10)); + const past = new Date(Date.now() - 2 * 3_600_000); + for (const name of SNAPSHOTS) fs.utimesSync(shared(name), past, past); + // Rollout B began after that, and the baselines are read before the next pull. + write([event('rollout-b.jsonl', 'stop', 30, { prompts: 2 })]); + const { reportedBaselines } = await import('../team-push.js'); + const { filterEventsByScope } = await import('../dashboard-scope.js'); + const { readEvents, aggregateSessionMetrics } = await import('../dashboard-collector.js'); + const { aggregateDailySessions } = await import('../session-trends.js'); + const events = await filterEventsByScope(await readEvents(), project); + await reportedBaselines(events, aggregateSessionMetrics(events), aggregateDailySessions(events), project, false); + + expect(fs.existsSync(path.join(getDataHome(project), 'dashboard', 'reported-prompt-tokens.json'))).toBe(false); + expect((await stats()).prompts).toBe(2); + }); + + it('a rollout\'s intervention change alone still updates its kept totals', async () => { + const { project } = await setup(); + const { event, write, stats } = await codexLog(project); + const stop = (rollout: string, minute: number, toolReject: number) => + event(rollout, 'stop', minute, { prompts: 1, interventions: { interrupt: 0, toolReject } }); + write([stop('rollout-a.jsonl', 0, 1)]); + await stats(); + // Only the rejection count moves; prompts and tokens stay the same. + write([stop('rollout-a.jsonl', 0, 1), stop('rollout-a.jsonl', 1, 2)]); + await stats(); + write([stop('rollout-b.jsonl', 30, 1)]); + + expect((await stats()).interventions).toMatchObject({ toolReject: 3 }); + }); + + it('a Codex rollout resumed after compaction reports its daily prompts and interventions too', async () => { + const { project } = await setup(); + const key = await dataHomeKeyOf(project); + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + fs.mkdirSync(path.dirname(log), { recursive: true }); + const stop = (rollout: string, prompts: number, toolReject: number) => JSON.stringify({ + type: 'stop', timestamp: new Date().toISOString(), sessionId: 'codex-s', tool: 'codex', dataHomeKey: key, + transcriptPath: path.join(tmp, rollout), tokenScope: 'transcript', prompts, + tokens: { input: 10, output: 1, cacheRead: 0, cacheCreation: 0 }, interventions: { interrupt: 0, toolReject }, + }); + fs.writeFileSync(log, stop('rollout-a.jsonl', 5, 1) + '\n'); + await report(project); + // Compaction dropped rollout A; the resumed rollout B restarts its counters. + fs.writeFileSync(log, stop('rollout-b.jsonl', 2, 1) + '\n'); + const stats = await report(project); + const day = stats && typeof stats === 'object' && 'daily' in stats && stats.daily && typeof stats.daily === 'object' + ? Object.values(stats.daily)[0] : undefined; + const interventions = stats && typeof stats === 'object' && 'interventions' in stats ? stats.interventions : undefined; + + expect(day).toMatchObject({ promptTurns: 7 }); + expect(interventions).toMatchObject({ toolReject: 2 }); + }); + + it('a compacted split session credits the cumulative Stop interventions once', async () => { + const { user, project } = await setup(); + const { rootQ, projectQ } = await setupQ(); + const today = new Date().toISOString().slice(0, 10); + // P's Stop reported 3 prompts and 1 interruption; Q's later, cumulative Stop + // 5 prompts and 2; the user scope 1 prompt before any Stop (no daily entry). + writeSharedSnapshots({ split: 3 }, today, project); + writeSharedSnapshots({ split: 5 }, today, projectQ); + writeSharedSnapshots({ split: 1 }, today, user); + fs.rmSync(path.join(teamaiHome(), 'dashboard', 'user-reported-daily-sessions.json')); + const interrupts = (config: LocalConfig, file: string, interrupt: number) => fs.writeFileSync( + path.join(getDataHome(config), 'dashboard', file), + JSON.stringify({ split: { interrupt, toolReject: 0, correction: 0 } })); + interrupts(project, 'reported-interventions.json', 1); + interrupts(projectQ, 'reported-interventions.json', 2); + // Resumed in Q: its next cumulative Stop carries 7 prompts and 3 interruptions. + await resume('split', rootQ, 7); + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + fs.writeFileSync(log, fs.readFileSync(log, 'utf-8').split('\n').filter(Boolean).map((line) => { + const event = JSON.parse(line); + return JSON.stringify(event.type === 'stop' ? { ...event, interventions: { interrupt: 3, toolReject: 0 } } : event); + }).join('\n') + '\n'); + const stats = await report(projectQ); + const interventions = stats && typeof stats === 'object' && 'interventions' in stats ? stats.interventions : undefined; + + expect(interventions).toMatchObject({ interrupt: 1 }); + }); + + it('a project that reported a session past the shared snapshot\'s total owns it', async () => { + const { project } = await setup(); + const { rootQ, projectQ } = await setupQ(); + const today = new Date().toISOString().slice(0, 10); + // The shared file held 1 prompt; P, seeded from it, then reported through prompt 2. + writeSharedSnapshots({ resumed: 1 }, today); + writeSharedSnapshots({ resumed: 2 }, today, project); + await resume('resumed', rootQ, 3); + + expect(await report(projectQ)).toBeNull(); + expect(await reportedPrompts(project)).toBe(1); + }); + + it('a session a project reported with no prompts, only its intervention count, stays that project\'s', async () => { + const { project } = await setup(); + const { rootQ, projectQ } = await setupQ(); + writeSharedSnapshots({ quiet: 0 }, new Date().toISOString().slice(0, 10), project); + for (const name of ['prompt-tokens', 'daily-sessions']) { + fs.rmSync(path.join(getDataHome(project), 'dashboard', `reported-${name}.json`)); + } + await resume('quiet', rootQ, 0); + + expect(await reportedInterventionSessions(projectQ)).toBe(0); + }); + + it('a session of a project whose data home is in its workspace stays that project\'s', async () => { + await setup(); + const { rootQ, projectQ } = await setupQ(); + const { rootW, projectW } = await setupWorkspaceProject(); + expect(getDataHome(projectW)).toBe(path.join(rootW, '.teamai')); + writeSharedSnapshots({ resumed: 1 }, new Date().toISOString().slice(0, 10), projectW); + // Another of W's sessions is still in the log. + await session('claude', { session_id: 'other-w', cwd: rootW }); + await resume('resumed', rootQ, 2); + + expect(await report(projectQ)).toBeNull(); + }); + + /** A project with no git repo, so its data home is `/.teamai`, under no partition. */ + async function setupWorkspaceProject(): Promise<{ rootW: string; projectW: LocalConfig }> { + const rootW = path.join(tmp, 'plain-dir'); + fs.mkdirSync(rootW); + const dataHomeW = await resolveProjectDataHome(rootW); + fs.mkdirSync(path.join(dataHomeW, 'team-repo'), { recursive: true }); + await saveLocalConfigForScope({ + repo: { localPath: path.join(dataHomeW, 'team-repo'), remote: 'https://example.test/acme/team-w.git', kind: 'git' }, + username: 'tester', scope: 'project', projectRoot: rootW, additionalRoles: [], dataHome: dataHomeW, + }); + const projectW = await resolveConfigForDir(rootW); + if (!projectW) throw new Error('fixture config W did not resolve'); + return { rootW, projectW }; + } + + it.each([ + ['claude', (cwd: string) => ({ type: 'user', sessionId: 'resumed', cwd, message: { role: 'user', content: 'start' } })], + ['codex', (cwd: string) => ({ type: 'session_meta', payload: { id: 'resumed', cwd } })], + ])('a %s session resumed elsewhere after compaction goes to the project its transcript started in', async (tool, first) => { + await setup(); + const { rootQ, projectQ } = await setupQ(); + const { rootW, projectW } = await setupWorkspaceProject(); + // An earlier release reported it in W; compaction dropped every W event. + writeSharedSnapshots({ resumed: 1 }, new Date().toISOString().slice(0, 10), projectW); + // The resume appends to the transcript the session started, whose first cwd is W. + const transcript = path.join(tmp, `${tool}-resumed.jsonl`); + fs.writeFileSync(transcript, JSON.stringify(first(rootW)) + '\n'); + await session(tool, { session_id: 'resumed', cwd: rootQ, transcript_path: transcript }); + + expect(await report(projectQ)).toBeNull(); + }); + + it('a Copilot session resumed elsewhere after compaction goes to the project its session log started in', async () => { + await setup(); + const { rootQ, projectQ } = await setupQ(); + const { rootW, projectW } = await setupWorkspaceProject(); + writeSharedSnapshots({ 'copilot-w': 1 }, new Date().toISOString().slice(0, 10), projectW); + // Copilot's own session log, found by the session ID: TeamAI stores no path of it (#666). + const log = path.join(tmp, 'home', '.copilot', 'session-state', 'copilot-w', 'events.jsonl'); + fs.mkdirSync(path.dirname(log), { recursive: true }); + fs.writeFileSync(log, JSON.stringify({ type: 'session.start', data: { context: { cwd: rootW } } }) + '\n'); + await session('copilot', { session_id: 'copilot-w', cwd: rootQ }); + + expect(await report(projectQ)).toBeNull(); + }); + + it('main split a session across projects before any Stop: the owner credits what each already reported', async () => { + const { root, project } = await setup(); + const { rootQ, projectQ } = await setupQ(); + // Main attributed each event to its scope by `dataHome` and reported each + // part as its own count: 3 prompts in P, then 2 in Q, no Stop yet. + const at = (s: number, type: string, cwd: string, dataHome: string) => JSON.stringify({ + type, timestamp: new Date(Date.now() - 600_000 + s * 1000).toISOString(), sessionId: 'split', tool: 'claude', cwd, dataHome, + }); + const p = getDataHome(project); + const q = getDataHome(projectQ); + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + fs.mkdirSync(path.dirname(log), { recursive: true }); + fs.writeFileSync(log, [ + at(0, 'session_start', root, p), at(1, 'prompt_submit', root, p), at(2, 'prompt_submit', root, p), at(3, 'prompt_submit', root, p), + at(10, 'prompt_submit', rootQ, q), at(11, 'prompt_submit', rootQ, q), + ].join('\n') + '\n'); + const today = new Date().toISOString().slice(0, 10); + writeSharedSnapshots({ split: 3 }, today, project); + writeSharedSnapshots({ split: 2 }, today, projectQ); + + expect(await report(project)).toBeNull(); + expect(await report(projectQ)).toBeNull(); + }); + + it('main split a session with the same prompts in each part: the owner still credits every part\'s active time', async () => { + const { root, project } = await setup(); + const { rootQ, projectQ } = await setupQ(); + const p = getDataHome(project); + const q = getDataHome(projectQ); + const tokens = (input: number) => ({ input, output: 0, cacheRead: 0, cacheCreation: 0 }); + const at = (minute: number, type: string, cwd: string, dataHome: string, extra: Record = {}) => JSON.stringify({ + type, timestamp: new Date(Date.now() - 3_600_000 + minute * 60_000).toISOString(), sessionId: 'split', tool: 'claude', cwd, dataHome, ...extra, + }); + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + fs.mkdirSync(path.dirname(log), { recursive: true }); + // One prompt. P's part was active 4 minutes; Q's, after a pause, 3 more, with no new prompt. + fs.writeFileSync(log, [ + at(0, 'prompt_submit', root, p), at(4, 'stop', root, p, { prompts: 1, tokens: tokens(100) }), + at(10, 'tool_use', rootQ, q), at(12, 'tool_use', rootQ, q), at(13, 'stop', rootQ, q, { prompts: 1, tokens: tokens(150) }), + ].join('\n') + '\n'); + const date = new Date(Date.now() - 3_600_000).toISOString().slice(0, 10); + for (const [config, input, minutes] of [[project, 100, 4], [projectQ, 150, 3]] as const) { + const dir = path.join(getDataHome(config), 'dashboard'); + fs.mkdirSync(dir, { recursive: true }); + fs.writeFileSync(path.join(dir, 'reported-prompt-tokens.json'), JSON.stringify({ split: { prompts: 1, tokens: tokens(input) } })); + fs.writeFileSync(path.join(dir, 'reported-interventions.json'), JSON.stringify({ split: { interrupt: 0, toolReject: 0, correction: 0 } })); + fs.writeFileSync(path.join(dir, 'reported-daily-sessions.json'), JSON.stringify({ split: { + date, prompts: 1, durationMs: minutes * 60_000, succeeded: 1, corrected: 0, requestDaily: {}, + } })); + } + + // Both parts already reported their time: the owner sends none of it again. + expect(await report(projectQ)).toBeNull(); + expect(await report(project)).toBeNull(); + }); + + it('main split a session whose parts each ended in a cumulative Stop: a new prompt is reported once', async () => { + const { root, project } = await setup(); + const { rootQ, projectQ } = await setupQ(); + const at = (s: number, type: string, cwd: string, dataHome: string, extra: Record = {}) => JSON.stringify({ + type, timestamp: new Date(Date.now() - 600_000 + s * 1000).toISOString(), sessionId: 'split', tool: 'claude', cwd, dataHome, ...extra, + }); + const p = getDataHome(project); + const q = getDataHome(projectQ); + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + fs.mkdirSync(path.dirname(log), { recursive: true }); + fs.writeFileSync(log, [ + at(0, 'session_start', root, p), at(1, 'prompt_submit', root, p), at(2, 'stop', root, p, { prompts: 3 }), + at(10, 'prompt_submit', rootQ, q), at(11, 'stop', rootQ, q, { prompts: 5 }), + ].join('\n') + '\n'); + const today = new Date().toISOString().slice(0, 10); + // Each Stop carried the transcript's total, so Q's snapshot already spans P's part. + writeSharedSnapshots({ split: 3 }, today, project); + writeSharedSnapshots({ split: 5 }, today, projectQ); + expect(await report(project)).toBeNull(); + expect(await report(projectQ)).toBeNull(); + fs.appendFileSync(log, [at(20, 'prompt_submit', rootQ, q), at(21, 'stop', rootQ, q, { prompts: 6 })].join('\n') + '\n'); + + expect((await reportedPrompts(project)) + (await reportedPrompts(projectQ))).toBe(1); + }); + + it('a transcript that started in a project whose snapshot lacks the session does not hand it there', async () => { + await setup(); + const { rootQ, projectQ } = await setupQ(); + const { rootW } = await setupWorkspaceProject(); + const transcript = path.join(tmp, 'forked.jsonl'); + // A fork copies W's history under a new ID W never reported: Q's as usual. + fs.writeFileSync(transcript, JSON.stringify({ type: 'user', sessionId: 'forked', cwd: rootW }) + '\n'); + await session('claude', { session_id: 'forked', cwd: rootQ, transcript_path: transcript }); + + expect(await reportedSessions(projectQ)).toBe(1); + }); + + it('the session owners a report keeps hold no path (#666)', async () => { + const { root, project } = await setup(); + await session('copilot', { session_id: 'copilot-p', cwd: root }); + await report(project); + + const owners = fs.readFileSync(path.join(teamaiHome(), 'dashboard', 'session-owners.jsonl'), 'utf-8'); + expect(owners).toContain('copilot-p'); + expect(owners).not.toContain(root); + expect(owners).not.toContain(getDataHome(project)); + }); + + it('a session resumed after compaction dropped its events is still the one already reported', async () => { + const { root, project } = await setup(); + await session('claude', { session_id: 'resumed', cwd: root }); + await hook('session-end', 'claude', { session_id: 'resumed', cwd: root, hook_event_name: 'SessionEnd' }); + expect(await reportedSessions(project)).toBe(1); + fs.writeFileSync(path.join(teamaiHome(), 'dashboard', 'events.jsonl'), ''); + // `claude --resume`: the same session ID, compared against what P reported. + await session('claude', { session_id: 'resumed', cwd: root }); + + expect(await reportedSessions(project)).toBe(1); + expect(await reportedPrompts(project)).toBe(1); + }); + + it.each([ + ['a shared snapshot, before #785', false], + ['the scope\'s own snapshot, main since #795', true], + ])('a later run of a bare ID that release never reported is sent: %s', async (_, ownSnapshot) => { + const { root, project } = await setup(); + const [tool, payload] = ownSnapshot ? ['copilot', {}] : ['claude', { cwd: root }]; + const run = async () => { + await session(tool, { ...payload, cwd: root }); + await hook('session-end', tool, { ...payload, cwd: root, hook_event_name: 'SessionEnd' }); + }; + // That release reported the first run; the second came after its last report. + await asEarlierRelease(async () => { await run(); await run(); }, ownSnapshot ? getDataHome(project) : undefined); + const events = fs.readFileSync(path.join(teamaiHome(), 'dashboard', 'events.jsonl'), 'utf-8'); + const id: string = JSON.parse(events.split('\n')[0]).sessionId; + writeSharedSnapshots({ [id]: 1 }, new Date().toISOString().slice(0, 10), ownSnapshot ? project : undefined); + + expect(await reportedSessions(project)).toBe(1); + expect(await reportedInterventionSessions(project)).toBe(1); + expect(await reportedPrompts(project)).toBe(1); + }); + + it('a pre-upgrade exit reported before the next run\'s first prompt does not count that run twice', async () => { + const { project } = await setup(); + const p = await dataHomeKeyOf(project); + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + const at = (type: string, s: number) => + JSON.stringify({ type, timestamp: new Date(Date.now() - 600_000 + s * 1000).toISOString(), sessionId: 'pid-9', tool: 'copilot', dataHomeKey: p }); + fs.mkdirSync(path.dirname(log), { recursive: true }); + // Run 1 ends; run 2 starts; a dashboard from before processExitAfter appends + // the exit it observed for run 1. A pull runs before run 2's first prompt. + fs.writeFileSync(log, [at('session_start', 0), at('prompt_submit', 1), at('stop', 2), at('session_end', 3), + at('session_start', 10), at('process_exit', 11)].join('\n') + '\n'); + expect(await reportedInterventionSessions(project)).toBe(2); + // Run 2 goes on: the exit is now followed by its activity, so it was run 1's, + // and run 2 keeps its ID, compared against what the first pull reported. + fs.appendFileSync(log, [at('prompt_submit', 20), at('stop', 21)].join('\n') + '\n'); + + expect(await reportedInterventionSessions(project)).toBe(2); + expect(await reportedSessions(project)).toBe(2); + expect(await reportedPrompts(project)).toBe(2); + }); + + it('a fallback ID reused in a project after a user-scope run that never ended is the project\'s', async () => { + const { root, user, project } = await setup(); + const elsewhere = path.join(tmp, 'elsewhere'); + fs.mkdirSync(elsewhere); + const log = path.join(teamaiHome(), 'dashboard', 'events.jsonl'); + await session('copilot', { cwd: elsewhere }); + // That invocation crashed with no dashboard running: nothing ended it. Its + // process, recorded at SessionStart, is not the next invocation's. + fs.writeFileSync(log, fs.readFileSync(log, 'utf-8').split('\n').filter(Boolean) + .map((line) => JSON.stringify({ ...JSON.parse(line), ...(line.includes('"monitorPid"') ? { monitorPid: 99999999 } : {}) })) + .join('\n') + '\n'); + await session('copilot', { cwd: root }); + + expect(await reportedSessions(project)).toBe(1); + expect(await reportedPrompts(project)).toBe(1); + expect(await reportedSessions(user)).toBe(1); + expect(await reportedPrompts(user)).toBe(1); + }); + + it('a shared bare entry is consumed in the order of every scope\'s runs, not one scope\'s', async () => { + const { root, user, project } = await setup(); + const elsewhere = path.join(tmp, 'elsewhere'); + fs.mkdirSync(elsewhere); + // A release before #666 recorded Copilot's cwd. It reported the user-scope + // run of `pid-7` as 1 prompt, then recorded P's run of that ID unreported. + const timestamp = (s: number) => new Date(Date.now() - 600_000 + s * 1000).toISOString(); + const event = (type: string, s: number, cwd: string) => ({ type, timestamp: timestamp(s), sessionId: 'pid-7', tool: 'copilot', cwd }); + const log = [ + event('session_start', 0, elsewhere), event('prompt_submit', 1, elsewhere), event('stop', 2, elsewhere), event('session_end', 3, elsewhere), + event('session_start', 10, root), event('prompt_submit', 11, root), event('stop', 12, root), event('session_end', 13, root), + ]; + fs.mkdirSync(path.join(teamaiHome(), 'dashboard'), { recursive: true }); + fs.writeFileSync(path.join(teamaiHome(), 'dashboard', 'events.jsonl'), log.map((e) => JSON.stringify(e)).join('\n') + '\n'); + writeSharedSnapshots({ 'pid-7': 1 }, new Date().toISOString().slice(0, 10)); + + expect(await reportedSessions(project)).toBe(1); + expect(await reportedPrompts(project)).toBe(1); + expect(await report(user)).toBeNull(); }); it('the first report after the upgrade sends nothing a shared snapshot already reported', async () => { const { root, user, project } = await setup(); const elsewhere = path.join(tmp, 'elsewhere'); fs.mkdirSync(elsewhere); - await session('claude', { session_id: 'old-p', cwd: root }); - await session('claude', { session_id: 'old-u', cwd: elsewhere }); + await asEarlierRelease(async () => { + await session('claude', { session_id: 'old-p', cwd: root }); + await session('claude', { session_id: 'old-u', cwd: elsewhere }); + }); writeSharedSnapshots({ 'old-p': 1, 'old-u': 1 }, new Date().toISOString().slice(0, 10)); expect(await report(user)).toBeNull(); @@ -284,9 +1269,56 @@ describe('each scope keeps its own reported snapshot (#786)', () => { expect(await reportedPrompts(user)).toBe(2); }); + it.each([ + ['nothing new', 1], + ['a new prompt', 0], + ])('a bare snapshot entry a run adopts is written back under the run ID only, with %s to report', async (_, reported) => { + const { root, project } = await setup(); + const pid = `pid-${process.ppid}`; + // Main since #795 recorded and reported this run under its bare session ID. + await asEarlierRelease(() => session('copilot', { cwd: root }), getDataHome(project)); + // Main writes this acknowledgement to P's own snapshot, not the shared file. + writeSharedSnapshots({ [pid]: reported }, new Date().toISOString().slice(0, 10), project); + await report(project); + + for (const name of SNAPSHOTS) { + const own = JSON.parse(fs.readFileSync(path.join(getDataHome(project), 'dashboard', `reported-${name}.json`), 'utf-8')); + expect(Object.keys(own)).toEqual([expect.stringMatching(new RegExp(`^${pid}@`))]); + } + // Compaction dropped that run; the next one in P reuses its ID and is a new session. + fs.writeFileSync(path.join(teamaiHome(), 'dashboard', 'events.jsonl'), ''); + await session('copilot', { cwd: root }); + + expect(await reportedSessions(project)).toBe(1); + }); + + it('a session the shared snapshot reported is not sent again after compaction dropped its events', async () => { + const { root, project } = await setup(); + // Reported before #795 into the shared snapshot and compacted; P's first + // pull seeds its own snapshot; then it is resumed in P, its Stop carrying 2. + writeSharedSnapshots({ resumed: 1 }, new Date().toISOString().slice(0, 10)); + expect(await report(project)).toBeNull(); + await resume('resumed', root, 2); + + expect(await reportedPrompts(project)).toBe(1); + }); + + it('a snapshot entry an earlier release wrote without tokens does not stop the report', async () => { + const { root, project } = await setup(); + const pid = `pid-${process.ppid}`; + await asEarlierRelease(() => session('copilot', { cwd: root }), getDataHome(project)); + // Hand-edited or truncated: no `tokens`, under the bare fallback ID. + const dir = path.join(getDataHome(project), 'dashboard'); + fs.mkdirSync(dir, { recursive: true }); + fs.writeFileSync(path.join(dir, 'reported-prompt-tokens.json'), JSON.stringify({ [pid]: { prompts: 1 } })); + + expect(await report(project)).not.toBeNull(); + expect(await reportedPrompts(project)).toBe(0); + }); + it('a scope first seeded after a rollback skips what the earlier release reported', async () => { const { root, project } = await setup(); - await session('claude', { session_id: 'rollback-p', cwd: root }); + await asEarlierRelease(() => session('claude', { session_id: 'rollback-p', cwd: root })); writeSharedSnapshots({ 'rollback-p': 1 }, new Date().toISOString().slice(0, 10)); expect(await report(project)).toBeNull(); diff --git a/src/__tests__/e2e/repo-attribution-worktrees-809.test.ts b/src/__tests__/e2e/repo-attribution-worktrees-809.test.ts index 85eff184c..9f45b4fdc 100644 --- a/src/__tests__/e2e/repo-attribution-worktrees-809.test.ts +++ b/src/__tests__/e2e/repo-attribution-worktrees-809.test.ts @@ -195,7 +195,8 @@ describe('one repo, many worktrees (#809)', () => { it('stats --by-repo shows one row for the repo, counting its worktree sessions', async () => { const r = await runCLI(['stats', '--by-repo'], repo, home); expect(r.code, r.output).toBe(0); - expect(byRepo(r.output), r.output).toEqual([['my-repo', 5]]); + // cd-1 started in HOME: the user scope's, whole (#785), so the repo's own four. + expect(byRepo(r.output), r.output).toEqual([['my-repo', 4]]); }); it('stats --by-repo labels two unrelated repos with the same name apart', async () => { diff --git a/src/__tests__/e2e/report-scope-reuse.test.ts b/src/__tests__/e2e/report-scope-reuse.test.ts new file mode 100644 index 000000000..4fb9ce8be --- /dev/null +++ b/src/__tests__/e2e/report-scope-reuse.test.ts @@ -0,0 +1,230 @@ +import { afterEach, expect, it } from 'vitest'; +import { execFileSync, spawn, type ChildProcess } from 'node:child_process'; +import { once } from 'node:events'; +import fs from 'node:fs'; +import os from 'node:os'; +import path from 'node:path'; +import YAML from 'yaml'; +import type { DashboardEvent } from '../../types.js'; + +const cli = path.resolve('dist/index.js'); +let sandbox = ''; +let dashboard: ChildProcess | undefined; + +async function stopDashboard(): Promise { + if (dashboard && dashboard.exitCode === null) { + const exited = once(dashboard, 'exit'); + dashboard.kill('SIGTERM'); + await exited; + } + dashboard = undefined; +} + +afterEach(async () => { + await stopDashboard(); + if (sandbox) fs.rmSync(sandbox, { recursive: true, force: true }); +}); + +/** A git project P whose data home lives in the project, and the real CLI to drive it. */ +function fixture() { + sandbox = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'teamai-scope-reuse-'))); + const home = path.join(sandbox, 'home'); + const project = path.join(sandbox, 'project'); + const dataHome = path.join(project, '.teamai'); + const remote = path.join(sandbox, 'remote.git'); + const logDir = path.join(home, '.teamai', 'dashboard'); + const logPath = path.join(logDir, 'events.jsonl'); + fs.mkdirSync(logDir, { recursive: true }); + const env = { + ...process.env, HOME: home, USERPROFILE: home, XDG_CONFIG_HOME: path.join(home, '.config'), + GIT_CONFIG_GLOBAL: path.join(home, '.gitconfig'), GIT_CONFIG_NOSYSTEM: '1', + GIT_AUTHOR_NAME: 'Scope Test', GIT_AUTHOR_EMAIL: 'test@example.invalid', + GIT_COMMITTER_NAME: 'Scope Test', GIT_COMMITTER_EMAIL: 'test@example.invalid', NO_COLOR: '1', + }; + const git = (...args: string[]) => execFileSync('git', args, { cwd: project, env, encoding: 'utf8', stdio: 'pipe' }); + /** + * A project at `root` with its own team at `teamRemote`. A git project moves + * its data home to a partition on its first pull; without git (`git: false`) + * it stays in the workspace, `/.teamai`, under no partition. + */ + const makeProject = (root: string, teamRemote: string, { git: inGit = true } = {}) => { + const seed = inGit ? root : `${root}-team-seed`; + const at = (...args: string[]) => execFileSync('git', args, { cwd: seed, env, encoding: 'utf8', stdio: 'pipe' }); + fs.mkdirSync(path.join(root, '.claude'), { recursive: true }); + fs.mkdirSync(seed, { recursive: true }); + at('init', '-q', '-b', 'main'); + fs.writeFileSync(path.join(seed, 'teamai.yaml'), YAML.stringify({ team: 'scope-test', repo: teamRemote, provider: 'git' })); + at('add', 'teamai.yaml'); + at('commit', '-qm', 'fixture'); + at('clone', '-q', '--bare', seed, teamRemote); + const teamClone = path.join(root, '.teamai', 'team-repo'); + at('clone', '-q', teamRemote, teamClone); + fs.writeFileSync(path.join(root, '.teamai', 'config.yaml'), YAML.stringify({ + repo: { localPath: teamClone, remote: teamRemote, kind: 'git' }, username: 'tester', scope: 'project', projectRoot: root, + updatePolicy: 'skip', enabledAgents: ['claude'], additionalRoles: [], + })); + return { + pull: () => execFileSync(process.execPath, [cli, 'pull'], { cwd: root, env, encoding: 'utf8', stdio: 'pipe', timeout: 30_000 }), + stats: () => YAML.parse(at('--git-dir', teamRemote, 'show', 'teamai-reports:stats/tester.yaml')), + reportsHead: () => { + try { return at('--git-dir', teamRemote, 'rev-parse', '--verify', '-q', 'teamai-reports'); } catch { return ''; } + }, + /** A hook event the way the installed hooks send it. */ + hook: (hookName: string, data: Record) => execFileSync(process.execPath, + [cli, 'hook-dispatch', hookName, '--tool', 'claude', '--stdin'], + { cwd: root, env, input: JSON.stringify({ cwd: root, ...data }), encoding: 'utf8', stdio: 'pipe', timeout: 30_000 }), + }; + }; + const main = makeProject(project, remote); + const writeLog = (events: DashboardEvent[]) => fs.writeFileSync(logPath, events.map(e => JSON.stringify(e)).join('\n') + '\n'); + /** The shared snapshots an earlier release left, `prompts` reported per bare session ID. */ + const writeSharedSnapshots = (prompts: Record, date: string) => { + const entries = (value: (n: number) => unknown) => Object.fromEntries(Object.entries(prompts).map(([sid, n]) => [sid, value(n)])); + const snapshots = { + interventions: entries(() => ({ interrupt: 0, toolReject: 0, correction: 0 })), + 'prompt-tokens': entries((n) => ({ prompts: n, tokens: { input: 0, output: 0, cacheRead: 0, cacheCreation: 0 } })), + 'daily-sessions': entries((n) => ({ date, prompts: n, durationMs: 3_600_000, succeeded: 1, corrected: 0 })), + }; + for (const [name, value] of Object.entries(snapshots)) { + fs.writeFileSync(path.join(logDir, `reported-${name}.json`), JSON.stringify(value)); + } + }; + const { pull, stats, reportsHead, hook } = main; + /** Another project with its own team. */ + const addProject = (name: string, options: { git?: boolean } = {}) => { + const root = path.join(sandbox, name); + return { root, ...makeProject(root, path.join(sandbox, `${name}-remote.git`), options) }; + }; + return { home, project, dataHome, remote, logPath, env, git, writeLog, writeSharedSnapshots, pull, stats, reportsHead, hook, addProject }; +} + +it('real CLI preserves a path-keyed reuse and replays a delayed exit emitted by the real monitor', async () => { + const { project, dataHome, logPath, env, git, remote, writeLog, writeSharedSnapshots, pull, stats } = fixture(); + const now = Date.now(); + const timestamp = (offset: number) => new Date(now + offset).toISOString(); + const event = (type: DashboardEvent['type'], offset: number): DashboardEvent => ({ + type, timestamp: timestamp(offset), sessionId: 'pid-123', tool: 'copilot', + }); + const oldUser = [event('prompt_submit', -120_000), event('session_end', -119_000)]; + const firstRun = [ + { ...event('session_start', -60_000), dataHome, monitorPid: 99999999 }, + { ...event('prompt_submit', -59_000), dataHome }, + { ...event('stop', -58_000), dataHome }, + ]; + writeLog([...oldUser, ...firstRun]); + writeSharedSnapshots({ 'pid-123': 1 }, timestamp(0).slice(0, 10)); + pull(); + expect(stats()).toMatchObject({ prompts: 1, interventions: { sessions: 1 } }); + + // Capture an actual monitor event, then replay the race: SessionEnd and the + // next invocation arrive after the monitor read but before its exit append. + dashboard = spawn(process.execPath, [cli, 'dashboard', '--port', '0'], { cwd: project, env, stdio: 'pipe' }); + let output = ''; + dashboard.stdout?.on('data', data => { output += data.toString(); }); + dashboard.stderr?.on('data', data => { output += data.toString(); }); + let exit: DashboardEvent | undefined; + const deadline = Date.now() + 25_000; + while (!exit && Date.now() < deadline) { + if (dashboard.exitCode !== null) throw new Error(output); + const events: DashboardEvent[] = fs.readFileSync(logPath, 'utf8').trim().split('\n').map(line => JSON.parse(line)); + exit = events.find(e => e.type === 'process_exit'); + if (!exit) await new Promise(resolve => setTimeout(resolve, 100)); + } + await stopDashboard(); + expect(exit?.processExitAfter, output).toBe(firstRun[2].timestamp); + if (!exit) throw new Error('The dashboard monitor did not record an exit.'); + writeLog([ + ...oldUser, ...firstRun, + { ...event('session_end', -57_000), dataHome }, + { ...event('session_start', -56_000), dataHome }, + exit, + { ...event('prompt_submit', 0), timestamp: new Date(Date.parse(exit.timestamp) + 1000).toISOString(), dataHome }, + { ...event('stop', 0), timestamp: new Date(Date.parse(exit.timestamp) + 2000).toISOString(), dataHome }, + ]); + pull(); + expect(stats()).toMatchObject({ prompts: 2, interventions: { sessions: 2 } }); + const reportedHead = git('--git-dir', remote, 'rev-parse', 'teamai-reports'); + pull(); + expect(git('--git-dir', remote, 'rev-parse', 'teamai-reports')).toBe(reportedHead); + const shown = execFileSync(process.execPath, [cli, 'stats'], { cwd: project, env, encoding: 'utf8', timeout: 15_000 }); + expect(shown).toMatch(/Sessions:\s+2/); + expect(shown).toMatch(/Conversation turns:\s+2/); +}, 60_000); + +it('real CLI sends no run of an aggregated bare ID again and splits a reused ID a crash left open', () => { + const { project, dataHome, writeLog, writeSharedSnapshots, pull, stats, reportsHead } = fixture(); + const now = Date.now(); + let offset = -600_000; + const event = (type: DashboardEvent['type'], sessionId: string, fields: Partial = {}): DashboardEvent => + ({ type, timestamp: new Date(now + (offset += 1000)).toISOString(), sessionId, tool: 'claude', ...fields }); + const legacyId = `pid-321-${project}`; + const legacyRun = () => [ + event('session_start', legacyId, { cwd: project }), event('prompt_submit', legacyId, { cwd: project }), + event('stop', legacyId, { cwd: project }), event('session_end', legacyId, { cwd: project }), + ]; + // An earlier release recorded two runs of one fallback ID and reported them as one. + const legacy = [...legacyRun(), ...legacyRun()]; + writeLog(legacy); + writeSharedSnapshots({ [legacyId]: 2 }, new Date(now).toISOString().slice(0, 10)); + pull(); + expect(reportsHead()).toBe(''); + + // A run that crashed with no dashboard running: nothing ends it. + const crashed = [ + event('session_start', 'pid-9', { dataHome, monitorPid: 99999998 }), + event('prompt_submit', 'pid-9', { dataHome }), event('stop', 'pid-9', { dataHome }), + ]; + writeLog([...legacy, ...crashed]); + pull(); + expect(stats()).toMatchObject({ prompts: 1, interventions: { sessions: 1 } }); + + // The next invocation reuses the ID from another process. + const next = [ + event('session_start', 'pid-9', { dataHome, monitorPid: 99999999 }), + event('prompt_submit', 'pid-9', { dataHome }), event('stop', 'pid-9', { dataHome }), + ]; + writeLog([...legacy, ...crashed, ...next]); + pull(); + expect(stats()).toMatchObject({ prompts: 2, interventions: { sessions: 2 } }); +}, 60_000); + +it('real CLI gives a session resumed in another project after compaction to the project its transcript started in', () => { + const { home, logPath, addProject } = fixture(); + // W keeps its data home in the workspace; Q is a git project. + const w = addProject('project-w', { git: false }); + const q = addProject('project-q'); + const transcript = path.join(home, '.claude', 'projects', 'w', 'resumed-e2e.jsonl'); + fs.mkdirSync(path.dirname(transcript), { recursive: true }); + const turn = (sessionId: string, cwd: string, n: number) => JSON.stringify({ + type: 'user', sessionId, uuid: `${sessionId}-${n}`, cwd, timestamp: new Date().toISOString(), + message: { role: 'user', content: `turn ${n}` }, + }); + const run = (project: typeof w, sessionId: string, transcriptPath: string) => { + project.hook('prompt-submit', { session_id: sessionId, hook_event_name: 'UserPromptSubmit', prompt: 'turn', transcript_path: transcriptPath }); + project.hook('stop', { session_id: sessionId, hook_event_name: 'Stop', transcript_path: transcriptPath }); + }; + // W records and reports the session. + fs.writeFileSync(transcript, turn('resumed-e2e', w.root, 1) + '\n'); + run(w, 'resumed-e2e', transcript); + w.pull(); + expect(w.stats()).toMatchObject({ prompts: 1, interventions: { sessions: 1 } }); + expect(fs.existsSync(path.join(w.root, '.teamai', 'dashboard', 'reported-prompt-tokens.json'))).toBe(true); + // As if an earlier release had reported it: no owners index. Compaction then + // drops every W event, so nothing on disk outside W points to W. + fs.rmSync(path.join(home, '.teamai', 'dashboard', 'session-owners.jsonl'), { force: true }); + fs.writeFileSync(logPath, ''); + + // `claude --resume` in Q appends to the same transcript; Q also runs a session of its own. + fs.appendFileSync(transcript, turn('resumed-e2e', q.root, 2) + '\n'); + run(q, 'resumed-e2e', transcript); + const own = path.join(home, '.claude', 'projects', 'q', 'q-own.jsonl'); + fs.mkdirSync(path.dirname(own), { recursive: true }); + fs.writeFileSync(own, turn('q-own', q.root, 1) + '\n'); + run(q, 'q-own', own); + q.pull(); + w.pull(); + + // Q reports only its own session; W reports the resumed turn. + expect(q.stats()).toMatchObject({ prompts: 1, interventions: { sessions: 1 } }); + expect(w.stats()).toMatchObject({ prompts: 2, interventions: { sessions: 1 } }); +}, 90_000); diff --git a/src/__tests__/e2e/report-timeout.test.ts b/src/__tests__/e2e/report-timeout.test.ts index 95d7fc72b..4eed66517 100644 --- a/src/__tests__/e2e/report-timeout.test.ts +++ b/src/__tests__/e2e/report-timeout.test.ts @@ -5,6 +5,7 @@ import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; import YAML from 'yaml'; +import { dataHomeKey } from '../../dashboard-collector.js'; const root = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../../..'); const cli = path.join(root, 'dist/index.js'); @@ -42,15 +43,15 @@ function fixture(agent: keyof typeof agents, provider: string, team: Record JSON.stringify(e)).join('\n') + '\n'); } function receiver(mode: 'slow' | 'reject' | 'normal') { @@ -91,10 +92,12 @@ function fixture(agent: keyof typeof agents, provider: string, team: Record { @@ -109,11 +112,11 @@ describe('real CLI report completion', () => { it(`acknowledges slow pushes once: ${provider}/${agent}`, async () => { const f = fixture(agent, provider); await f.pull(); // Warm reports worktree without any session data. - f.seedEvents(); + await f.seedEvents(); f.receiver('slow'); const output = await f.pull(() => { expect(fs.readFileSync(f.usage, 'utf8')).toBe(f.usageLine); - expect(f.snapshot('prompt-tokens').s1).toBeUndefined(); + expect(f.snapshot('prompt-tokens')[f.runId]).toBeUndefined(); expect(fs.existsSync(path.join(f.home, '.teamai/.sync-lock'))).toBe(true); // An event arriving during the push must survive cleanup of the batch. fs.appendFileSync(f.usage, f.usageLine); @@ -122,7 +125,7 @@ describe('real CLI report completion', () => { expect(f.stats().skills.review.count).toBe(1); expect(fs.readFileSync(f.usage, 'utf8')).toBe(f.usageLine); for (const name of ['interventions', 'prompt-tokens', 'daily-sessions']) { - expect(f.snapshot(name).s1).toBeDefined(); + expect(f.snapshot(name)[f.runId]).toBeDefined(); } expect(fs.existsSync(path.join(f.home, '.teamai/.sync-lock'))).toBe(false); f.receiver('normal'); @@ -142,11 +145,11 @@ describe('real CLI report completion', () => { it('retains a rejected report and retries its committed tree without counting twice', async () => { const f = fixture('codex', 'git'); await f.pull(); - f.seedEvents(); + await f.seedEvents(); f.receiver('reject'); await f.pull(); expect(fs.readFileSync(f.usage, 'utf8')).toBe(f.usageLine); - expect(f.snapshot('prompt-tokens').s1).toBeUndefined(); + expect(f.snapshot('prompt-tokens')[f.runId]).toBeUndefined(); f.receiver('normal'); await f.pull(); const stats = f.stats(); diff --git a/src/__tests__/git-kind-reports.test.ts b/src/__tests__/git-kind-reports.test.ts index 5e29a8ad2..ac46a3d34 100644 --- a/src/__tests__/git-kind-reports.test.ts +++ b/src/__tests__/git-kind-reports.test.ts @@ -12,6 +12,7 @@ import { getDataHome, getReportsDir, REPORTS_WORKTREE_DIRNAME, type LocalConfig import { commitAndPushReports, ensureReportsWorktree, refreshReportsWorktree, updateReports } from '../utils/reports-branch.js'; import { pushRepoDirectly } from '../utils/git.js'; import { reportUsageToTeam } from '../team-push.js'; +import { dataHomeKey } from '../dashboard-collector.js'; import { listMembers } from '../members.js'; import { resolveProjectDataHome, saveLocalConfigForScope } from '../config.js'; import { buildHandlerRegistry } from '../hook-handlers.js'; @@ -106,12 +107,12 @@ describe('git-kind reports branch', () => { const ts = new Date().toISOString(); const eventsDir = path.join(process.env.HOME!, '.teamai', 'dashboard'); // A session the user scope recorded. - const dataHome = getDataHome(cfg); + const key = await dataHomeKey(getDataHome(cfg)); fs.mkdirSync(eventsDir, { recursive: true }); fs.writeFileSync( path.join(eventsDir, 'events.jsonl'), - `${JSON.stringify({ type: 'session_start', timestamp: ts, sessionId: 's1', tool: 'claude', cwd: '/p', dataHome })}\n` + - `${JSON.stringify({ type: 'stop', timestamp: ts, sessionId: 's1', tool: 'claude', dataHome, interventions: { interrupt: 1, toolReject: 0 } })}\n`, + `${JSON.stringify({ type: 'session_start', timestamp: ts, sessionId: 's1', tool: 'claude', cwd: '/p', dataHomeKey: key })}\n` + + `${JSON.stringify({ type: 'stop', timestamp: ts, sessionId: 's1', tool: 'claude', dataHomeKey: key, interventions: { interrupt: 1, toolReject: 0 } })}\n`, ); await reportUsageToTeam(clone, 'alice', { skipTruncate: true, selfConfig: cfg }); diff --git a/src/__tests__/hook-handlers.test.ts b/src/__tests__/hook-handlers.test.ts index 08afd3ee1..68648db1e 100644 --- a/src/__tests__/hook-handlers.test.ts +++ b/src/__tests__/hook-handlers.test.ts @@ -38,6 +38,7 @@ vi.mock('../dashboard-collector.js', () => ({ appendEvent: mockAppendEvent, compactEvents: vi.fn().mockResolvedValue(undefined), dashboardReport: mockDashboardReport, + dataHomeKey: vi.fn().mockResolvedValue('key'), eventProjectAnchor: vi.fn().mockResolvedValue(undefined), })); diff --git a/src/__tests__/scope-filter.test.ts b/src/__tests__/scope-filter.test.ts index 0a039fb58..f632de5de 100644 --- a/src/__tests__/scope-filter.test.ts +++ b/src/__tests__/scope-filter.test.ts @@ -2,12 +2,19 @@ import { describe, it, expect } from 'vitest'; import fs from 'node:fs'; import os from 'node:os'; import path from 'node:path'; -import { filterEventsByScope } from '../team-push.js'; +import { adoptBareKeys } from '../team-push.js'; +import { filterEventsByScope } from '../dashboard-scope.js'; +import { dataHomeKey } from '../dashboard-collector.js'; import type { DashboardEvent, LocalConfig } from '../types.js'; -/** An event; without `dataHome` it is one written before events carried one, attributed by cwd. */ -function makeEvent(cwd: string | undefined, sessionId = 's1', dataHome?: string): DashboardEvent { - return { type: 'prompt_submit', timestamp: new Date().toISOString(), sessionId, tool: 'claude', cwd, dataHome }; +/** An event written before events carried a data home key, attributed by cwd. */ +function makeEvent(cwd: string | undefined, sessionId = 's1'): DashboardEvent { + return { type: 'prompt_submit', timestamp: new Date().toISOString(), sessionId, tool: 'claude', cwd }; +} + +/** An event recorded by the scope whose data home is `dataHome`. */ +async function scopedEvent(cwd: string | undefined, sessionId: string, dataHome: string): Promise { + return { ...makeEvent(cwd, sessionId), dataHomeKey: await dataHomeKey(dataHome) }; } const repo = { localPath: '/unused/team-repo', remote: 'https://example.test/team.git' }; @@ -20,8 +27,10 @@ function userScope(dataHome = '/home/jeff/.teamai'): LocalConfig { return { repo, username: 'jeff', scope: 'user', additionalRoles: [], dataHome }; } +/** The session of each event a scope keeps: its run ID without a fallback run's start. */ async function ids(events: DashboardEvent[], config?: LocalConfig): Promise { - return (await filterEventsByScope(events, config)).map((e) => e.sessionId); + return (await filterEventsByScope(events, config)).map((e) => + e.sessionId.startsWith('pid-') ? e.sessionId.slice(0, e.sessionId.lastIndexOf('@')) : e.sessionId); } describe('filterEventsByScope', () => { @@ -37,31 +46,252 @@ describe('filterEventsByScope', () => { expect(await filterEventsByScope(events)).toEqual(events); }); - describe('events that carry a data home', () => { - const scoped: DashboardEvent[] = [ - makeEvent('/Users/jeff/project-a', 'p1', '/home/jeff/.teamai/projects/p'), - makeEvent(undefined, 'p2', '/home/jeff/.teamai/projects/p'), - makeEvent('/Users/jeff/project-a', 'u1', '/home/jeff/.teamai'), - makeEvent('/Users/jeff/project-a', 'q1', '/home/jeff/.teamai/projects/q'), - ]; + describe('events that carry a data home key', () => { + const scoped = (): Promise => Promise.all([ + scopedEvent('/Users/jeff/project-a', 'p1', '/home/jeff/.teamai/projects/p'), + scopedEvent(undefined, 'p2', '/home/jeff/.teamai/projects/p'), + scopedEvent('/Users/jeff/project-a', 'u1', '/home/jeff/.teamai'), + scopedEvent('/Users/jeff/project-a', 'q1', '/home/jeff/.teamai/projects/q'), + ]); it('a project keeps its own, whatever their cwd', async () => { - expect(await ids(scoped, projectScope('/Users/jeff/project-a'))).toEqual(['p1', 'p2']); + expect(await ids(await scoped(), projectScope('/Users/jeff/project-a'))).toEqual(['p1', 'p2']); }); it('the user scope keeps its own and no project\'s', async () => { - expect(await ids(scoped, userScope())).toEqual(['u1']); + expect(await ids(await scoped(), userScope())).toEqual(['u1']); + }); + + it('a session goes whole to the scope of its first keyed event', async () => { + // A Stop carries the whole transcript's totals, so P must not count them again. + const moved = [ + makeEvent('/Users/jeff/other-work', 'm1'), + await scopedEvent('/Users/jeff/other-work', 'm1', '/home/jeff/.teamai'), + { ...(await scopedEvent('/Users/jeff/project-a', 'm1', '/home/jeff/.teamai/projects/p')), type: 'stop' as const, prompts: 5 }, + ]; + expect(await ids(moved, userScope())).toEqual(['m1', 'm1', 'm1']); + expect(await ids(moved, projectScope('/Users/jeff/project-a'))).toEqual([]); + }); + + it('a session ID reused after its session ended is decided anew', async () => { + // A PID-fallback ID: the user-scope run ended, a later run in P reuses the ID. + for (const end of ['session_end', 'process_exit'] as const) { + const reused = [ + await scopedEvent(undefined, 'pid-1', '/home/jeff/.teamai'), + { ...(await scopedEvent(undefined, 'pid-1', '/home/jeff/.teamai')), type: end }, + await scopedEvent(undefined, 'pid-1', '/home/jeff/.teamai/projects/p'), + ]; + const project = await filterEventsByScope(reused, projectScope('/Users/jeff/project-a')); + const user = await filterEventsByScope(reused, userScope()); + // Each run is its own session, under the ID plus its first event's timestamp. + const first = `pid-1@${reused[0].timestamp}`; + expect(project).toEqual([{ ...reused[2], sessionId: `pid-1@${reused[2].timestamp}` }]); + expect(user).toEqual([{ ...reused[0], sessionId: first }, { ...reused[1], sessionId: first }]); + } + }); + + it('a second end of a run, with nothing recorded between the two, joins the run that just closed', async () => { + // The dashboard monitor appends process_exit after the SessionEnd hook's session_end. + const p = '/home/jeff/.teamai/projects/p'; + const at = (e: DashboardEvent, timestamp: string): DashboardEvent => ({ ...e, timestamp }); + const log = [ + at(await scopedEvent(undefined, 'pid-1', p), '2026-01-01T00:00:00.000Z'), + at({ ...(await scopedEvent(undefined, 'pid-1', p)), type: 'session_end' }, '2026-01-01T00:01:00.000Z'), + at({ ...(await scopedEvent(undefined, 'pid-1', p)), type: 'process_exit' }, '2026-01-01T00:01:05.000Z'), + at(await scopedEvent(undefined, 'pid-1', p), '2026-01-01T00:02:00.000Z'), + at({ ...(await scopedEvent(undefined, 'pid-1', p)), type: 'process_exit' }, '2026-01-01T00:03:00.000Z'), + ]; + expect((await filterEventsByScope(log, projectScope('/Users/jeff/project-a'))).map((e) => e.sessionId)).toEqual([ + 'pid-1@2026-01-01T00:00:00.000Z', 'pid-1@2026-01-01T00:00:00.000Z', 'pid-1@2026-01-01T00:00:00.000Z', + 'pid-1@2026-01-01T00:02:00.000Z', 'pid-1@2026-01-01T00:02:00.000Z', + ]); + }); + + it('a run keeps its ID after compaction drops the earlier runs of its ID', async () => { + const ended = [ + await scopedEvent(undefined, 'pid-1', '/home/jeff/.teamai/projects/p'), + { ...(await scopedEvent(undefined, 'pid-1', '/home/jeff/.teamai/projects/p')), type: 'session_end' as const }, + ]; + const later = { ...(await scopedEvent(undefined, 'pid-1', '/home/jeff/.teamai/projects/p')), timestamp: '2099-01-01T00:00:00.000Z' }; + const before = await filterEventsByScope([...ended, later], projectScope('/Users/jeff/project-a')); + const after = await filterEventsByScope([later], projectScope('/Users/jeff/project-a')); + expect(after.map((e) => e.sessionId)).toEqual(['pid-1@2099-01-01T00:00:00.000Z']); + expect(before[2].sessionId).toBe(after[0].sessionId); + }); + + it('an exit a dashboard before processExitAfter wrote just after the next run began ends the earlier run', async () => { + // That monitor read the log before SessionEnd and the next SessionStart, + // then appended its exit, unannotated, after them. + const p = '/home/jeff/.teamai/projects/p'; + const at = async (type: DashboardEvent['type'], timestamp: string) => + ({ ...(await scopedEvent(undefined, 'pid-1', p)), type, timestamp }); + const log = [ + await at('session_start', '2026-01-01T00:00:00.000Z'), + await at('session_end', '2026-01-01T00:01:00.000Z'), + await at('session_start', '2026-01-01T00:01:01.000Z'), + await at('process_exit', '2026-01-01T00:01:02.000Z'), + await at('prompt_submit', '2026-01-01T00:01:03.000Z'), + ]; + const project = await filterEventsByScope(log, projectScope('/Users/jeff/project-a')); + expect(project.map((e) => e.sessionId)).toEqual([ + 'pid-1@2026-01-01T00:00:00.000Z', 'pid-1@2026-01-01T00:00:00.000Z', + 'pid-1@2026-01-01T00:01:01.000Z', 'pid-1@2026-01-01T00:00:00.000Z', 'pid-1@2026-01-01T00:01:01.000Z', + ]); + }); + + it('an unannotated exit followed by more of the open run did not end it, however late it came', async () => { + // A delayed callback or skewed clock: a dead process records nothing more, + // so the prompt after the exit shows the open run went on. + const p = '/home/jeff/.teamai/projects/p'; + const at = async (type: DashboardEvent['type'], timestamp: string) => + ({ ...(await scopedEvent(undefined, 'pid-1', p)), type, timestamp }); + const log = [ + await at('session_start', '2026-01-01T00:00:00.000Z'), + await at('session_end', '2026-01-01T00:01:00.000Z'), + await at('session_start', '2026-01-01T00:02:00.000Z'), + await at('process_exit', '2026-01-01T00:03:00.000Z'), + await at('prompt_submit', '2026-01-01T00:04:00.000Z'), + ]; + const project = await filterEventsByScope(log, projectScope('/Users/jeff/project-a')); + expect(project.map((e) => e.sessionId)).toEqual([ + 'pid-1@2026-01-01T00:00:00.000Z', 'pid-1@2026-01-01T00:00:00.000Z', + 'pid-1@2026-01-01T00:02:00.000Z', 'pid-1@2026-01-01T00:00:00.000Z', 'pid-1@2026-01-01T00:02:00.000Z', + ]); + }); + + it('an unannotated exit of a run a new process superseded does not end the new run', async () => { + // The first run crashed with nothing ending it; a dashboard before + // processExitAfter appends its exit after the next invocation started. + const p = '/home/jeff/.teamai/projects/p'; + const at = async (type: DashboardEvent['type'], timestamp: string, fields: Partial = {}) => + ({ ...(await scopedEvent(undefined, 'pid-1', p)), type, timestamp, ...fields }); + const log = [ + await at('session_start', '2026-01-01T00:00:00.000Z', { monitorPid: 100 }), + await at('prompt_submit', '2026-01-01T00:00:30.000Z'), + await at('session_start', '2026-01-01T00:02:00.000Z', { monitorPid: 200 }), + await at('process_exit', '2026-01-01T00:02:05.000Z'), + await at('prompt_submit', '2026-01-01T00:03:00.000Z'), + ]; + const project = await filterEventsByScope(log, projectScope('/Users/jeff/project-a')); + expect(project.map((e) => e.sessionId)).toEqual([ + 'pid-1@2026-01-01T00:00:00.000Z', 'pid-1@2026-01-01T00:00:00.000Z', + 'pid-1@2026-01-01T00:02:00.000Z', 'pid-1@2026-01-01T00:00:00.000Z', 'pid-1@2026-01-01T00:02:00.000Z', + ]); + }); + + it('an unannotated exit with nothing more of the open run ends it', async () => { + const p = '/home/jeff/.teamai/projects/p'; + const at = async (type: DashboardEvent['type'], timestamp: string) => + ({ ...(await scopedEvent(undefined, 'pid-1', p)), type, timestamp }); + const log = [ + await at('session_start', '2026-01-01T00:00:00.000Z'), + await at('session_end', '2026-01-01T00:01:00.000Z'), + await at('session_start', '2026-01-01T00:02:00.000Z'), + await at('process_exit', '2026-01-01T00:03:00.000Z'), + await at('session_start', '2026-01-01T00:04:00.000Z'), + ]; + const project = await filterEventsByScope(log, projectScope('/Users/jeff/project-a')); + expect(project.map((e) => e.sessionId)).toEqual([ + 'pid-1@2026-01-01T00:00:00.000Z', 'pid-1@2026-01-01T00:00:00.000Z', + 'pid-1@2026-01-01T00:02:00.000Z', 'pid-1@2026-01-01T00:02:00.000Z', 'pid-1@2026-01-01T00:04:00.000Z', + ]); + }); + + it('a delayed observed exit targets the earlier run across scopes and is ignored after that run is compacted', async () => { + const old = { ...(await scopedEvent(undefined, 'pid-1', '/home/jeff/.teamai')), timestamp: '2026-01-01T00:00:00Z' }; + const ended = { ...old, type: 'session_end' as const, timestamp: '2026-01-01T00:01:00Z' }; + const next = { ...(await scopedEvent(undefined, 'pid-1', '/home/jeff/.teamai/projects/p')), timestamp: '2026-01-01T00:02:00Z' }; + const delayed = { ...old, type: 'process_exit' as const, timestamp: '2026-01-01T00:03:00Z', processExitAfter: old.timestamp }; + const prompt = { ...next, timestamp: '2026-01-01T00:04:00Z' }; + const exit = { ...next, type: 'process_exit' as const, timestamp: '2026-01-01T00:05:00Z', processExitAfter: prompt.timestamp }; + const later = { ...next, timestamp: '2026-01-01T00:06:00Z' }; + for (const history of [[old, ended], []]) { + const log = [...history, next, delayed, prompt, exit, later]; + const project = await filterEventsByScope(log, projectScope('/Users/jeff/project-a')); + expect(project.map(e => e.sessionId)).toEqual([ + `pid-1@${next.timestamp}`, `pid-1@${next.timestamp}`, `pid-1@${next.timestamp}`, `pid-1@${later.timestamp}`, + ]); + const user = await filterEventsByScope(log, userScope()); + expect(user).toHaveLength(history.length ? 3 : 0); + } + }); + + it('a tool\'s own session ID stays one run across an end and a resume', async () => { + // `claude --resume` after the dashboard recorded the exit: its Stop carries + // the whole transcript, so a second run would count it again. + for (const end of ['session_end', 'process_exit'] as const) { + const log = [ + await scopedEvent(undefined, 'claude-uuid', '/home/jeff/.teamai'), + { ...(await scopedEvent(undefined, 'claude-uuid', '/home/jeff/.teamai')), type: end }, + { ...(await scopedEvent(undefined, 'claude-uuid', '/home/jeff/.teamai/projects/p')), type: 'session_start' as const }, + await scopedEvent(undefined, 'claude-uuid', '/home/jeff/.teamai/projects/p'), + ]; + const user = await filterEventsByScope(log, userScope()); + // Keyed by the ID alone, so it stays the same session after compaction drops its events. + expect(user.map((e) => e.sessionId)).toEqual(Array(4).fill('claude-uuid')); + expect(await ids(log, projectScope('/Users/jeff/project-a'))).toEqual([]); + } + }); + + it('a fallback ID started by another process is a new run, though the last one never ended', async () => { + // The user-scope run crashed with no dashboard running, so nothing ended it. + const start = (sessionId: string, dataHome: string, monitorPid: number) => scopedEvent(undefined, sessionId, dataHome) + .then((e) => ({ ...e, type: 'session_start' as const, monitorPid })); + for (const id of ['pid-1', 'pid-1-/Users/jeff/project-a']) { + const log = [ + await start(id, '/home/jeff/.teamai', 100), + await scopedEvent(undefined, id, '/home/jeff/.teamai'), + await start(id, '/home/jeff/.teamai/projects/p', 200), + await scopedEvent(undefined, id, '/home/jeff/.teamai/projects/p'), + ]; + expect(await ids(log, projectScope('/Users/jeff/project-a'))).toEqual([id, id]); + expect(await ids(log, userScope())).toEqual([id, id]); + } + }); + + it('a start from the same process, or under a tool\'s own session ID, stays in the open run', async () => { + // Claude fires SessionStart again on compact and on resume, a new process for + // the same session; its Stop carries the whole transcript, so it must stay one run. + const cases: Array<[string, number]> = [['pid-1', 100], ['claude-uuid', 200]]; + for (const [id, pid] of cases) { + const log = [ + { ...(await scopedEvent(undefined, id, '/home/jeff/.teamai')), type: 'session_start' as const, monitorPid: 100 }, + { ...(await scopedEvent(undefined, id, '/home/jeff/.teamai/projects/p')), type: 'session_start' as const, monitorPid: pid }, + await scopedEvent(undefined, id, '/home/jeff/.teamai/projects/p'), + ]; + expect(await ids(log, userScope())).toEqual([id, id, id]); + expect(await ids(log, projectScope('/Users/jeff/project-a'))).toEqual([]); + } + }); + + it('an event that records its data home as a path, before keys were hashed, is keyed by it', async () => { + const unhashed: DashboardEvent[] = [ + { ...makeEvent(undefined, 'c1'), tool: 'copilot', dataHome: '/home/jeff/.teamai/projects/p' }, + { ...makeEvent('/Users/jeff/project-a', 'u1'), dataHome: '/home/jeff/.teamai' }, + ]; + expect(await ids(unhashed, projectScope('/Users/jeff/project-a'))).toEqual(['c1']); + expect(await ids(unhashed, userScope())).toEqual(['u1']); + }); + + it('a keyed event decides a session over an earlier unkeyed one', async () => { + // Recorded across the upgrade: the unkeyed event's cwd is P's, the key the user scope's. + const upgraded = [ + makeEvent('/Users/jeff/project-a', 'x1'), + await scopedEvent('/Users/jeff/project-a', 'x1', '/home/jeff/.teamai'), + ]; + expect(await ids(upgraded, projectScope('/Users/jeff/project-a'))).toEqual([]); + expect(await ids(upgraded, userScope())).toEqual(['x1', 'x1']); }); it('a project keeps what it recorded under its in-repo .teamai before moving to a partition', async () => { - const legacy = [makeEvent(undefined, 'l1', '/Users/jeff/project-a/.teamai')]; + const legacy = [await scopedEvent(undefined, 'l1', '/Users/jeff/project-a/.teamai')]; expect(await ids(legacy, projectScope('/Users/jeff/project-a'))).toEqual(['l1']); expect(await ids(legacy, userScope())).toEqual([]); }); it('a project rooted at HOME does not take the user scope\'s events', async () => { const home = os.homedir(); - const evts = [makeEvent(undefined, 'u1', path.join(home, '.teamai'))]; + const evts = [await scopedEvent(undefined, 'u1', path.join(home, '.teamai'))]; expect(await ids(evts, projectScope(home))).toEqual([]); }); @@ -70,7 +300,7 @@ describe('filterEventsByScope', () => { try { fs.mkdirSync(path.join(tmp, 'real', '.teamai'), { recursive: true }); fs.symlinkSync(path.join(tmp, 'real'), path.join(tmp, 'link'), 'dir'); - const evts = [makeEvent(undefined, 'k1', path.join(tmp, 'link', '.teamai'))]; + const evts = [await scopedEvent(undefined, 'k1', path.join(tmp, 'link', '.teamai'))]; const config = projectScope(path.join(tmp, 'real'), path.join(tmp, 'real', '.teamai')); expect(await ids(evts, config)).toEqual(['k1']); } finally { @@ -78,86 +308,110 @@ describe('filterEventsByScope', () => { } }); + it('keeps what a project recorded in its in-repo .teamai after migration removed it', async () => { + const tmp = fs.realpathSync(fs.mkdtempSync(path.join(os.tmpdir(), 'teamai-scope-gone-'))); + try { + fs.mkdirSync(path.join(tmp, 'real', '.teamai'), { recursive: true }); + fs.symlinkSync(path.join(tmp, 'real'), path.join(tmp, 'link'), 'dir'); + const evts = [await scopedEvent(undefined, 'g1', path.join(tmp, 'link', '.teamai'))]; + fs.rmSync(path.join(tmp, 'real', '.teamai'), { recursive: true }); + expect(await ids(evts, projectScope(path.join(tmp, 'link')))).toEqual(['g1']); + } finally { + fs.rmSync(tmp, { recursive: true, force: true }); + } + }); + it('a Windows data home matches whatever its case or separators', async () => { - const win = [makeEvent(undefined, 'w1', 'C:\\Users\\Jeff\\.teamai'), makeEvent(undefined, 'w2', 'C:\\Users\\jeff\\.teamai\\projects\\p')]; + const win = await Promise.all([scopedEvent(undefined, 'w1', 'C:\\Users\\Jeff\\.teamai'), scopedEvent(undefined, 'w2', 'C:\\Users\\jeff\\.teamai\\projects\\p')]); expect(await ids(win, userScope('c:/users/jeff/.teamai'))).toEqual(['w1']); }); }); - it('the user scope never keeps events without a data home', async () => { - expect(await ids(events, userScope())).toEqual([]); + it('an event whose key is not a string counts as written before keys existed', async () => { + // A hand-edited or corrupted line in the shared log: a later keyed event decides the session. + const corrupt: DashboardEvent[] = JSON.parse(JSON.stringify([ + { ...makeEvent('/Users/jeff/project-a', 'c1'), dataHomeKey: null }, + { ...makeEvent('/Users/jeff/project-a', 'c2'), dataHomeKey: 42 }, + ])); + const decided = [ + corrupt[0], await scopedEvent(undefined, 'c1', '/home/jeff/.teamai/projects/p'), + corrupt[1], await scopedEvent(undefined, 'c2', '/home/jeff/.teamai/projects/p'), + ]; + expect(await ids(decided, projectScope('/Users/jeff/project-a'))).toEqual(['c1', 'c1', 'c2', 'c2']); + expect(await ids(decided, userScope())).toEqual([]); }); +}); - // Events without a data home go to the project whose root holds their cwd. - it('filters to projectRoot (exact match and subdirectories)', async () => { - expect(await ids(events, projectScope('/Users/jeff/project-a'))).toEqual(['s1', 's2']); - }); +describe('adoptBareKeys', () => { + /** A run's event as filterEventsByScope returns it; an earlier release's unless `fields` key it. */ + const run = (sessionId: string, fields: Partial = {}): DashboardEvent => + ({ type: 'prompt_submit', timestamp: 't', sessionId, tool: 'copilot', ...fields }); + /** A counter's share: up to the run's own, while any of the sum is left. */ + const take = (own: number, left: number) => (left > 0 ? { taken: Math.min(own, left), left: left - Math.min(own, left) } : undefined); - it('projectRoot with trailing slash works the same', async () => { - expect(await ids(events, projectScope('/Users/jeff/project-a/'))).toEqual(['s1', 's2']); + it('consumes a bare entry, the sum of its runs, across them in order', () => { + // An earlier release summed every run of the ID under it: each run takes up + // to its own totals of what is left, so none is sent again. + const events = [run('pid-1@t1'), run('pid-1@t2'), run('s2@t0'), run('s3@t3')]; + const reported = { 'pid-1': 6, 's2@t0': 1 }; + expect(adoptBareKeys(reported, events, { 'pid-1@t1': 5, 'pid-1@t2': 3, 's3@t3': 4 }, take)) + .toEqual({ 'pid-1@t1': 5, 'pid-1@t2': 1, 's2@t0': 1 }); }); - it('events with undefined cwd are excluded by projectRoot', async () => { - expect(await ids(events, projectScope('/Users/jeff/project-a'))).not.toContain('s5'); + it('a later run the sum does not reach takes nothing: it was not reported', () => { + // Run 1 reported as 5; run 2 came after the last report of that release. + const adopted = adoptBareKeys({ 'pid-1': 5 }, [run('pid-1@t1'), run('pid-1@t2')], { 'pid-1@t1': 5, 'pid-1@t2': 3 }, take); + expect(adopted).toEqual({ 'pid-1@t1': 5 }); }); - it('does not match partial directory name prefixes', async () => { - const evts = [ - makeEvent('/Users/jeff/project-ab', 'x1'), - makeEvent('/Users/jeff/project-a', 'x2'), - ]; - expect(await ids(evts, projectScope('/Users/jeff/project-a'))).toEqual(['x2']); + it('the first run takes the entry even when nothing of it is left to share', () => { + // A bare entry means that release reported the ID, so its first run is not new. + expect(adoptBareKeys({ 'pid-1': 0 }, [run('pid-1@t1'), run('pid-1@t2')], { 'pid-1@t1': 2, 'pid-1@t2': 1 }, take)) + .toEqual({ 'pid-1@t1': 0 }); }); - // Windows paths are plain strings here, so these run on the ubuntu CI too. - // Both sides of the comparison are native paths in production: projectRoot is - // path.resolve(cwd) from init, and cwd is whatever the tool's hook payload - // carried. - describe('Windows paths', () => { - const winEvents: DashboardEvent[] = [ - makeEvent('C:\\Users\\jeff\\project-a', 'w1'), - makeEvent('C:\\Users\\jeff\\project-a\\src', 'w2'), - makeEvent('C:\\Users\\jeff\\project-ab', 'w3'), - makeEvent('C:\\Users\\jeff\\other-work', 'w4'), - ]; + it('never overrides a run\'s own entry, which still takes its share', () => { + expect(adoptBareKeys({ 'pid-1': 3, 'pid-1@t1': 5 }, [run('pid-1@t1')], { 'pid-1@t1': 5 }, take)['pid-1@t1']).toBe(5); + expect(adoptBareKeys({ 'pid-1': 3, 'pid-1@t1': 5 }, [run('pid-1@t1'), run('pid-1@t2')], { 'pid-1@t1': 2, 'pid-1@t2': 4 }, take)) + .toEqual({ 'pid-1@t1': 5, 'pid-1@t2': 1 }); + }); - it('filters to projectRoot including subdirectories', async () => { - expect(await ids(winEvents, projectScope('C:\\Users\\jeff\\project-a'))).toEqual(['w1', 'w2']); - }); + it('retires the bare entry its runs read, so it is never read again', () => { + expect(adoptBareKeys({ 'pid-1': 3, s2: 1 }, [run('pid-1@t1'), run('pid-1@t2')], { 'pid-1@t1': 1, 'pid-1@t2': 5 }, take)) + .toEqual({ 'pid-1@t1': 1, 'pid-1@t2': 2, s2: 1 }); + }); - it('matches a root and a cwd that disagree on separator style', async () => { - expect(await ids(winEvents, projectScope('C:/Users/jeff/project-a'))).toEqual(['w1', 'w2']); - }); + it('leaves a tool\'s own session ID alone: its run is keyed by the ID itself', () => { + const reported = { 'claude-uuid': 3 }; + expect(adoptBareKeys(reported, [run('claude-uuid')], { 'claude-uuid': 4 }, take)).toBe(reported); + }); - it('trailing backslash on the root works the same', async () => { - expect(await ids(winEvents, projectScope('C:\\Users\\jeff\\project-a\\'))).toEqual(['w1', 'w2']); - }); + it('returns the snapshot itself when it holds no bare entry to retire', () => { + const reported = { 'pid-1@t1': 5, s2: 1 }; + expect(adoptBareKeys(reported, [run('pid-1@t1'), run('s3@t3')], { 'pid-1@t1': 5 }, take)).toBe(reported); + }); - it('ignores drive-letter and directory casing', async () => { - expect(await ids(winEvents, projectScope('c:\\users\\JEFF\\Project-A'))).toEqual(['w1', 'w2']); - }); + it('gives no bare entry to a run this build recorded: only an earlier release wrote them', () => { + // Another scope's run, maybe: the bare entry says nothing about which scope reported it. + const reported = { 'pid-1': 3 }; + expect(adoptBareKeys(reported, [run('pid-1@t1', { dataHomeKey: 'k' }), run('pid-1@t2')], { 'pid-1@t1': 1, 'pid-1@t2': 1 }, take)) + .toBe(reported); + }); - it('matches a UNC root whatever its case or separators', async () => { - const uncEvents: DashboardEvent[] = [ - makeEvent('\\\\Server\\Share\\Proj', 'u1'), - makeEvent('\\\\server\\share\\proj\\src', 'u2'), - makeEvent('\\\\server\\share\\other', 'u3'), - ]; - expect(await ids(uncEvents, projectScope('\\\\SERVER\\SHARE\\proj'))).toEqual(['u1', 'u2']); - }); + it('a run recorded by main since #795, which wrote the data home path, is an earlier release\'s', () => { + expect(adoptBareKeys({ 'pid-1': 3 }, [run('pid-1@t1', { dataHome: '/home/jeff/.teamai' })], { 'pid-1@t1': 3 }, take)) + .toEqual({ 'pid-1@t1': 3 }); }); - // A POSIX path is case-sensitive, and `\` is a legal character in a POSIX - // filename, so neither folding may be applied to one. - describe('POSIX paths keep their own rules', () => { - it('does not fold case', async () => { - expect(await ids(events, projectScope('/users/jeff/PROJECT-A'))).toEqual([]); - }); + it('does not give a path-keyed run a shared bare entry, but still adopts its own scope entry', () => { + const reported = { 'pid-1': 3 }; + const events = [run('pid-1@t1', { dataHome: '/home/jeff/.teamai/projects/p' })]; + expect(adoptBareKeys(reported, events, { 'pid-1@t1': 3 }, take, 'shared')).toBe(reported); + expect(adoptBareKeys(reported, events, { 'pid-1@t1': 3 }, take, 'scope')).toEqual({ 'pid-1@t1': 3 }); + }); - it('treats a backslash in a filename as part of the name', async () => { - const evts = [makeEvent('/work/a\\b', 'p1'), makeEvent('/work/a/b', 'p2')]; - expect(await ids(evts, projectScope('/work/a/b'))).toEqual(['p2']); - expect(await ids(evts, projectScope('/work/a\\b'))).toEqual(['p1']); - }); + it('a run in progress across the upgrade is an earlier release\'s, by its first event', () => { + expect(adoptBareKeys({ 'pid-1': 3 }, [run('pid-1@t1'), run('pid-1@t1', { dataHomeKey: 'k' })], { 'pid-1@t1': 4 }, take)) + .toEqual({ 'pid-1@t1': 3 }); }); }); diff --git a/src/__tests__/self-mode-no-business-reset.test.ts b/src/__tests__/self-mode-no-business-reset.test.ts index d080649ee..19f0f65ea 100644 --- a/src/__tests__/self-mode-no-business-reset.test.ts +++ b/src/__tests__/self-mode-no-business-reset.test.ts @@ -10,6 +10,7 @@ import os from 'node:os'; import { simpleGit } from 'simple-git'; import { reportUsageToTeam } from '../team-push.js'; +import { dataHomeKey } from '../dashboard-collector.js'; import type { LocalConfig } from '../types.js'; let tmp: string; @@ -96,12 +97,12 @@ describe('E2E self-mode: business repo working tree is never reset', () => { const ts = new Date().toISOString(); const eventsDir = path.join(process.env.HOME!, '.teamai', 'dashboard'); // A session the user scope recorded (#785). - const dataHome = path.join(tmp, 'home', '.teamai'); + const key = await dataHomeKey(path.join(tmp, 'home', '.teamai')); fs.mkdirSync(eventsDir, { recursive: true }); fs.writeFileSync( path.join(eventsDir, 'events.jsonl'), - `${JSON.stringify({ type: 'session_start', timestamp: ts, sessionId: 's1', tool: 'claude', cwd: '/p', dataHome })}\n` + - `${JSON.stringify({ type: 'stop', timestamp: ts, sessionId: 's1', tool: 'claude', dataHome, interventions: { interrupt: 1, toolReject: 0 } })}\n`, + `${JSON.stringify({ type: 'session_start', timestamp: ts, sessionId: 's1', tool: 'claude', cwd: '/p', dataHomeKey: key })}\n` + + `${JSON.stringify({ type: 'stop', timestamp: ts, sessionId: 's1', tool: 'claude', dataHomeKey: key, interventions: { interrupt: 1, toolReject: 0 } })}\n`, ); const cfg: LocalConfig = { diff --git a/src/__tests__/session-trends.test.ts b/src/__tests__/session-trends.test.ts index e760b378a..4277b8b46 100644 --- a/src/__tests__/session-trends.test.ts +++ b/src/__tests__/session-trends.test.ts @@ -5,6 +5,8 @@ import { computeDailyStatsDelta, mergeDailyStats, summarizeTrendWindow, + takeDailySession, + type DailySessionSnapshot, } from '../session-trends.js'; import type { DashboardEvent, DailyUserStats } from '../types.js'; @@ -125,6 +127,24 @@ describe('daily session trends', () => { expect(delta['2026-09-02']).toMatchObject({ cacheReadTokens: 0, cacheEligibleInputTokens: 0 }); }); + it('a run taking its share of an earlier release\'s sum reports no status change of its own', () => { + // Two reported runs of one ID, one successful and one interrupted: the sum is + // unsuccessful, but each run keeps its own status, so neither moves the totals. + const day = '2026-09-01'; + const snapshot = (prompts: number, succeeded: 0 | 1, corrected: 0 | 1): DailySessionSnapshot => + ({ date: day, prompts, durationMs: 1000, succeeded, corrected, requestDaily: {} }); + const runs = new Map([['pid-1@t1', snapshot(1, 1, 0)], ['pid-1@t2', snapshot(1, 0, 1)]]); + let left = snapshot(2, 0, 1); + const reported: Record = {}; + for (const [runId, run] of runs) { + const share = takeDailySession(run, left); + reported[runId] = share.taken; + left = share.left; + } + const { delta } = computeDailyStatsDelta(runs, reported); + expect(delta[day]).toMatchObject({ sessionsEnded: 0, sessionsSucceeded: 0, sessionsCorrected: 0, promptTurns: 0 }); + }); + it('compares the latest seven UTC days with the prior seven days', () => { const daily: Record = { '2026-08-27': { sessionsEnded: 10, sessionsSucceeded: 5, promptTurns: 80, durationMs: 600_000, sessionsCorrected: 4, pricedRequests: 10, costMicros: 1_000_000, cacheReadTokens: 20, cacheEligibleInputTokens: 100 }, diff --git a/src/__tests__/stats-scope.test.ts b/src/__tests__/stats-scope.test.ts index 9b386f651..e4b912900 100644 --- a/src/__tests__/stats-scope.test.ts +++ b/src/__tests__/stats-scope.test.ts @@ -209,6 +209,9 @@ beforeEach(() => { consoleLog = vi.spyOn(console, 'log').mockImplementation(() => undefined); initWorkspace(); DIRS = projectDirs(); + // An older, unkeyed event counts only while its cwd exists (#785). + fs.mkdirSync(DIRS.project, { recursive: true }); + fs.mkdirSync(DIRS.other, { recursive: true }); }); afterEach(() => { @@ -297,11 +300,13 @@ describe('showStats scope and idempotency', () => { expect(outputNumber(out, 'Conversation turns:')).toBe(1); }); - it('keeps sessions recorded before events carried a data home out of the user scope', async () => { - // A user-scope run from a plain directory. These events carry no dataHome, - // and the user scope never reports such events (#785), so it does not count - // them either: the display side follows the report side. + it('leaves a project\'s sessions out of the user scope even when no project resolves for the cwd', async () => { + // A user-scope run from a plain directory. The user scope reports only its + // own sessions (#785): an older unkeyed session goes to the scope its cwd + // resolves to now, so P's stays out and the one elsewhere counts. await seedUserScopeWithProject(); + fs.mkdirSync(DIRS.project, { recursive: true }); + fs.mkdirSync(DIRS.other, { recursive: true }); await appendEvents([ ...session('sess-1', DIRS.project), ...session('sess-2', DIRS.other), @@ -321,7 +326,8 @@ describe('showStats scope and idempotency', () => { fs.mkdirSync(plainDir, { recursive: true }); const out = await showStatsFromPlainDir(plainDir); - expect(out).toContain('No usage data yet.'); + expect(outputNumber(out, 'Sessions:')).toBe(1); + expect(outputNumber(out, 'Conversation turns:')).toBe(1); }); it('subtracts the scope\'s own reported snapshot, not the shared one', async () => { diff --git a/src/__tests__/team-push-interventions.test.ts b/src/__tests__/team-push-interventions.test.ts index 1b421ef39..040be75bf 100644 --- a/src/__tests__/team-push-interventions.test.ts +++ b/src/__tests__/team-push-interventions.test.ts @@ -35,11 +35,14 @@ vi.mock('../utils/fs.js', async (importActual) => { }); import { reportUsageToTeam } from '../team-push.js'; +import { dataHomeKey } from '../dashboard-collector.js'; import { withTimeout } from '../utils/async.js'; let tmpDir: string; let repoDir: string; let originalHome: string; +/** `dataHomeKey()` of the user scope that `gitConfig()` reports. */ +let userKey: string; function gitConfig(): LocalConfig { return { @@ -63,10 +66,11 @@ function reportedSnapshot(name: string, scoped = true): Record return fs.existsSync(p) ? JSON.parse(fs.readFileSync(p, 'utf-8')) : {}; } -beforeEach(() => { +beforeEach(async () => { tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'teamai-tp-iv-')); originalHome = process.env.HOME ?? ''; process.env.HOME = tmpDir; + userKey = await dataHomeKey(path.join(tmpDir, '.teamai')); repoDir = path.join(tmpDir, 'repo'); fs.mkdirSync(repoDir, { recursive: true }); pushRepoDirectly.mockReset().mockResolvedValue(undefined); @@ -88,8 +92,7 @@ afterEach(() => { function writeDashboardEvents(lines: object[]): void { const p = path.join(tmpDir, '.teamai', 'dashboard', 'events.jsonl'); fs.mkdirSync(path.dirname(p), { recursive: true }); - const dataHome = path.join(tmpDir, '.teamai'); - fs.writeFileSync(p, lines.map((l) => JSON.stringify({ ...l, dataHome })).join('\n') + '\n'); + fs.writeFileSync(p, lines.map((l) => JSON.stringify({ ...l, dataHomeKey: userKey })).join('\n') + '\n'); } describe('reportUsageToTeam — intervention reporting', () => { diff --git a/src/dashboard-collector.ts b/src/dashboard-collector.ts index defbf7e4a..916777ef1 100644 --- a/src/dashboard-collector.ts +++ b/src/dashboard-collector.ts @@ -7,7 +7,7 @@ import { deriveSessionId } from './utils/session-id.js'; import { resolveHookCwd } from './utils/hook-cwd.js'; import { ensureDir } from './utils/fs.js'; import { repoKeys, repoLabel } from './utils/repo-attribution.js'; -import { resolveMonitorPid } from './pid-monitor.js'; +import { isProcessAlive, resolveMonitorPid } from './pid-monitor.js'; import { normalizeToolName } from './utils/tool-names.js'; import { redactWithEnv } from './utils/redact.js'; import { @@ -48,7 +48,7 @@ import { estimateClaudeRequest } from './model-pricing.js'; // │ extract: session_id / cwd / tool_name / prompt // ▼ // DashboardEvent -// │ dataHome = data home of the scope the hook resolved +// │ dataHomeKey = key of the data home of the scope the hook resolved // ▼ // appendEvent(event) → events.jsonl // @@ -224,6 +224,30 @@ interface JsonlTranscriptScan { } /** Scan the Claude/Codex JSONL transcript once. */ +/** + * Whether a Claude transcript entry is a human turn, what the Stop scan counts + * as a prompt: plain-string user content, or user content with text that is + * neither an interrupt marker nor a system injection; never meta or sidechain. + * One human turn per user entry (tool_result-only entries have no human text). + */ +export function isHumanPromptEntry(entry: unknown): boolean { + if (!entry || typeof entry !== 'object' || !('type' in entry) || entry.type !== 'user') return false; + if (('isMeta' in entry && entry.isMeta === true) || ('isSidechain' in entry && entry.isSidechain === true)) return false; + const message = 'message' in entry && entry.message && typeof entry.message === 'object' ? entry.message : undefined; + const content: unknown = message && 'content' in message ? message.content : undefined; + if (typeof content === 'string') { + const text = content.trim(); + return !!text && !text.startsWith(TRANSCRIPT_INTERRUPT_PREFIX) && !TRANSCRIPT_SYSTEM_PREFIXES.some((p) => text.startsWith(p)); + } + if (!Array.isArray(content)) return false; + return content.some((item: unknown) => { + if (!item || typeof item !== 'object' || !('type' in item) || item.type !== 'text' || !('text' in item)) return false; + if (typeof item.text !== 'string' || item.text.startsWith(TRANSCRIPT_INTERRUPT_PREFIX)) return false; + const text = item.text.trim(); + return !!text && !TRANSCRIPT_SYSTEM_PREFIXES.some((p) => text.startsWith(p)); + }); +} + async function scanJsonlTranscriptOnce(transcriptPath: string, modelAliases?: Record): Promise { let interrupt = 0; let toolReject = 0; @@ -352,33 +376,13 @@ async function scanJsonlTranscriptOnce(transcriptPath: string, modelAliases?: Re if (entry.type !== 'user') continue; - const isMeta = entry.isMeta === true || entry.isSidechain === true; + if (isHumanPromptEntry(entry)) prompts++; const content = entry.message?.content; - - // Plain-string user content = a genuine human prompt (older transcript shape). - if (typeof content === 'string') { - const trimContent = content.trim(); - if ( - !isMeta && - trimContent && - !trimContent.startsWith(TRANSCRIPT_INTERRUPT_PREFIX) && - !TRANSCRIPT_SYSTEM_PREFIXES.some((p) => trimContent.startsWith(p)) - ) { - prompts++; - } - continue; - } if (!Array.isArray(content)) continue; - let hasHumanText = false; for (const item of content as Array>) { if (item?.type === 'text' && typeof item.text === 'string') { - const txt = item.text.trim(); - if (item.text.startsWith(TRANSCRIPT_INTERRUPT_PREFIX)) { - interrupt++; - } else if (txt && !TRANSCRIPT_SYSTEM_PREFIXES.some((p) => txt.startsWith(p))) { - hasHumanText = true; - } + if (item.text.startsWith(TRANSCRIPT_INTERRUPT_PREFIX)) interrupt++; } else if (item?.type === 'tool_result' && item.is_error === true) { const text = typeof item.content === 'string' ? item.content @@ -395,8 +399,6 @@ async function scanJsonlTranscriptOnce(transcriptPath: string, modelAliases?: Re } } } - // One human turn per user entry (tool_result-only entries have no human text). - if (hasHumanText && !isMeta) prompts++; } } catch (e) { log.warn(`dashboard: failed to scan transcript: ${(e as Error).message}`); @@ -784,7 +786,8 @@ async function waitForCopilotRunUsage( } /** Resolve Copilot's local event log without accepting path traversal via sessionId. */ -function resolveCopilotUsageTranscript( +/** Copilot's own session log for `sessionId`, derived from the ID alone (TeamAI stores no path of it, #666). */ +export function resolveCopilotUsageTranscript( sessionId: string, ): string | null { if (!COPILOT_SESSION_ID_RE.test(sessionId) || sessionId === '.' || sessionId === '..') return null; @@ -1225,6 +1228,15 @@ export async function parseHookEvent( if (!isCopilot) event.promptSummary = redactWithEnv(human).slice(0, 200); } + // The transcript a session runs in records where it started, which a report + // needs once compaction dropped the session's earlier events (#785). A + // SessionStart on a resume from another project names a file that never + // exists, so it is not kept there. + if ((eventType === 'prompt_submit' || eventType === 'session_end') && !isCopilot + && typeof hookData.transcript_path === 'string') { + event.transcriptPath = hookData.transcript_path; + } + // Extract transcript path, AI output and intervention counts from Stop event if (eventType === 'stop' && !isCopilot && typeof hookData.transcript_path === 'string') { event.transcriptPath = hookData.transcript_path; @@ -1481,6 +1493,22 @@ export async function reconcileRequestLog( } } +/** + * The key an event records for the scope that wrote it, and that a scope's + * report matches (#785): a hash of the data home, so the log stores no path + * (Copilot events persist none, #666). The data home is realpath'd, so a + * symlinked checkout or macOS `/tmp` vs `/private/tmp` keys the same; a + * Windows path also folds separators and case, as the report's cwd rule does. + * A data home that is gone (an in-repo `.teamai` removed after migration) + * keys through its parent's realpath. + */ +export async function dataHomeKey(dataHome: string): Promise { + const real = await fs.promises.realpath(dataHome).catch(() => !path.isAbsolute(dataHome) ? dataHome + : fs.promises.realpath(path.dirname(dataHome)).then((p) => path.join(p, path.basename(dataHome)), () => dataHome)); + const norm = /^(?:[A-Za-z]:[\\/]|\\\\)/.test(real) ? real.replace(/\\/g, '/').toLowerCase() : real; + return createHash('sha256').update(norm.replace(/\/+$/, '')).digest('hex').slice(0, 16); +} + /** * Append a DashboardEvent to the events JSONL file. * Silently fails on I/O errors to avoid disrupting the AI session. @@ -1809,6 +1837,19 @@ export function aggregateSessionMetrics( const unscopedTokens = new Map(); const sessionTokens = new Map(); const transcriptTokens = new Map>(); + // Per rollout, for a transcript-scoped session (Codex), whose counters restart + // with each rollout: see SessionMetrics.segments. + const rolloutSessions = new Set(); + const transcriptPrompts = new Map>(); + const transcriptInterventions = new Map>(); + const transcriptRequests = new Map>>(); + const transcriptCorrections = new Map>(); + const transcriptErrors = new Map>(); + const transcriptSubmits = new Map>(); + const stopAt = new Map>(); + const transcriptSince = new Map>(); + const lastTranscript = new Map(); + const timeline = new Map>(); for (const event of events) { let m = map.get(event.sessionId); @@ -1816,6 +1857,54 @@ export function aggregateSessionMetrics( m = { interrupt: 0, toolReject: 0, correction: 0, prompts: 0, tokens: emptyTokenUsage() }; map.set(event.sessionId, m); } + if (typeof event.transcriptPath === 'string') { + const since = transcriptSince.get(event.sessionId) ?? new Map(); + const first = since.get(event.transcriptPath); + if (first === undefined || Date.parse(event.timestamp) < Date.parse(first)) since.set(event.transcriptPath, event.timestamp); + transcriptSince.set(event.sessionId, since); + lastTranscript.set(event.sessionId, event.transcriptPath); + } + if (event.tokenScope === 'transcript' || (isCodexTool(event.tool) && typeof event.transcriptPath === 'string')) { + rolloutSessions.add(event.sessionId); + } + const events = timeline.get(event.sessionId) ?? []; + const rolloutOf = event.transcriptPath ?? lastTranscript.get(event.sessionId); + events.push({ at: Date.parse(event.timestamp), transcript: rolloutOf }); + timeline.set(event.sessionId, events); + if (event.status === 'error' && rolloutOf !== undefined) { + transcriptErrors.set(event.sessionId, (transcriptErrors.get(event.sessionId) ?? new Set()).add(rolloutOf)); + } + if (event.type === 'prompt_submit' && rolloutOf !== undefined) { + const submits = transcriptSubmits.get(event.sessionId) ?? new Map(); + submits.set(rolloutOf, (submits.get(rolloutOf) ?? 0) + 1); + transcriptSubmits.set(event.sessionId, submits); + } + if (event.type === 'stop' && typeof event.transcriptPath === 'string') { + const rollout = event.transcriptPath; + // The latest Stop by its timestamp, as for tokens: background Stop + // handlers may append an older scan after a newer one. + const at = Date.parse(event.timestamp); + const latest = stopAt.get(event.sessionId) ?? new Map(); + const seen = latest.get(rollout); + const newest = seen === undefined || !Number.isFinite(at) || !Number.isFinite(seen) || at >= seen; + if (newest) { + latest.set(rollout, at); + stopAt.set(event.sessionId, latest); + } + const record = (maps: Map>, value: T) => { + if (!newest) return; + const perRollout = maps.get(event.sessionId) ?? new Map(); + perRollout.set(rollout, value); + maps.set(event.sessionId, perRollout); + }; + if (typeof event.prompts === 'number') record(transcriptPrompts, event.prompts); + if (event.interventions) { + record(transcriptInterventions, { interrupt: event.interventions.interrupt, toolReject: event.interventions.toolReject }); + } + // An older Stop records its cost as one request, on its own day. + if (event.requestDaily) record(transcriptRequests, event.requestDaily); + else if (event.requestMetrics) record(transcriptRequests, { [event.timestamp.slice(0, 10)]: event.requestMetrics }); + } if (event.type === 'stop') { if (event.interventions) { @@ -1835,6 +1924,12 @@ export function aggregateSessionMetrics( const isCorrection = event.correction ?? isCorrectionPrompt(event.promptSummary); if (gap >= 0 && gap <= CORRECTION_WINDOW_MS && isCorrection) { m.correction++; + const rollout = event.transcriptPath ?? lastTranscript.get(event.sessionId); + if (rollout !== undefined) { + const corrections = transcriptCorrections.get(event.sessionId) ?? new Map(); + corrections.set(rollout, (corrections.get(rollout) ?? 0) + 1); + transcriptCorrections.set(event.sessionId, corrections); + } } // Each stop is consumed once — a later prompt is a new task, not a correction. lastStopAt.delete(event.sessionId); @@ -1870,6 +1965,7 @@ export function aggregateSessionMetrics( if (sessionSnapshot) { // The newer thread-level counter already spans rollout files. m.tokens = { ...sessionSnapshot.tokens }; + m.tokensSpanRollouts = true; } else if (segments && segments.size > 0) { let total = emptyTokenUsage(); for (const segment of segments.values()) total = addTokenUsage(total, segment.tokens); @@ -1878,7 +1974,41 @@ export function aggregateSessionMetrics( const unscoped = unscopedTokens.get(sid); if (unscoped) m.tokens = { ...unscoped.tokens }; } - m.prompts = Math.max(submitCount.get(sid) ?? 0, stopPrompts.get(sid) ?? 0); + if (rolloutSessions.has(sid)) { + // Active time per rollout: each gap goes to the rollout of the event it ends at. + const durations = new Map(); + const own = [...(timeline.get(sid) ?? [])].sort((a, b) => a.at - b.at); + for (let i = 1; i < own.length; i++) { + const gap = own[i].at - own[i - 1].at; + const rollout = own[i].transcript; + if (rollout !== undefined && Number.isFinite(gap) && gap >= 0 && gap <= DASHBOARD_IDLE_TIMEOUT_MS) { + durations.set(rollout, (durations.get(rollout) ?? 0) + gap); + } + } + const since = transcriptSince.get(sid) ?? new Map(); + m.segments = Object.fromEntries([...since].map(([transcript, first]) => [transcript, { + // A Codex Stop may count no prompts: the rollout's submits then do. + prompts: Math.max(transcriptPrompts.get(sid)?.get(transcript) ?? 0, transcriptSubmits.get(sid)?.get(transcript) ?? 0), + // A session-scoped counter already spans the rollouts: none holds tokens of its own. + tokens: { ...(sessionSnapshot ? emptyTokenUsage() : segments?.get(transcript)?.tokens ?? emptyTokenUsage()) }, + interrupt: transcriptInterventions.get(sid)?.get(transcript)?.interrupt ?? 0, + toolReject: transcriptInterventions.get(sid)?.get(transcript)?.toolReject ?? 0, + correction: transcriptCorrections.get(sid)?.get(transcript) ?? 0, + durationMs: durations.get(transcript) ?? 0, + requestDaily: transcriptRequests.get(sid)?.get(transcript) ?? {}, + since: first, + error: transcriptErrors.get(sid)?.has(transcript) ?? false, + }])); + // A rollout's Stop counts restart too; the session sums them. + const sum = (field: 'interrupt' | 'toolReject') => + Object.values(m.segments ?? {}).reduce((total, segment) => total + segment[field], 0); + m.interrupt = Math.max(m.interrupt, sum('interrupt')); + m.toolReject = Math.max(m.toolReject, sum('toolReject')); + } + // A rollout's prompt count restarts too: its segments sum where they exist. + const rolloutPrompts = m.segments + ? Object.values(m.segments).reduce((sum, segment) => sum + segment.prompts, 0) : 0; + m.prompts = Math.max(submitCount.get(sid) ?? 0, stopPrompts.get(sid) ?? 0, rolloutPrompts); } return map; @@ -1901,7 +2031,10 @@ export function aggregateSessionInterventions( /** * Compact events.jsonl by keeping only events for active sessions. - * Active = not stopped and last activity within STALE_TIMEOUT. + * Active = not stopped and last activity within STALE_TIMEOUT, or its tool + * process still running: an exit a dashboard wrote before `processExitAfter` + * existed can mark a live run stopped, and dropping that run's start would give + * its next activity a new run ID (#785). * Called when file exceeds COMPACTION_THRESHOLD lines. */ export async function compactEvents(eventsPath?: string): Promise { @@ -1918,6 +2051,13 @@ export async function compactEvents(eventsPath?: string): Promise { const events = await readEventsRaw(filePath); const activeSessions = rebuildSessions(events); const activeIds = new Set(activeSessions.map(s => s.sessionId)); + const monitored = new Map(); + for (const e of events) { + if (e.type === 'session_start' && typeof e.monitorPid === 'number') monitored.set(e.sessionId, e.monitorPid); + } + for (const [sessionId, pid] of monitored) { + if (isProcessAlive(pid)) activeIds.add(sessionId); + } // Keep only events for active sessions const kept = events.filter(e => activeIds.has(e.sessionId)); @@ -1986,7 +2126,7 @@ export async function dashboardReport(toolArg?: string): Promise { const event = await parseHookEvent(raw, toolArg ?? 'claude'); if (!event) return; - event.dataHome = getDataHome(config); + event.dataHomeKey = await dataHomeKey(getDataHome(config)); event.projectAnchor = await eventProjectAnchor(event.cwd); await appendEvent(event); diff --git a/src/dashboard-scope.ts b/src/dashboard-scope.ts new file mode 100644 index 000000000..e38dfdac3 --- /dev/null +++ b/src/dashboard-scope.ts @@ -0,0 +1,270 @@ +/** + * The dashboard events each scope reports (#785): the log split into runs, and + * each run given, whole, to the scope it started in. + */ +import fs from 'node:fs'; +import path from 'node:path'; +import { dataHomeKey, resolveCopilotUsageTranscript } from './dashboard-collector.js'; +import { pathExists, readJson } from './utils/fs.js'; +import type { DashboardEvent, LocalConfig } from './types.js'; +import { getDataHome } from './types.js'; +import { getUserHome } from './utils/home.js'; +import { REPORTED_SNAPSHOTS, readSessionOwners, snapshotPathIn } from './session-owners.js'; + +/** + * The dashboard events a scope reports (#785): every event of the sessions + * recorded in it. A session is the run of one ID up to its session_end or + * process_exit, since a PID-fallback ID comes back for a later run; each run + * is returned under its own ID (see {@link adoptBareKeys}). It is + * decided once, whole, by its first event keyed with a data home + * (`dataHomeKey`, or the path an unreleased build of #795 wrote), because a + * Stop carries the whole transcript's totals: split per event, a session that + * moved into another scope would count there again the part recorded before + * the move. A project also owns the key of its in-repo `.teamai`, where a + * hook recorded until migration moved the project to a partition. A session + * written before events carried a key is decided by its first cwd, by the + * dispatcher's rule: the scope that directory resolves to now, so a nested + * clone under a project is not the project's; one with no cwd, or a cwd gone + * since, is no scope's. A tool's own session ID a scope has recorded as its + * own (see `session-owners.jsonl`) is that scope's, whatever the log still + * holds. A caller without a scope config reads the whole log. + */ +export async function filterEventsByScope( + events: DashboardEvent[], + config?: LocalConfig, +): Promise { + if (!config) return events; + const ownKey = await dataHomeKey(getDataHome(config)); + const keys = new Set([ownKey]); + if (config.projectRoot) { + // Unless the project is rooted at HOME, where that is the user scope's. + const legacy = await dataHomeKey(path.join(config.projectRoot, '.teamai')); + if (legacy !== (await dataHomeKey(path.join(getUserHome(), '.teamai')))) keys.add(legacy); + } + const { resolveConfigForDir } = await import('./config.js'); + const resolvesHere = new Map>(); + const ownsCwd = (cwd: string): Promise => { + let owns = resolvesHere.get(cwd); + if (!owns) { + owns = pathExists(cwd).then(async (exists) => { + const resolved = exists ? await resolveConfigForDir(cwd) : null; + return !!resolved && keys.has(await dataHomeKey(getDataHome(resolved))); + }); + resolvesHere.set(cwd, owns); + } + return owns; + }; + const eventKeys = await keysOf(events); + const { runOf, runIds, deciding } = splitRuns(events, eventKeys); + const owners = await readSessionOwners(); + const transcripts = new Map(); + events.forEach((e, i) => { + const run = runOf[i]; + if (run === undefined) return; + // Copilot's session log is found by its ID; the event holds no path (#666). + const transcript = typeof e.transcriptPath === 'string' ? e.transcriptPath + : e.tool === 'copilot' ? resolveCopilotUsageTranscript(e.sessionId) : null; + const known = transcripts.get(run) ?? []; + if (transcript && !known.includes(transcript)) transcripts.set(run, [...known, transcript]); + }); + const owned = await Promise.all(deciding.map(async (i, run) => { + // Recorded by the scope that first reported it, so a session resumed + // elsewhere after compaction dropped its events stays that scope's. + const owner = owners.get(runIds[run]) ?? (runIds[run].startsWith('pid-') ? undefined + : await transcriptOwner(runIds[run], transcripts.get(run) ?? [], resolveConfigForDir)); + if (owner !== undefined) return keys.has(owner); + if (i === undefined) return false; + const key = eventKeys[i]; + const cwd = events[i].cwd; + return key !== undefined ? keys.has(key) : !!cwd && await ownsCwd(cwd); + })); + return events.flatMap((e, i) => { + const run = runOf[i]; + return run !== undefined && owned[run] ? [{ ...e, sessionId: runIds[run] }] : []; + }); +} + +/** + * The key of the scope a tool's own session started in, by its transcript, + * when that scope's snapshots already hold it: an earlier release reported it + * there, and compaction has since dropped the events that would say so. The + * latest transcript path first, since a tool may relocate the file. Undefined + * when there is no such transcript, origin, scope or snapshot entry (a fork + * under a new ID, a scope that never reported it). + */ +async function transcriptOwner( + sessionId: string, + paths: string[], + resolveConfigForDir: (dir: string) => Promise, +): Promise { + for (const transcript of [...paths].reverse()) { + const origin = await transcriptOrigin(transcript); + if (origin === undefined) continue; + const config = (await pathExists(origin)) ? await resolveConfigForDir(origin) : null; + if (!config) return undefined; + const dataHome = getDataHome(config); + const snapshots = await Promise.all(REPORTED_SNAPSHOTS.map((name) => + readJson>(snapshotPathIn(dataHome, name)))); + const held = snapshots.some((snapshot) => !!snapshot && typeof snapshot === 'object' && Object.hasOwn(snapshot, sessionId)); + return held ? dataHomeKey(dataHome) : undefined; + } + return undefined; +} + +/** + * The directory a transcript's session started in: the first `cwd` a Claude + * transcript records (a resume from another project appends to the same file), + * a Codex rollout's `session_meta`, or a Copilot session log's `session.start`. + * Reads a bounded head of the file; the format is the tool's own, so anything + * else is undefined. + */ +async function transcriptOrigin(transcript: string): Promise { + let head: string; + try { + const fh = await fs.promises.open(transcript, 'r'); + try { + const buffer = Buffer.alloc(64 * 1024); + const { bytesRead } = await fh.read(buffer, 0, buffer.length, 0); + head = buffer.subarray(0, bytesRead).toString('utf8'); + } finally { + await fh.close(); + } + } catch { + return undefined; + } + for (const line of head.split('\n')) { + let entry: unknown; + try { + entry = JSON.parse(line); + } catch { + continue; + } + if (!entry || typeof entry !== 'object') continue; + if ('cwd' in entry && typeof entry.cwd === 'string') return entry.cwd; + if ('type' in entry && entry.type === 'session_meta' && 'payload' in entry && entry.payload + && typeof entry.payload === 'object' && 'cwd' in entry.payload && typeof entry.payload.cwd === 'string') { + return entry.payload.cwd; + } + if ('type' in entry && entry.type === 'session.start' && 'data' in entry && entry.data && typeof entry.data === 'object' + && 'context' in entry.data && entry.data.context && typeof entry.data.context === 'object' + && 'cwd' in entry.data.context && typeof entry.data.context.cwd === 'string') { + return entry.data.context.cwd; + } + } + return undefined; +} + +/** + * Each event's data home key. The log is hand-editable: a key that is not a + * string counts as absent. An unreleased build of #795 recorded the data home + * path in place of its key. + */ +async function keysOf(events: DashboardEvent[]): Promise> { + const hashes = new Map>(); + return Promise.all(events.map((e) => { + if (typeof e.dataHomeKey === 'string') return e.dataHomeKey; + if (typeof e.dataHome !== 'string') return undefined; + let key = hashes.get(e.dataHome); + if (!key) { + key = dataHomeKey(e.dataHome); + hashes.set(e.dataHome, key); + } + return key; + })); +} + +/** + * The runs of a log: each event's run (none for an exit whose run is gone), + * each run's ID, and the event that decides its scope. + * + * A tool's own session ID is one run, whatever ends it records: `claude + * --resume` continues it, in a new process, and its Stop carries the whole + * transcript. It is returned under the ID itself, so it stays the session + * already reported after compaction drops its events. A PID-fallback ID + * (`pid-…`) names one run until it ends, then comes back for a later one, + * maybe in another scope, so each run is decided on its own and returned under + * the ID plus its first event's timestamp, which stays the same whichever + * earlier runs compaction has dropped. A second end with nothing recorded since + * the first (the dashboard monitor's process_exit after SessionEnd) belongs to + * the run just closed. A start from another process than its open run's begins + * a new run, though nothing ended that one (a crash with no dashboard running). + * The monitor's exit names the last event it observed (`processExitAfter`). + * A dashboard started before that field existed wrote none; since a dead + * process records nothing more, such an exit followed by more events of its + * fallback ID before the next start did not end the open run: it was observed + * before that run began, and belongs to the run closed before it. + */ +function splitRuns(events: DashboardEvent[], eventKeys: Array): { + runOf: Array; + runIds: string[]; + deciding: Array; +} { + const runOf: Array = []; + const runPid: Array = []; + const observedRuns = new Map(); + const openRun = new Map(); + const closedRun = new Map(); + const deciding: Array = []; + const runIds: string[] = []; + // Whether each event is followed by activity of its ID (anything but a start + // or a monitor exit) before that ID's next start. + const continues: boolean[] = []; + const active = new Map(); + for (let i = events.length - 1; i >= 0; i--) { + const { sessionId, type } = events[i]; + continues[i] = active.get(sessionId) ?? false; + if (type === 'session_start') active.set(sessionId, false); + else if (type !== 'process_exit') active.set(sessionId, true); + } + events.forEach((e, i) => { + const fallback = e.sessionId.startsWith('pid-'); + const ends = e.type === 'session_end' || e.type === 'process_exit'; + const observed = e.type === 'process_exit' && typeof e.processExitAfter === 'string'; + const starts = e.type === 'session_start' && typeof e.monitorPid === 'number'; + const open = openRun.get(e.sessionId); + if (starts && open !== undefined && fallback + && typeof runPid[open] === 'number' && runPid[open] !== e.monitorPid) { + // Nothing ended it (a crash), but a late exit of it must still find it. + openRun.delete(e.sessionId); + closedRun.set(e.sessionId, open); + } + const closed = closedRun.get(e.sessionId); + const stale = e.type === 'process_exit' && !observed && fallback && open !== undefined && closed !== undefined + && continues[i]; + let run = observed ? observedRuns.get(`${e.sessionId}@${e.processExitAfter}`) + : stale ? closed : openRun.get(e.sessionId) ?? (ends ? closed : undefined); + // The observed run may have been compacted. Its delayed exit must not + // manufacture a new session or close a later reuse of the same ID. + if (observed && run === undefined) { + runOf.push(undefined); + return; + } + if (run === undefined) { + run = deciding.push(undefined) - 1; + runIds.push(fallback ? `${e.sessionId}@${e.timestamp}` : e.sessionId); + } + if (ends && fallback && !stale && (!observed || openRun.get(e.sessionId) === run)) { + openRun.delete(e.sessionId); + closedRun.set(e.sessionId, run); + } else if (!ends || !fallback) { + openRun.set(e.sessionId, run); + } + if (starts) runPid[run] ??= e.monitorPid; + runOf.push(run); + observedRuns.set(`${e.sessionId}@${e.timestamp}`, run); + const current = deciding[run]; + if (eventKeys[i] !== undefined ? current === undefined || eventKeys[current] === undefined + : current === undefined && !!e.cwd) deciding[run] = i; + }); + return { runOf, runIds, deciding }; +} + +/** Every event of the log under its run ID, whichever scope it belongs to. */ +export async function runsOfLog(events: DashboardEvent[]): Promise { + const { runOf, runIds } = splitRuns(events, await keysOf(events)); + return events.flatMap((e, i) => { + const run = runOf[i]; + return run !== undefined ? [{ ...e, sessionId: runIds[run] }] : []; + }); +} + diff --git a/src/dashboard.ts b/src/dashboard.ts index 3f94a03d0..874c8a9d5 100644 --- a/src/dashboard.ts +++ b/src/dashboard.ts @@ -121,10 +121,11 @@ export async function startDashboard(port?: number): Promise { type: 'process_exit', timestamp: new Date().toISOString(), sessionId: session.sessionId, + processExitAfter: session.lastActivity, tool: session.tool, cwd: session.cwd, // The session's own data home, so its scope's report still sees it end. - dataHome: events.find((e) => e.sessionId === session.sessionId && e.dataHome)?.dataHome, + dataHomeKey: events.find((e) => e.sessionId === session.sessionId && e.dataHomeKey)?.dataHomeKey, // And its repo, which the cwd may no longer lead to (a removed worktree). projectAnchor: [...events].reverse().find((e) => e.sessionId === session.sessionId && e.projectAnchor)?.projectAnchor, }; diff --git a/src/hook-handlers.ts b/src/hook-handlers.ts index 2fdca7978..877d56498 100644 --- a/src/hook-handlers.ts +++ b/src/hook-handlers.ts @@ -191,7 +191,7 @@ const dashboardReportHandler: HookHandler = { async execute(stdin, tool, config) { // Registered with requiresConfig: a session outside any scope is not recorded. if (!config) return null; - const { parseHookEvent, appendEvent, compactEvents, eventProjectAnchor } = await import('./dashboard-collector.js'); + const { parseHookEvent, appendEvent, compactEvents, dataHomeKey, eventProjectAnchor } = await import('./dashboard-collector.js'); const { getDataHome } = await import('./types.js'); const raw = JSON.stringify(stdin); const event = await parseHookEvent(raw, tool, { @@ -199,9 +199,9 @@ const dashboardReportHandler: HookHandler = { modelAliases: await userModelAliases(stdin), }); if (event) { - // The dispatcher's scope, which knows the project even when the host - // sends no cwd (Copilot) or a symlinked one (#785). - event.dataHome = getDataHome(config); + // The dispatcher's scope, which knows the project even when the event + // records no cwd (Copilot) or a symlinked one (#785). + event.dataHomeKey = await dataHomeKey(getDataHome(config)); event.projectAnchor = await eventProjectAnchor(event.cwd); await appendEvent(event); // Non-blocking compaction diff --git a/src/session-owners.ts b/src/session-owners.ts new file mode 100644 index 000000000..b762d9312 --- /dev/null +++ b/src/session-owners.ts @@ -0,0 +1,389 @@ +/** + * Which scope a tool's own session belongs to once compaction has dropped the + * events that would say so (#785): the machine-level session-owners index, its + * seeding from the snapshots an earlier release left, and the snapshot files it + * reads for that. + */ +import fs from 'node:fs'; +import path from 'node:path'; +import { readEvents, dataHomeKey, isHumanPromptEntry } from './dashboard-collector.js'; +import { readFileSafe, ensureDir, pathExists, readJson } from './utils/fs.js'; +import { log } from './utils/logger.js'; +import type { RequestCostMetrics, TokenUsage } from './types.js'; +import { getDataHome, getTeamaiHomeDir, emptyTokenUsage } from './types.js'; +import { projectsRootDir } from './utils/partition.js'; +import { parseDailySnapshot, type DailySessionSnapshot } from './session-trends.js'; + +/** Snapshot of already-reported per-session intervention counts (idempotency basis). */ +export type ReportedInterventions = Record; + +/** A rollout's reported totals, keyed by a hash of its transcript path (no path is stored). */ +export type ReportedSegments = Record; + /** The rollout errored, was interrupted or corrected: its session did not succeed. */ + failed?: boolean; +}>; + +/** Snapshot of already-reported per-session prompt counts + token usage (idempotency basis). */ +export type ReportedPromptTokens = Record; + + +export const REPORTED_SNAPSHOTS = ['interventions', 'prompt-tokens', 'daily-sessions'] as const; +export type ReportedSnapshotName = typeof REPORTED_SNAPSHOTS[number]; + +/** The machine-level snapshot every scope shared before #786 (evaluated at call time for tests). */ +export function sharedSnapshotPath(name: ReportedSnapshotName): string { + return path.join(getTeamaiHomeDir(), 'dashboard', `reported-${name}.json`); +} + +/** The snapshot of the scope whose data home is `dataHome`. */ +export function snapshotPathIn(dataHome: string, name: ReportedSnapshotName): string { + const file = `reported-${name}.json`; + if (path.resolve(dataHome) !== path.resolve(getTeamaiHomeDir())) return path.join(dataHome, 'dashboard', file); + return path.join(dataHome, 'dashboard', `user-${file}`); +} + +// ─── Session owners ───────────────────────────────────── +// +// ~/.teamai/dashboard/session-owners.jsonl {"sessionId":"","dataHomeKey":""} per line +// +// A tool's own session ID can be resumed anywhere, long after compaction +// dropped its events, and its Stop carries the whole transcript. So the scope +// that first reports it records itself here, append-only, and the session +// stays that scope's. Only the ID and the key, never a path (#666). The first +// line for an ID wins. +// +// A release before this file kept per-scope snapshots only, so the file is +// first written from them: each tool's own ID in any snapshot of a scope is +// the scope's that holds its greatest total (prompts, then tokens), since a +// session that release split per event holds only part of it elsewhere. A tie +// names no owner: that release copied the shared file into every scope. The +// scopes read are the user scope, every partition, and a project whose data +// home is in its workspace that a session still in the log leads to. Each +// report also records the IDs of its own snapshots that have no owner yet, +// for such a project the log no longer leads to, when they show it reported +// them: absent from the shared snapshot, or past its total there. +// + +function sessionOwnersPath(): string { + return path.join(getTeamaiHomeDir(), 'dashboard', 'session-owners.jsonl'); +} + +/** The data homes whose snapshots an earlier release may have written. */ +async function knownDataHomes(): Promise { + const slugs = await fs.promises.readdir(projectsRootDir()).catch(() => []); + const homes = [getTeamaiHomeDir(), ...[...slugs].sort().map((slug) => path.join(projectsRootDir(), slug))]; + // A project whose data home is in its workspace is under no partition; a + // session of it still in the log leads to it. + const { resolveConfigForDir } = await import('./config.js'); + const cwds = new Set((await readEvents()).flatMap((e) => (typeof e.cwd === 'string' ? [e.cwd] : []))); + for (const cwd of cwds) { + const config = (await pathExists(cwd)) ? await resolveConfigForDir(cwd) : null; + if (config) homes.push(getDataHome(config)); + } + return [...new Set(homes.map((home) => path.resolve(home)))]; +} + +/** A snapshot entry's reported prompts and tokens, 0 for what it does not hold. */ +export function reportedSize(entry: unknown): { prompts: number; tokens: number } { + if (!entry || typeof entry !== 'object') return { prompts: 0, tokens: 0 }; + const prompts = 'prompts' in entry && typeof entry.prompts === 'number' ? entry.prompts : 0; + const tokens = 'tokens' in entry && entry.tokens && typeof entry.tokens === 'object' + ? Object.values(entry.tokens).reduce((sum: number, n: unknown) => sum + (typeof n === 'number' ? n : 0), 0) + : 0; + return { prompts, tokens }; +} + +/** A scope's three snapshots (the shared ones for no data home), as a file holds them. */ +export interface ScopeSnapshots { + interventions: Record | null; + promptTokens: Record | null; + daily: Record | null; +} + +export async function readSnapshotsIn(dataHome: string | undefined): Promise { + const read = (name: ReportedSnapshotName) => + readJson>(dataHome === undefined ? sharedSnapshotPath(name) : snapshotPathIn(dataHome, name)); + const [interventions, promptTokens, daily] = await Promise.all(REPORTED_SNAPSHOTS.map(read)); + return { interventions, promptTokens, daily }; +} + +/** An intervention entry read from a file, zero for what it does not hold. */ +export function interventionsEntry(value: unknown): ReportedInterventions[string] { + const count = (field: 'interrupt' | 'toolReject' | 'correction') => + (value && typeof value === 'object' && field in value && typeof Object.entries(value).find(([k]) => k === field)?.[1] === 'number' + ? Number(Object.entries(value).find(([k]) => k === field)?.[1]) : 0); + return { interrupt: count('interrupt'), toolReject: count('toolReject'), correction: count('correction') }; +} + +/** A prompt-token entry read from a file, zero for what it does not hold. */ +export function promptTokensEntry(value: unknown): ReportedPromptTokens[string] { + const tokens = emptyTokenUsage(); + if (value && typeof value === 'object' && 'tokens' in value && value.tokens && typeof value.tokens === 'object') { + for (const [field, n] of Object.entries(value.tokens)) { + if (typeof n !== 'number') continue; + if (field === 'input' || field === 'output' || field === 'cacheRead' || field === 'cacheCreation') tokens[field] = n; + } + } + return { prompts: reportedSize(value).prompts, tokens }; +} + +/** What a scope's snapshots hold of `id`: prompts, tokens and intervention counts. */ +function heldIn(snapshots: ScopeSnapshots, id: string): { prompts: number; tokens: number; interventions: number } { + const fromTokens = reportedSize(snapshots.promptTokens?.[id]); + const iv = interventionsEntry(snapshots.interventions?.[id]); + return { + prompts: Math.max(fromTokens.prompts, reportedSize(snapshots.daily?.[id]).prompts), + tokens: fromTokens.tokens, + interventions: iv.interrupt + iv.toolReject + iv.correction, + }; +} + +/** + * Whether a scope's snapshots show it reported `id`: the shared snapshots hold + * none of it, or the scope is past their total. An earlier release copied the + * shared snapshots into every scope it ran in, so a copy shows nothing. + */ +export function showsReported(own: ScopeSnapshots, shared: ScopeSnapshots, id: string): boolean { + const inShared = [shared.interventions, shared.promptTokens, shared.daily].some((snapshot) => + !!snapshot && typeof snapshot === 'object' && Object.hasOwn(snapshot, id)); + if (!inShared) return true; + const a = heldIn(own, id); + const b = heldIn(shared, id); + return a.prompts !== b.prompts ? a.prompts > b.prompts + : a.tokens !== b.tokens ? a.tokens > b.tokens : a.interventions > b.interventions; +} + +/** What several scopes had reported of one session, credited to its owner (numbers only). */ +export interface OwnerCredit { + interventions: ReportedInterventions[string]; + promptTokens: ReportedPromptTokens[string]; + daily?: DailySessionSnapshot; + /** Each part's scope key, reported prompts, and whether it ended in a Stop, for {@link creditedPrompts}. */ + parts?: CreditPart[]; +} + +interface CreditPart { key: string; prompts: number; stop: boolean } + +/** + * The credit for a session an earlier release split across scopes per event, + * from their snapshots alone (its events are gone). A part whose daily entry + * shows it ended in a Stop carries the transcript's cumulative total, so the + * greatest such part counts once; a part with no Stop counted its own prompts, + * so those add. Interruptions, rejections and tokens, from Stops, take the + * greatest; corrections, counted per prompt, add. A part without a Stop that came before another's Stop is + * credited twice: that undercounts, once, but never sends a prompt again, + * unless the session's transcript can place it (see {@link creditedPrompts}). + */ +function creditOf(held: Array<{ key: string; snapshots: ScopeSnapshots }>, id: string): OwnerCredit { + const parts = held.map((part) => part.snapshots); + const stops = parts.filter((part) => parseDailySnapshot(part.daily?.[id]) !== undefined); + const loose = parts.filter((part) => !stops.includes(part)); + const prompts = Math.max(0, ...stops.map((part) => heldIn(part, id).prompts)) + + loose.reduce((sum, part) => sum + heldIn(part, id).prompts, 0); + const tokens = emptyTokenUsage(); + const interventions = { interrupt: 0, toolReject: 0, correction: 0 }; + for (const part of parts) { + const entry = promptTokensEntry(part.promptTokens?.[id]); + for (const field of ['input', 'output', 'cacheRead', 'cacheCreation'] as const) { + tokens[field] = Math.max(tokens[field], entry.tokens[field]); + } + // Interruptions and rejections come from Stops, cumulative; corrections + // are counted per prompt in each part's own events. + const iv = interventionsEntry(part.interventions?.[id]); + interventions.interrupt = Math.max(interventions.interrupt, iv.interrupt); + interventions.toolReject = Math.max(interventions.toolReject, iv.toolReject); + interventions.correction += iv.correction; + } + const days = stops.flatMap((part) => { + const day = parseDailySnapshot(part.daily?.[id]); + return day ? [day] : []; + }); + const latest = days.reduce((a, b) => (!a || b.prompts > a.prompts ? b : a), undefined); + return { + interventions, + promptTokens: { prompts, tokens }, + parts: held.map((part) => ({ + key: part.key, prompts: heldIn(part.snapshots, id).prompts, stop: stops.includes(part.snapshots), + })), + ...(latest ? { daily: { ...latest, prompts, durationMs: days.reduce((sum, day) => sum + day.durationMs, 0) } } : {}), + }; +} + +/** + * The owners the per-scope snapshots of an earlier release imply, as the file's + * first lines: for each tool's own ID, the scope whose snapshots show it reported + * it with the greatest total, and, when several did, the credit of their parts. + * A tie names no owner. + */ +async function ownersFromSnapshots(): Promise { + const shared = await readSnapshotsIn(undefined); + const parts = new Map>(); + for (const dataHome of await knownDataHomes()) { + const snapshots = await readSnapshotsIn(dataHome); + const ids = new Set([snapshots.interventions, snapshots.promptTokens, snapshots.daily].flatMap((snapshot) => + (snapshot && typeof snapshot === 'object' ? Object.keys(snapshot) : []))); + const key = await dataHomeKey(dataHome); + for (const sessionId of ids) { + if (sessionId.startsWith('pid-') || !showsReported(snapshots, shared, sessionId)) continue; + parts.set(sessionId, [...(parts.get(sessionId) ?? []), { key, snapshots }]); + } + } + const lines: string[] = []; + for (const [sessionId, held] of parts) { + const size = (part: { snapshots: ScopeSnapshots }) => heldIn(part.snapshots, sessionId); + const best = held.reduce((a, b) => { + const x = size(a); + const y = size(b); + return y.prompts > x.prompts || (y.prompts === x.prompts && y.tokens > x.tokens) ? b : a; + }); + const tied = held.some((part) => part.key !== best.key + && size(part).prompts === size(best).prompts && size(part).tokens === size(best).tokens); + if (tied) continue; + const credit = held.length > 1 ? creditOf(held, sessionId) : undefined; + lines.push(JSON.stringify({ sessionId, dataHomeKey: best.key, ...(credit ? { credit } : {}) })); + } + return lines.map((line) => `${line}\n`).join(''); +} + +/** The credits the file's first line for each ID carries (see {@link creditOf}). */ +export async function readOwnerCredits(): Promise> { + const credits = new Map(); + const seen = new Set(); + for (const line of ((await readFileSafe(sessionOwnersPath())) ?? '').split('\n')) { + let parsed: unknown; + try { + parsed = JSON.parse(line); + } catch { + continue; + } + if (!parsed || typeof parsed !== 'object' || !('sessionId' in parsed) || typeof parsed.sessionId !== 'string') continue; + if (seen.has(parsed.sessionId)) continue; + seen.add(parsed.sessionId); + if (!('credit' in parsed) || !parsed.credit || typeof parsed.credit !== 'object') continue; + const credit = parsed.credit; + const daily = 'daily' in credit ? parseDailySnapshot(credit.daily) : undefined; + const parts: CreditPart[] = []; + if ('parts' in credit && Array.isArray(credit.parts)) { + for (const part of credit.parts) { + if (!part || typeof part !== 'object' || !('key' in part) || typeof part.key !== 'string') continue; + parts.push({ key: part.key, prompts: reportedSize(part).prompts, stop: 'stop' in part && part.stop === true }); + } + } + credits.set(parsed.sessionId, { + interventions: interventionsEntry('interventions' in credit ? credit.interventions : undefined), + promptTokens: promptTokensEntry('promptTokens' in credit ? credit.promptTokens : undefined), + ...(daily ? { daily } : {}), + ...(parts.length > 0 ? { parts } : {}), + }); + } + return credits; +} + +export async function readSessionOwners(): Promise> { + const owners = new Map(); + if (!(await pathExists(sessionOwnersPath()))) { + try { + await ensureDir(path.dirname(sessionOwnersPath())); + // Exclusive: a report in another scope may be writing it too. + await fs.promises.writeFile(sessionOwnersPath(), await ownersFromSnapshots(), { flag: 'wx' }); + } catch (e) { + if ((e as NodeJS.ErrnoException).code !== 'EEXIST') log.debug(`Could not seed session owners: ${(e as Error).message}`); + } + } + const content = await readFileSafe(sessionOwnersPath()); + for (const line of (content ?? '').split('\n')) { + if (!line.trim()) continue; + try { + const parsed: unknown = JSON.parse(line); + if (parsed && typeof parsed === 'object' && 'sessionId' in parsed && 'dataHomeKey' in parsed + && typeof parsed.sessionId === 'string' && typeof parsed.dataHomeKey === 'string' + && !owners.has(parsed.sessionId)) owners.set(parsed.sessionId, parsed.dataHomeKey); + } catch { + // A torn or hand-edited line records no owner. + } + } + return owners; +} + +/** The IDs of a scope's snapshots that show it reported them (see {@link showsReported}). */ +export async function reportedBeyondShared( + promptTokens: Record, + interventions: Record, + daily: Record, +): Promise { + const shared = await readSnapshotsIn(undefined); + const own: ScopeSnapshots = { interventions, promptTokens, daily }; + const ids = new Set([...Object.keys(promptTokens), ...Object.keys(interventions), ...Object.keys(daily)]); + return [...ids].filter((id) => showsReported(own, shared, id)); +} + +/** Records `key` as the owner of the tool-own session IDs among `sessionIds` that have none yet. */ +export async function recordSessionOwners(sessionIds: Iterable, key: string): Promise { + const owners = await readSessionOwners(); + const ids = new Set([...sessionIds].filter((id) => !id.startsWith('pid-') && !owners.has(id))); + if (ids.size === 0) return; + try { + await ensureDir(path.dirname(sessionOwnersPath())); + await fs.promises.appendFile(sessionOwnersPath(), + [...ids].map((sessionId) => JSON.stringify({ sessionId, dataHomeKey: key })).join('\n') + '\n'); + } catch (e) { + log.debug(`Could not record session owners: ${(e as Error).message}`); + } +} + + +/** + * The prompts a credit covers, placed by the session's transcript, which keeps + * every prompt in order with the directory it was typed in (Claude). A part + * that ended in a Stop counted the transcript's first prompts, cumulatively, + * so the greatest such part covers that many; a part with no Stop counted its + * own scope's first prompts, which add only where they come after those. + * Undefined when the credit mixes no such parts, or no transcript places them. + */ +export async function creditedPrompts(credit: OwnerCredit, transcripts: string[]): Promise { + const parts = credit.parts ?? []; + const stopped = parts.filter((part) => part.stop); + const loose = parts.filter((part) => !part.stop); + if (stopped.length === 0 || loose.length === 0) return undefined; + const covered = Math.max(...stopped.map((part) => part.prompts)); + const { resolveConfigForDir } = await import('./config.js'); + const keys = new Map>(); + const keyOf = (cwd: string) => { + let key = keys.get(cwd); + if (!key) { + key = pathExists(cwd).then(async (exists) => { + const config = exists ? await resolveConfigForDir(cwd) : null; + return config ? dataHomeKey(getDataHome(config)) : undefined; + }); + keys.set(cwd, key); + } + return key; + }; + for (const transcript of [...transcripts].reverse()) { + const content = await readFileSafe(transcript); + if (content === null) continue; + const prompts: string[] = []; + for (const line of content.split('\n')) { + let entry: unknown; + try { + entry = JSON.parse(line); + } catch { + continue; + } + if (!isHumanPromptEntry(entry) || !entry || typeof entry !== 'object' || !('cwd' in entry)) continue; + if (typeof entry.cwd === 'string') prompts.push(entry.cwd); + } + if (prompts.length < covered) continue; + const scopes = await Promise.all(prompts.map(keyOf)); + let total = covered; + for (const part of loose) { + const own = scopes.flatMap((key, i) => (key === part.key ? [i] : [])).slice(0, part.prompts); + if (own.length < part.prompts) return undefined; + total += own.filter((i) => i >= covered).length; + } + return total; + } + return undefined; +} diff --git a/src/session-trends.ts b/src/session-trends.ts index b36299bed..70849099a 100644 --- a/src/session-trends.ts +++ b/src/session-trends.ts @@ -77,7 +77,10 @@ export function aggregateDailySessions(events: DashboardEvent[]): Map event.status === 'error'); - const requestDaily = latestRequestDaily(own); + // A Codex rollout's costs restart with it: the session sums its rollouts. + const requestDaily = metric.segments + ? Object.values(metric.segments).reduce((sum, segment) => addRequestDaily(sum, segment.requestDaily), {}) + : latestRequestDaily(own); const corrected = metric.correction > 0 ? 1 : 0; result.set(sessionId, { date: firstStop.timestamp.slice(0, 10), @@ -97,6 +100,157 @@ function positiveDelta(current: number, previous: number | undefined): number { return Math.max(0, current - (previous ?? 0)); } +/** Request costs per day read from a file; empty for what is not one. */ +export function parseRequestDaily(value: unknown): Record { + const requestDaily: Record = {}; + if (!value || typeof value !== 'object') return requestDaily; + for (const [date, request] of Object.entries(value)) { + if (!request || typeof request !== 'object') continue; + const num = (field: string) => { + const n: unknown = Object.entries(request).find(([k]) => k === field)?.[1]; + return typeof n === 'number' ? n : 0; + }; + const version: unknown = Object.entries(request).find(([k]) => k === 'priceVersion')?.[1]; + requestDaily[date] = { + pricedRequests: num('pricedRequests'), costMicros: num('costMicros'), cacheReadTokens: num('cacheReadTokens'), + cacheEligibleInputTokens: num('cacheEligibleInputTokens'), priceVersion: typeof version === 'string' ? version : '', + }; + } + return requestDaily; +} + +/** Request costs per day summed, `a`'s price version where both hold a day. */ +export function addRequestDaily( + a: Record, + b: Record, +): Record { + const sum = { ...a }; + for (const [date, request] of Object.entries(b)) { + const own = sum[date]; + sum[date] = own ? { + pricedRequests: own.pricedRequests + request.pricedRequests, costMicros: own.costMicros + request.costMicros, + cacheReadTokens: own.cacheReadTokens + request.cacheReadTokens, + cacheEligibleInputTokens: own.cacheEligibleInputTokens + request.cacheEligibleInputTokens, priceVersion: own.priceVersion, + } : request; + } + return sum; +} + +/** A daily snapshot entry read from a file, or undefined when it is not one. */ +export function parseDailySnapshot(value: unknown): DailySessionSnapshot | undefined { + if (!value || typeof value !== 'object') return undefined; + if (!('date' in value) || typeof value.date !== 'string' || !('prompts' in value) || typeof value.prompts !== 'number' + || !('durationMs' in value) || typeof value.durationMs !== 'number') return undefined; + // A snapshot from before costs were kept per day holds them as session fields, + // on the session's day: read them as that day's, as the delta does. + const legacy = (field: string): number => { + const n: unknown = Object.entries(value).find(([k]) => k === field)?.[1]; + return typeof n === 'number' ? n : 0; + }; + const version: unknown = Object.entries(value).find(([k]) => k === 'priceVersion')?.[1]; + const requestDaily = 'requestDaily' in value + ? parseRequestDaily(value.requestDaily) + : legacy('pricedRequests') || legacy('costMicros') || legacy('cacheReadTokens') || legacy('cacheEligibleInputTokens') + ? { [value.date]: { + pricedRequests: legacy('pricedRequests'), costMicros: legacy('costMicros'), cacheReadTokens: legacy('cacheReadTokens'), + cacheEligibleInputTokens: legacy('cacheEligibleInputTokens'), priceVersion: typeof version === 'string' ? version : '', + } } + : {}; + return { + date: value.date, prompts: value.prompts, durationMs: value.durationMs, + succeeded: 'succeeded' in value && value.succeeded === 1 ? 1 : 0, + corrected: 'corrected' in value && value.corrected === 1 ? 1 : 0, + requestDaily, + ...('sessionCacheReadTokens' in value && typeof value.sessionCacheReadTokens === 'number' + ? { sessionCacheReadTokens: value.sessionCacheReadTokens } : {}), + ...('sessionCacheEligibleTokens' in value && typeof value.sessionCacheEligibleTokens === 'number' + ? { sessionCacheEligibleTokens: value.sessionCacheEligibleTokens } : {}), + }; +} + +/** A reported snapshot's request costs per day, from its session-level fields if it predates them. */ +function reportedRequestDaily(previous: DailySessionSnapshot | undefined): Record { + return previous?.requestDaily ?? ( + previous?.pricedRequests || previous?.costMicros || previous?.cacheReadTokens || previous?.cacheEligibleInputTokens + ? { [previous.date]: { + pricedRequests: previous.pricedRequests ?? 0, + costMicros: previous.costMicros ?? 0, + cacheReadTokens: previous.cacheReadTokens ?? 0, + cacheEligibleInputTokens: previous.cacheEligibleInputTokens ?? 0, + priceVersion: previous.priceVersion ?? '', + } } + : {} + ); +} + +/** + * A reported snapshot's session cache tokens. A session reported under the old + * code has no sessionCache* fields but does have requestDaily cache: summed as + * the baseline, so the first post-upgrade report doesn't re-add counted tokens. + */ +function reportedSessionCache( + previous: DailySessionSnapshot | undefined, + requestDaily: Record, +): { read: number; eligible: number } { + return { + read: previous?.sessionCacheReadTokens ?? Object.values(requestDaily).reduce((sum, r) => sum + r.cacheReadTokens, 0), + eligible: previous?.sessionCacheEligibleTokens + ?? Object.values(requestDaily).reduce((sum, r) => sum + r.cacheEligibleInputTokens, 0), + }; +} + +/** + * `run`'s share of `left`, what remains of the snapshot an earlier release kept + * for all the runs of one session ID: each counter up to the run's own. The + * sum's success and correction flags are not any one run's (a successful run + * and an interrupted one sum to unsuccessful), so each run takes its own and + * reports no status change: that release already counted the sum's. + */ +export function takeDailySession( + run: DailySessionSnapshot, + left: DailySessionSnapshot, +): { taken: DailySessionSnapshot; left: DailySessionSnapshot } { + const upTo = (own: number, rest: number) => Math.max(0, Math.min(own, rest)); + const leftDaily = reportedRequestDaily(left); + const leftCache = reportedSessionCache(left, leftDaily); + const takenDaily: Record = {}; + const restDaily: Record = { ...leftDaily }; + for (const [date, request] of Object.entries(run.requestDaily)) { + const rest = leftDaily[date]; + if (!rest) continue; + const taken: RequestCostMetrics = { + pricedRequests: upTo(request.pricedRequests, rest.pricedRequests), + costMicros: upTo(request.costMicros, rest.costMicros), + cacheReadTokens: upTo(request.cacheReadTokens, rest.cacheReadTokens), + cacheEligibleInputTokens: upTo(request.cacheEligibleInputTokens, rest.cacheEligibleInputTokens), + priceVersion: rest.priceVersion, + }; + takenDaily[date] = taken; + restDaily[date] = { + pricedRequests: rest.pricedRequests - taken.pricedRequests, + costMicros: rest.costMicros - taken.costMicros, + cacheReadTokens: rest.cacheReadTokens - taken.cacheReadTokens, + cacheEligibleInputTokens: rest.cacheEligibleInputTokens - taken.cacheEligibleInputTokens, + priceVersion: rest.priceVersion, + }; + } + const prompts = upTo(run.prompts, left.prompts); + const durationMs = upTo(run.durationMs, left.durationMs); + const cacheRead = upTo(run.sessionCacheReadTokens ?? 0, leftCache.read); + const cacheEligible = upTo(run.sessionCacheEligibleTokens ?? 0, leftCache.eligible); + return { + taken: { + date: run.date, prompts, durationMs, succeeded: run.succeeded, corrected: run.corrected, + requestDaily: takenDaily, sessionCacheReadTokens: cacheRead, sessionCacheEligibleTokens: cacheEligible, + }, + left: { + date: left.date, prompts: left.prompts - prompts, durationMs: left.durationMs - durationMs, + succeeded: left.succeeded, corrected: left.corrected, requestDaily: restDaily, + sessionCacheReadTokens: leftCache.read - cacheRead, sessionCacheEligibleTokens: leftCache.eligible - cacheEligible, + }, + }; +} + /** Compute idempotent daily deltas while keeping resumed work on the first Stop day. */ export function computeDailyStatsDelta( current: Map, @@ -121,28 +275,13 @@ export function computeDailyStatsDelta( bucket.sessionsCorrected += positiveDelta(snapshot.corrected, previous?.corrected); delta[date] = bucket; - const previousDaily = previous?.requestDaily ?? ( - previous?.pricedRequests || previous?.costMicros || previous?.cacheReadTokens || previous?.cacheEligibleInputTokens - ? { [previous.date]: { - pricedRequests: previous.pricedRequests ?? 0, - costMicros: previous.costMicros ?? 0, - cacheReadTokens: previous.cacheReadTokens ?? 0, - cacheEligibleInputTokens: previous.cacheEligibleInputTokens ?? 0, - priceVersion: previous.priceVersion ?? '', - } } - : {} - ); + const previousDaily = reportedRequestDaily(previous); // Cache-read share is pricing-independent: fold session-level cache tokens onto // the firstStop day (like prompts/duration), so it works even when the model - // can't be priced. Legacy fallback: a session reported under the old code has no - // sessionCache* fields but does have requestDaily cache — sum it as the baseline - // so this first post-upgrade report doesn't re-add already-counted tokens. - const prevCacheRead = previous?.sessionCacheReadTokens - ?? Object.values(previousDaily).reduce((sum, r) => sum + r.cacheReadTokens, 0); - const prevCacheEligible = previous?.sessionCacheEligibleTokens - ?? Object.values(previousDaily).reduce((sum, r) => sum + r.cacheEligibleInputTokens, 0); - bucket.cacheReadTokens += positiveDelta(snapshot.sessionCacheReadTokens ?? 0, prevCacheRead); - bucket.cacheEligibleInputTokens += positiveDelta(snapshot.sessionCacheEligibleTokens ?? 0, prevCacheEligible); + // can't be priced. + const prevCache = reportedSessionCache(previous, previousDaily); + bucket.cacheReadTokens += positiveDelta(snapshot.sessionCacheReadTokens ?? 0, prevCache.read); + bucket.cacheEligibleInputTokens += positiveDelta(snapshot.sessionCacheEligibleTokens ?? 0, prevCache.eligible); delta[date] = bucket; // Cost stays per-requestDate from pricing; cache no longer flows through here. for (const [requestDate, request] of Object.entries(snapshot.requestDaily)) { diff --git a/src/stats.ts b/src/stats.ts index 8a226ec4a..8e39969c6 100644 --- a/src/stats.ts +++ b/src/stats.ts @@ -7,7 +7,7 @@ import { readEvents, aggregateSessionMetrics } from './dashboard-collector.js'; import { totalTokens, addTokenUsage, emptyTokenUsage } from './types.js'; import { attributeByRepo, timeAnalytics, renderHourSparkline } from './session-analytics.js'; import { formatTokenCount } from './digest.js'; -import type { UsageEvent, UserStats, TokenUsage, SessionMetrics, LocalConfig } from './types.js'; +import type { UsageEvent, UserStats, TokenUsage, SessionMetrics, LocalConfig, DashboardEvent } from './types.js'; interface SkillStats { name: string; @@ -148,25 +148,28 @@ function aggregateDashboardStats(metrics: Map): Aggregat * per-session delta `teamai pull` pushes, so the displayed total is * reported + unreported rather than reported + everything. * - * The caller passes the scope's own metrics, already filtered the way the - * report path filters them. + * The caller passes the scope's own events and their metrics, already + * filtered the way the report path filters them. */ async function unreportedDashboardStats( + events: DashboardEvent[], metrics: Map, config: LocalConfig, ): Promise { const { - computeInterventionDelta, computePromptTokenDelta, readReportedInterventions, readReportedPromptTokens, + computeInterventionDelta, computePromptTokenDelta, droppedRollouts, interventionCounts, metricsAsOf, reportedBaselines, + snapshotWrittenAt, withDroppedRollouts, } = await import('./team-push.js'); - // The scope's own snapshots, the ones its report compares against (#786). - const interventions = await readReportedInterventions(config); - const promptTokens = await readReportedPromptTokens(config); + const { aggregateDailySessions } = await import('./session-trends.js'); + // The scope's own snapshots, compared as its report compares them (#786). + const writtenAt = await snapshotWrittenAt(config); + const currentDaily = aggregateDailySessions(events); + const { promptTokens, interventions, daily } = await reportedBaselines(events, metrics, currentDaily, config, false); + const dropped = droppedRollouts(metrics, promptTokens, interventions, daily, writtenAt, metricsAsOf(events, writtenAt)); + const currentInterventions = withDroppedRollouts(interventionCounts(metrics), currentDaily, dropped).interventions; - const interventionDelta = computeInterventionDelta( - new Map([...metrics].map(([sid, m]) => [sid, { interrupt: m.interrupt, toolReject: m.toolReject, correction: m.correction }])), - interventions, - ); - const promptTokenDelta = computePromptTokenDelta(metrics, promptTokens); + const interventionDelta = computeInterventionDelta(currentInterventions, interventions); + const promptTokenDelta = computePromptTokenDelta(metrics, promptTokens, dropped); return { sessions: interventionDelta.delta.sessions, @@ -229,7 +232,7 @@ export async function showStats(options: ShowStatsOptions = {}): Promise { // events.jsonl until compaction, so counting the full local aggregate would // count each one twice and pull in other projects' sessions). Same filter, // same scope config as the report path (#785). - const { filterEventsByScope } = await import('./team-push.js'); + const { filterEventsByScope } = await import('./dashboard-scope.js'); const scopedEvents = await filterEventsByScope(await readEvents(), config ?? undefined); const metricsMap = aggregateSessionMetrics(scopedEvents); // Only subtract what the team already holds. Two guards, because a scope's @@ -251,7 +254,7 @@ export async function showStats(options: ShowStatsOptions = {}): Promise { || totalTokens(reported.tokens ?? emptyTokenUsage()) > 0 ); const localDashboard = config && teamHasReported - ? await unreportedDashboardStats(metricsMap, config) + ? await unreportedDashboardStats(scopedEvents, metricsMap, config) : aggregateDashboardStats(metricsMap); const dashboard = mergeDashboardAndReported(localDashboard, reported); const hasDashboardData = diff --git a/src/team-push.ts b/src/team-push.ts index 9b003d1e3..5f65065b2 100644 --- a/src/team-push.ts +++ b/src/team-push.ts @@ -1,9 +1,10 @@ import YAML from 'yaml'; import fs from 'node:fs'; +import { createHash } from 'node:crypto'; import path from 'node:path'; import { readUsageEvents, truncateUsageAfterReport } from './usage-tracker.js'; import { aggregateUsage } from './stats.js'; -import { readEvents, aggregateSessionMetrics } from './dashboard-collector.js'; +import { readEvents, aggregateSessionMetrics, dataHomeKey } from './dashboard-collector.js'; import { createGit, pushRepoDirectly, @@ -14,22 +15,28 @@ import { } from './utils/git.js'; import { writeFile, readFileSafe, ensureDir, pathExists, readJson, writeJson } from './utils/fs.js'; import { log } from './utils/logger.js'; -import type { UserStats, UserInterventionStats, SessionMetrics, TokenUsage, DashboardEvent, LocalConfig } from './types.js'; -import { getVotesDir, getDataHome, getTeamaiHomeDir, emptyTokenUsage, addTokenUsage, usesBranchWorktree } from './types.js'; -import { getUserHome } from './utils/home.js'; +import type { + UserStats, UserInterventionStats, SessionMetrics, TokenUsage, DashboardEvent, LocalConfig, RequestCostMetrics, +} from './types.js'; +import { getVotesDir, getDataHome, getReportsDir, emptyTokenUsage, addTokenUsage, usesBranchWorktree } from './types.js'; +import { filterEventsByScope, runsOfLog } from './dashboard-scope.js'; +import { + creditedPrompts, interventionsEntry, promptTokensEntry, readOwnerCredits, recordSessionOwners, reportedBeyondShared, reportedSize, + sharedSnapshotPath, snapshotPathIn, + type ReportedInterventions, type ReportedPromptTokens, type ReportedSegments, type ReportedSnapshotName, +} from './session-owners.js'; import { aggregateDailySessions, computeDailyStatsDelta, mergeDailyStats, + takeDailySession, + parseDailySnapshot, + parseRequestDaily, + addRequestDaily, + type DailySessionSnapshot, type ReportedDailySessions, } from './session-trends.js'; -/** Snapshot of already-reported per-session intervention counts (idempotency basis). */ -type ReportedInterventions = Record; - -/** Snapshot of already-reported per-session prompt counts + token usage (idempotency basis). */ -type ReportedPromptTokens = Record; - /** Cumulative delta for conversation-turn count + token usage (Issue #75). */ interface PromptTokenDelta { prompts: number; @@ -145,35 +152,57 @@ export function mergeStats( // // A session can record events in two scopes (a `cd` mid-session), so each // scope compares against what it reported itself. The first time a scope needs -// a snapshot, it copies the shared one, so nothing an earlier release reported -// is sent again; after that only its own file is read. No scope writes the -// shared file any more, only an earlier release after a rollback (and a caller -// without a scope config, which reads the whole log and reports into it). +// a snapshot, it migrates the shared entries for its retained unkeyed runs; +// path-keyed releases already wrote scope snapshots. After that only its own +// file is read. No scope writes the shared file any more, only an earlier +// release after a rollback (and a caller without a scope config, which reads +// the whole log and reports into it). // -type ReportedSnapshotName = 'interventions' | 'prompt-tokens' | 'daily-sessions'; - -/** The machine-level snapshot every scope shared before #786 (evaluated at call time for tests). */ -function sharedSnapshotPath(name: ReportedSnapshotName): string { - return path.join(getTeamaiHomeDir(), 'dashboard', `reported-${name}.json`); -} - /** A scope's own snapshot. The user scope's data home holds the shared one, hence its prefix. */ function scopeSnapshotPath(name: ReportedSnapshotName, config: LocalConfig | undefined): string { - if (!config) return sharedSnapshotPath(name); - const dataHome = getDataHome(config); - const file = `reported-${name}.json`; - if (path.resolve(dataHome) !== path.resolve(getTeamaiHomeDir())) return path.join(dataHome, 'dashboard', file); - return path.join(dataHome, 'dashboard', `user-${file}`); + return config ? snapshotPathIn(getDataHome(config), name) : sharedSnapshotPath(name); } -/** A scope's snapshot, seeded from the shared one when the scope has none yet. */ -async function readSnapshot(name: ReportedSnapshotName, config: LocalConfig | undefined): Promise { + +/** + * A scope's snapshot, seeded from the shared one when the scope has none yet. + * The shared file summed every scope's runs of an ID, so they consume it in + * the order of the whole log, and the scope keeps its own runs' shares. + * `split` gives each run of the log its entry at its own totals and its share + * of a bare entry, for {@link adoptBareKeys}. + */ +async function readSnapshot( + name: ReportedSnapshotName, + config: LocalConfig | undefined, + split: (events: DashboardEvent[]) => Promise<{ current: Record; take: TakeReported }>, + persist = true, +): Promise | null> { const own = scopeSnapshotPath(name, config); - if (!config || await pathExists(own)) return readJson(own); - const seed = await readJson(sharedSnapshotPath(name)); + if (!config || await pathExists(own)) return readJson>(own); + const shared = await readJson>(sharedSnapshotPath(name)); + const logged = await readEvents(); + const all = await runsOfLog(logged); + const events = await filterEventsByScope(logged, config); + const { current, take } = await split(all); + const adopted = adoptBareKeys(shared ?? {}, all, current, take, 'shared'); + // A tool's own session ID is one session, so its entry is copied whole, as + // before: resumed after compaction dropped its events, it is not sent again. + // A fallback entry is copied only as a run of this scope: an unmatched bare + // one cannot be allowed to attach to a future reuse of its PID. + const ownRuns = new Set(events.map((e) => e.sessionId)); + const seed: Record = {}; + for (const [id, entry] of Object.entries(adopted)) { + if (!id.startsWith('pid-') || ownRuns.has(id)) seed[id] = entry; + } + // A read-only caller (`teamai stats`) leaves the seed to the report. + if (!persist) return seed; try { - await writeJson(own, seed ?? {}); + await writeJson(own, seed); + // The seed covers what the shared file had by its last write, and a later + // report reads that from this file's time (see droppedRollouts). + const sharedTime = await fs.promises.stat(sharedSnapshotPath(name)).then((stat) => stat.mtime, () => undefined); + if (sharedTime) await fs.promises.utimes(own, sharedTime, sharedTime); } catch (e) { // Seeded again next time: the shared file is not written any more. log.debug(`Could not seed ${own}: ${(e as Error).message}`); @@ -181,8 +210,11 @@ async function readSnapshot(name: ReportedSnapshotName, config: LocalConfig | return seed; } -export async function readReportedInterventions(config: LocalConfig | undefined): Promise { - const parsed = await readSnapshot('interventions', config); +export async function readReportedInterventions(config: LocalConfig | undefined, persist = true): Promise { + const parsed = await readSnapshot('interventions', config, async (events) => ({ + current: Object.fromEntries(interventionCounts(aggregateSessionMetrics(events))), + take: takeInterventions(await sharedCoverage(events)), + }), persist); return parsed && typeof parsed === 'object' ? parsed : {}; } @@ -194,6 +226,73 @@ async function writeReportedInterventions(data: ReportedInterventions, config: L } } +/** Each session's intervention counts, the shape its snapshot entry holds. */ +export function interventionCounts(metrics: Map): Map { + return new Map([...metrics].map(([sid, m]) => [sid, { interrupt: m.interrupt, toolReject: m.toolReject, correction: m.correction }])); +} + +/** A counter's share of what is left: up to the run's own. */ +function upTo(own: number, left: number): number { + return Math.max(0, Math.min(own, left)); +} + +/** + * A run's prompt and token share of a bare entry. This snapshot decides which + * runs of the ID that release reported: the ones its prompts and tokens reach. + */ +const takePromptTokens: TakeReported = (run, entry) => { + // Read from a snapshot file an earlier release wrote, so parsed, not trusted. + const left = promptTokensEntry(entry); + if (left.prompts <= 0 && !hasPromptTokenDelta({ prompts: 0, tokens: left.tokens })) return undefined; + const taken = { + prompts: upTo(run.prompts, left.prompts), + tokens: { + input: upTo(run.tokens.input, left.tokens.input), + output: upTo(run.tokens.output, left.tokens.output), + cacheRead: upTo(run.tokens.cacheRead, left.tokens.cacheRead), + cacheCreation: upTo(run.tokens.cacheCreation, left.tokens.cacheCreation), + }, + }; + return { taken, left: { prompts: left.prompts - taken.prompts, tokens: tokenDelta(left.tokens, taken.tokens) } }; +}; + +/** + * A run's intervention share, for the runs `covered`, the prompt-token + * snapshot's: these counts are mostly zero, so running out says nothing. + */ +function takeInterventions(covered: (runId: string) => boolean): TakeReported { + return (run, entry, runId) => { + if (!covered(runId)) return undefined; + const left = interventionsEntry(entry); + const taken = { + interrupt: upTo(run.interrupt, left.interrupt), + toolReject: upTo(run.toolReject, left.toolReject), + correction: upTo(run.correction, left.correction), + }; + return { + taken, + left: { + interrupt: left.interrupt - taken.interrupt, + toolReject: left.toolReject - taken.toolReject, + correction: left.correction - taken.correction, + }, + }; + }; +} + +/** The runs of the whole log (`all`) the shared prompt-token snapshot covers, for seeding. */ +async function sharedCoverage(all: DashboardEvent[]): Promise<(runId: string) => boolean> { + const shared = await readJson(sharedSnapshotPath('prompt-tokens')); + const current = computePromptTokenDelta(aggregateSessionMetrics(all), {}).nextReported; + const covered = adoptBareKeys(shared ?? {}, all, current, takePromptTokens, 'shared'); + return (runId) => Object.hasOwn(covered, runId); +} + +/** A run's daily share, for the runs `covered`, as for interventions. */ +function takeDaily(covered: (runId: string) => boolean): TakeReported { + return (run, left, runId) => (covered(runId) ? takeDailySession(run, left) : undefined); +} + /** * Compute the intervention delta to report: for each current session, the positive * change since it was last reported. A session not seen before contributes +1 to @@ -247,8 +346,11 @@ function hasInterventionDelta(d: UserInterventionStats): boolean { // Separate snapshot from interventions so each metric stays independently idempotent. // -export async function readReportedPromptTokens(config: LocalConfig | undefined): Promise { - const parsed = await readSnapshot('prompt-tokens', config); +export async function readReportedPromptTokens(config: LocalConfig | undefined, persist = true): Promise { + const parsed = await readSnapshot('prompt-tokens', config, async (events) => ({ + current: computePromptTokenDelta(aggregateSessionMetrics(events), {}).nextReported, + take: takePromptTokens, + }), persist); return parsed && typeof parsed === 'object' ? parsed : {}; } @@ -270,29 +372,209 @@ function tokenDelta(cur: TokenUsage, prev: TokenUsage | undefined): TokenUsage { }; } +/** A rollout's key in a snapshot: its transcript path, hashed. */ +function segmentKey(transcript: string): string { + return createHash('sha256').update(transcript).digest('hex').slice(0, 16); +} + +/** The reported rollouts of a snapshot entry read from a file; empty when it holds none. */ +function reportedSegments(entry: unknown): ReportedSegments { + const segments: ReportedSegments = {}; + if (!entry || typeof entry !== 'object' || !('segments' in entry) || !entry.segments || typeof entry.segments !== 'object') { + return segments; + } + for (const [key, segment] of Object.entries(entry.segments)) { + const iv = interventionsEntry(segment); + const durationMs: unknown = segment && typeof segment === 'object' && 'durationMs' in segment ? segment.durationMs : 0; + segments[key] = { + ...promptTokensEntry(segment), interrupt: iv.interrupt, toolReject: iv.toolReject, correction: iv.correction, + durationMs: typeof durationMs === 'number' ? durationMs : 0, + requestDaily: parseRequestDaily(segment && typeof segment === 'object' && 'requestDaily' in segment ? segment.requestDaily : undefined), + failed: !!segment && typeof segment === 'object' && 'failed' in segment && segment.failed === true, + }; + } + return segments; +} + +/** A reported rollout compaction has dropped: its key and the totals it was reported with. */ +export interface DroppedRollout { + key: string; prompts: number; tokens: TokenUsage; interrupt: number; toolReject: number; correction: number; + durationMs: number; requestDaily: Record; failed: boolean; +} + +/** + * The rollouts of each transcript-scoped session (Codex) that were reported and + * that compaction has since dropped. A rollout's counters restart, so the + * session's totals must keep what those were reported with, or a later rollout + * is compared against them and reported late or never. + * + * An entry written before rollouts were kept is one total. An earlier release + * rewrote every session in the log on each report, so that total covers the + * rollouts that had begun by `writtenAt`, when its file was last written, as + * far as each had got by then (`before`: the metrics of the events up to + * `writtenAt`): those still in the log consume it in order, and what is left + * is the dropped ones', kept as one prior rollout. A rollout begun after it is + * new, as is what one begun before it has done since. Without `writtenAt`, + * every rollout in the log is taken as covered, as it is now. + */ +export function droppedRollouts( + current: Map, + promptTokens: ReportedPromptTokens, + interventions: ReportedInterventions, + daily: ReportedDailySessions, + writtenAt?: number, + before?: Map, +): Map { + const dropped = new Map(); + for (const [sid, cur] of current) { + if (!cur.segments) continue; + const prev = promptTokens[sid]; + if (prev === undefined) continue; + const previous = reportedSegments(prev); + const present = new Set(Object.keys(cur.segments).map(segmentKey)); + const gone: DroppedRollout[] = []; + if (Object.keys(previous).length > 0) { + for (const [key, segment] of Object.entries(previous)) { + if (present.has(key)) continue; + gone.push({ + key, prompts: segment.prompts, tokens: segment.tokens, interrupt: segment.interrupt ?? 0, + toolReject: segment.toolReject ?? 0, correction: segment.correction ?? 0, durationMs: segment.durationMs ?? 0, + requestDaily: segment.requestDaily ?? {}, failed: segment.failed ?? false, + }); + } + } else { + const total = promptTokensEntry(prev); + const iv = interventionsEntry(interventions[sid]); + const day = parseDailySnapshot(daily[sid]); + const left = { + prompts: total.prompts, tokens: { ...total.tokens }, interrupt: iv.interrupt, toolReject: iv.toolReject, + correction: iv.correction, durationMs: day?.durationMs ?? 0, requestDaily: { ...(day?.requestDaily ?? {}) }, + }; + const covered = Object.values((writtenAt !== undefined && before?.get(sid)?.segments) || cur.segments) + .filter((segment) => writtenAt === undefined || Date.parse(segment.since) <= writtenAt) + .sort((a, b) => Date.parse(a.since) - Date.parse(b.since)); + const take = (own: number, rest: number) => Math.max(0, Math.min(own, rest)); + for (const segment of covered) { + for (const field of ['prompts', 'interrupt', 'toolReject', 'correction', 'durationMs'] as const) { + left[field] -= take(segment[field], left[field]); + } + for (const field of ['input', 'output', 'cacheRead', 'cacheCreation'] as const) { + left.tokens[field] -= take(segment.tokens[field], left.tokens[field]); + } + for (const [date, request] of Object.entries(segment.requestDaily)) { + const rest = left.requestDaily[date]; + if (!rest) continue; + left.requestDaily[date] = { + ...rest, + pricedRequests: rest.pricedRequests - take(request.pricedRequests, rest.pricedRequests), + costMicros: rest.costMicros - take(request.costMicros, rest.costMicros), + cacheReadTokens: rest.cacheReadTokens - take(request.cacheReadTokens, rest.cacheReadTokens), + cacheEligibleInputTokens: rest.cacheEligibleInputTokens - take(request.cacheEligibleInputTokens, rest.cacheEligibleInputTokens), + }; + } + } + const anyLeft = left.prompts > 0 || left.interrupt > 0 || left.toolReject > 0 || left.correction > 0 || left.durationMs > 0 + || hasPromptTokenDelta({ prompts: 0, tokens: left.tokens }) + || Object.values(left.requestDaily).some((r) => r.pricedRequests > 0 || r.costMicros > 0); + // What that release counted as the session's outcome stays with its part. + if (anyLeft) gone.push({ key: 'prior', ...left, failed: day?.succeeded === 0 }); + } + // A thread-level counter already holds every rollout's tokens. + if (cur.tokensSpanRollouts) for (const rollout of gone) rollout.tokens = emptyTokenUsage(); + if (gone.length > 0) dropped.set(sid, gone); + } + return dropped; +} + /** * Compute the prompt-count + token delta to report: for each current session, the * positive change since it was last reported. Idempotent (a re-run reports nothing * new), and never negative if a snapshot shrinks. The next snapshot keeps only * sessions still present in events.jsonl (compacted sessions stay folded into totals). + * + * A transcript-scoped session (Codex) is reported per rollout: each rollout's + * counters restart, and the session's total sums those still in the log plus + * the `dropped` ones (see {@link droppedRollouts}), which the next snapshot + * keeps, so a later rollout of the session is reported in full. */ export function computePromptTokenDelta( current: Map, reported: ReportedPromptTokens, + dropped: Map = droppedRollouts(current, reported, {}, {}), ): { delta: PromptTokenDelta; nextReported: ReportedPromptTokens } { const delta: PromptTokenDelta = { prompts: 0, tokens: emptyTokenUsage() }; const nextReported: ReportedPromptTokens = {}; for (const [sid, cur] of current) { const prev = reported[sid]; - delta.prompts += Math.max(0, cur.prompts - (prev?.prompts ?? 0)); - delta.tokens = addTokenUsage(delta.tokens, tokenDelta(cur.tokens, prev?.tokens)); - nextReported[sid] = { prompts: cur.prompts, tokens: cur.tokens }; + let prompts = cur.prompts; + let tokens = cur.tokens; + let segments: ReportedSegments | undefined; + if (cur.segments) { + segments = {}; + for (const [transcript, segment] of Object.entries(cur.segments)) { + segments[segmentKey(transcript)] = { + prompts: segment.prompts, tokens: { ...segment.tokens }, interrupt: segment.interrupt, toolReject: segment.toolReject, + correction: segment.correction, durationMs: segment.durationMs, requestDaily: segment.requestDaily, + failed: segment.error || segment.interrupt > 0 || segment.correction > 0, + }; + } + for (const gone of dropped.get(sid) ?? []) { + const { key, ...totals } = gone; + segments[key] = totals; + prompts += gone.prompts; + tokens = addTokenUsage(tokens, gone.tokens); + } + } + delta.prompts += Math.max(0, prompts - (prev?.prompts ?? 0)); + delta.tokens = addTokenUsage(delta.tokens, tokenDelta(tokens, prev?.tokens)); + nextReported[sid] = { prompts, tokens, ...(segments ? { segments } : {}) }; } return { delta, nextReported }; } +/** + * The intervention counts and daily snapshots a report compares, with each + * session's dropped rollouts added (see {@link droppedRollouts}): Stop counts + * and prompts restart per rollout like the tokens do. + */ +export function withDroppedRollouts( + interventions: Map, + daily: Map, + dropped: Map, +): { interventions: Map; daily: Map } { + const nextInterventions = new Map(interventions); + const nextDaily = new Map(daily); + for (const [sid, gone] of dropped) { + const sum = (field: 'prompts' | 'interrupt' | 'toolReject' | 'correction' | 'durationMs') => + gone.reduce((total, rollout) => total + rollout[field], 0); + const tokens = gone.reduce((total, rollout) => addTokenUsage(total, rollout.tokens), emptyTokenUsage()); + const counts = nextInterventions.get(sid); + if (counts) { + nextInterventions.set(sid, { + interrupt: counts.interrupt + sum('interrupt'), toolReject: counts.toolReject + sum('toolReject'), + correction: counts.correction + sum('correction'), + }); + } + const day = nextDaily.get(sid); + if (day) { + nextDaily.set(sid, { + ...day, + prompts: day.prompts + sum('prompts'), + durationMs: day.durationMs + sum('durationMs'), + sessionCacheReadTokens: (day.sessionCacheReadTokens ?? 0) + tokens.cacheRead, + sessionCacheEligibleTokens: (day.sessionCacheEligibleTokens ?? 0) + tokens.input + tokens.cacheRead + tokens.cacheCreation, + requestDaily: gone.reduce((total, rollout) => addRequestDaily(total, rollout.requestDaily), day.requestDaily), + // A session one of whose rollouts failed or was corrected did not succeed. + succeeded: gone.some((rollout) => rollout.failed) ? 0 : day.succeeded, + corrected: gone.some((rollout) => rollout.correction > 0) ? 1 : day.corrected, + }); + } + } + return { interventions: nextInterventions, daily: nextDaily }; +} + /** Accumulate a prompt/token delta onto the user's existing totals. */ export function mergePromptTokenStats( existingPrompts: number | undefined, @@ -311,8 +593,11 @@ function hasPromptTokenDelta(d: PromptTokenDelta): boolean { || d.tokens.cacheRead > 0 || d.tokens.cacheCreation > 0; } -async function readReportedDailySessions(config: LocalConfig | undefined): Promise { - return (await readSnapshot('daily-sessions', config)) ?? {}; +async function readReportedDailySessions(config: LocalConfig | undefined, persist = true): Promise { + return (await readSnapshot('daily-sessions', config, async (events) => ({ + current: computeDailyStatsDelta(aggregateDailySessions(events), {}).nextReported, + take: takeDaily(await sharedCoverage(events)), + }), persist)) ?? {}; } async function writeReportedDailySessions(data: ReportedDailySessions, config: LocalConfig | undefined): Promise { @@ -331,90 +616,271 @@ function hasDailyDelta(delta: ReturnType['delta'] } /** - * A Windows-style root: a drive letter (`C:\`, `c:/`) or a UNC share - * (`\\server\share`). Tested per path rather than per platform, because the - * dashboard event log is shared — a team repo can hold events pushed from - * Windows and from Linux in the same file. + * A session main split across scopes, crediting every part it reported. Main + * (#795) gave each event to the scope its `dataHome` names and reported each + * part against that scope's own snapshot, so a scope's entry may hold only its + * part (prompts counted before any Stop carried the transcript's total). The + * scope that owns the whole run now takes as reported the metrics of the union + * of those parts: for each scope, the shortest prefix of its events whose own + * metrics reach its snapshot. Only an entry that union exceeds is replaced, so a + * run already reported past it is left alone; runs whose events name at most + * one data home are untouched. */ -const WINDOWS_ROOT = /^(?:[A-Za-z]:[\\/]|\\\\)/; - -type ScopeRoot = { key: string; windows: boolean }; +async function creditSplitRuns( + events: DashboardEvent[], + reported: { interventions: ReportedInterventions; promptTokens: ReportedPromptTokens; daily: ReportedDailySessions }, +): Promise<{ + interventions: ReportedInterventions; promptTokens: ReportedPromptTokens; daily: ReportedDailySessions; changed: boolean; +}> { + const byRun = new Map(); + for (const e of events) byRun.set(e.sessionId, [...(byRun.get(e.sessionId) ?? []), e]); + const result = { + interventions: { ...reported.interventions }, promptTokens: { ...reported.promptTokens }, daily: { ...reported.daily }, + changed: false, + }; + const credits = await readOwnerCredits(); + for (const [runId, runEvents] of byRun) { + const homes = new Set(runEvents.flatMap((e) => (typeof e.dataHome === 'string' ? [e.dataHome] : []))); + // Its parts' events are gone: the credit the owners file seeded from their snapshots. + const credit = homes.size < 2 ? credits.get(runId) : undefined; + if (credit) { + // Placed by the transcript where it can tell what the parts overlapped. + const transcripts = runEvents.flatMap((e) => (typeof e.transcriptPath === 'string' ? [e.transcriptPath] : [])); + const prompts = (await creditedPrompts(credit, transcripts)) ?? credit.promptTokens.prompts; + result.changed = raiseToCredit(result, runId, { + promptTokens: { ...credit.promptTokens, prompts }, + interventions: credit.interventions, + daily: credit.daily ? { ...credit.daily, prompts } : undefined, + }) || result.changed; + continue; + } + if (homes.size < 2) continue; + // Main keyed its snapshots by the session ID, before runs had their own. + const id = runId.startsWith('pid-') ? runId.slice(0, runId.lastIndexOf('@')) : runId; + const counted = new Set(); + for (const home of homes) { + const snapshot = await readJson>(snapshotPathIn(home, 'prompt-tokens')); + const target = reportedSize(snapshot?.[id]); + if (!snapshot || !Object.hasOwn(snapshot, id)) continue; + const part = runEvents.filter((e) => e.dataHome === home); + for (let k = 1; k <= part.length; k++) { + const size = aggregateSessionMetrics(part.slice(0, k)).get(runId); + const tokens = size ? size.tokens.input + size.tokens.output + size.tokens.cacheRead + size.tokens.cacheCreation : 0; + if (k === part.length || (size && size.prompts >= target.prompts && tokens >= target.tokens)) { + for (const e of part.slice(0, k)) counted.add(e); + break; + } + } + } + const union = runEvents.filter((e) => counted.has(e)); + if (union.length === 0) continue; + const metrics = aggregateSessionMetrics(union); + const promptTokens = computePromptTokenDelta(metrics, {}).nextReported[runId]; + if (!promptTokens) continue; + result.changed = raiseToCredit(result, runId, { + promptTokens, + interventions: interventionCounts(metrics).get(runId), + daily: computeDailyStatsDelta(aggregateDailySessions(union), {}).nextReported[runId], + }) || result.changed; + } + return result; +} /** - * Normalize a directory path for scope comparison. - * - * Both sides are native paths: `projectRoot` is stored as `path.resolve(cwd)` - * at init time, and an event's `cwd` is whatever the AI tool put in its hook - * payload. On Windows both use backslashes, so a literal `root + '/'` prefix - * can never match a subdirectory, and the two sources can also disagree on the - * case of the drive letter or of any directory along the way. Windows paths - * therefore get their separators unified and their case folded. - * - * POSIX paths keep both distinctions: they are case-sensitive, and `\` is a - * legal character in a POSIX filename, so `/work/a\b` and `/work/a/b` are two - * different directories and must not collapse onto one key. - * - * The root decides which set of rules applies to both sides, so a Windows root - * still matches a cwd the tool reported with forward slashes, and a POSIX root - * never has a backslash rewritten underneath it. + * Raises a run's baselines to at least `credit`, counter by counter: a part + * may have reported more time, tokens or costs with no more prompts. Flags + * and dates stay the run's own where it has an entry. Whether anything moved. */ -function scopeKey(dir: string, windows: boolean): string { - if (!windows) return dir.replace(/\/+$/, ''); - return dir.replace(/\\/g, '/').replace(/\/+$/, '').toLowerCase(); +function raiseToCredit( + result: { interventions: ReportedInterventions; promptTokens: ReportedPromptTokens; daily: ReportedDailySessions }, + runId: string, + credit: { + promptTokens: ReportedPromptTokens[string]; + interventions: ReportedInterventions[string] | undefined; + daily: DailySessionSnapshot | undefined; + }, +): boolean { + const before = JSON.stringify([result.promptTokens[runId], result.interventions[runId], result.daily[runId]]); + const own = result.promptTokens[runId]; + const ownTokens = promptTokensEntry(own); + const greater = (a: TokenUsage, b: TokenUsage): TokenUsage => ({ + input: Math.max(a.input, b.input), output: Math.max(a.output, b.output), + cacheRead: Math.max(a.cacheRead, b.cacheRead), cacheCreation: Math.max(a.cacheCreation, b.cacheCreation), + }); + result.promptTokens[runId] = { + ...(own ?? {}), + prompts: Math.max(ownTokens.prompts, credit.promptTokens.prompts), + tokens: greater(ownTokens.tokens, credit.promptTokens.tokens), + }; + if (credit.interventions) { + const iv = interventionsEntry(result.interventions[runId]); + result.interventions[runId] = { + interrupt: Math.max(iv.interrupt, credit.interventions.interrupt), + toolReject: Math.max(iv.toolReject, credit.interventions.toolReject), + correction: Math.max(iv.correction, credit.interventions.correction), + }; + } + if (credit.daily) { + const day = parseDailySnapshot(result.daily[runId]); + if (!day) { + result.daily[runId] = credit.daily; + } else { + const requestDaily = { ...day.requestDaily }; + for (const [date, request] of Object.entries(credit.daily.requestDaily)) { + const mine = requestDaily[date]; + requestDaily[date] = mine ? { + ...mine, + pricedRequests: Math.max(mine.pricedRequests, request.pricedRequests), costMicros: Math.max(mine.costMicros, request.costMicros), + cacheReadTokens: Math.max(mine.cacheReadTokens, request.cacheReadTokens), + cacheEligibleInputTokens: Math.max(mine.cacheEligibleInputTokens, request.cacheEligibleInputTokens), + } : request; + } + result.daily[runId] = { + ...day, + prompts: Math.max(day.prompts, credit.daily.prompts), + durationMs: Math.max(day.durationMs, credit.daily.durationMs), + sessionCacheReadTokens: Math.max(day.sessionCacheReadTokens ?? 0, credit.daily.sessionCacheReadTokens ?? 0), + sessionCacheEligibleTokens: Math.max(day.sessionCacheEligibleTokens ?? 0, credit.daily.sessionCacheEligibleTokens ?? 0), + requestDaily, + }; + } + } + return JSON.stringify([result.promptTokens[runId], result.interventions[runId], result.daily[runId]]) !== before; } -function scopeRoot(dir: string): ScopeRoot { - const windows = WINDOWS_ROOT.test(dir); - return { key: scopeKey(dir, windows), windows }; +/** The metrics of `events` up to `at`, what an entry written then saw; undefined without `at`. */ +export function metricsAsOf(events: DashboardEvent[], at: number | undefined): Map | undefined { + return at === undefined ? undefined : aggregateSessionMetrics(events.filter((e) => Date.parse(e.timestamp) <= at)); } -/** True when `cwd` is the root itself or sits below it. */ -function isUnderScopeRoot(cwd: string, root: ScopeRoot): boolean { - const key = scopeKey(cwd, root.windows); - return key === root.key || key.startsWith(root.key + '/'); +/** + * When the scope's prompt-token snapshot was last written, before this report + * writes it: its own file's, else the shared one it would be seeded from. An + * entry an earlier release left covers what that release had seen by then. + */ +export async function snapshotWrittenAt(config: LocalConfig | undefined): Promise { + const mtime = (file: string) => fs.promises.stat(file).then((stat) => stat.mtimeMs, () => undefined); + const own = await mtime(scopeSnapshotPath('prompt-tokens', config)); + if (own === undefined) return mtime(sharedSnapshotPath('prompt-tokens')); + // An earlier release wrote the snapshot after its push, and the team stats + // file in this scope's reports checkout before it, after reading the log: the + // earlier of the two is nearer what that report had read. + const stats = config ? await mtime(path.join(getReportsDir(config), 'stats', `${config.username}.yaml`)) : undefined; + return stats === undefined ? own : Math.min(own, stats); } /** - * The dashboard events a scope reports (#785): those recorded in it, keyed by - * its data home. A project also owns the key of its in-repo `.teamai`, where a - * hook recorded until migration moved the project to a partition. An event - * written before events carried a data home belongs to the project whose root holds - * its cwd, never to the user scope. `projectRoot` is realpath'd, but a cwd is - * raw as the host sent it (a symlinked checkout, macOS `/tmp` vs - * `/private/tmp`) and so is a non-git project's data home, so both are - * realpath'd while they still exist. A caller without a scope config reads the - * whole log. + * A scope's reported snapshots as its report and `teamai stats` compare them: + * bare entries adopted by the runs of `events` (prompt tokens first, since they + * decide which runs of a bare ID were reported), and a session an earlier + * release split across scopes credited with every part. With `persist`, what + * changed is written out at once, even with nothing to report, so the report's + * success writes, which merge into the files, cannot bring a retired entry back. */ -export async function filterEventsByScope( +export async function reportedBaselines( events: DashboardEvent[], - config?: LocalConfig, -): Promise { - if (!config) return events; - const realPaths = new Map>(); - const realPath = (dir: string): Promise => { - let real = realPaths.get(dir); - if (!real) { - real = fs.promises.realpath(dir).catch(() => dir); - realPaths.set(dir, real); - } - return real; + metrics: Map, + currentDaily: Map, + config: LocalConfig | undefined, + persist: boolean, +): Promise<{ interventions: ReportedInterventions; promptTokens: ReportedPromptTokens; daily: ReportedDailySessions }> { + const adopt = async ( + read: (config: LocalConfig | undefined, persist: boolean) => Promise>, + write: (data: Record, config: LocalConfig | undefined) => Promise, + current: Record, + take: TakeReported, + ): Promise> => { + const stored = await read(config, persist); + const adopted = adoptBareKeys(stored, events, current, take); + if (persist && adopted !== stored) await write(adopted, config); + return adopted; }; - const keyOf = async (home: string): Promise => scopeRoot(await realPath(home)); - const homeRoots = [await keyOf(getDataHome(config))]; - if (config.projectRoot) { - // Unless the project is rooted at HOME, where that is the user scope's. - const legacy = await keyOf(path.join(config.projectRoot, '.teamai')); - if (legacy.key !== (await keyOf(path.join(getUserHome(), '.teamai'))).key) homeRoots.push(legacy); + const promptTokens = await adopt( + readReportedPromptTokens, writeReportedPromptTokens, computePromptTokenDelta(metrics, {}).nextReported, takePromptTokens, + ); + const covered = (runId: string) => Object.hasOwn(promptTokens, runId); + const interventions = await adopt( + readReportedInterventions, writeReportedInterventions, Object.fromEntries(interventionCounts(metrics)), + takeInterventions(covered), + ); + const daily = await adopt( + readReportedDailySessions, writeReportedDailySessions, computeDailyStatsDelta(currentDaily, {}).nextReported, + takeDaily(covered), + ); + const credited = await creditSplitRuns(events, { interventions, promptTokens, daily }); + if (persist && credited.changed) { + await writeReportedInterventions(credited.interventions, config); + await writeReportedPromptTokens(credited.promptTokens, config); + await writeReportedDailySessions(credited.daily, config); } - const root = config.projectRoot ? scopeRoot(config.projectRoot) : undefined; - const kept = await Promise.all(events.map(async (e) => { - if (e.dataHome !== undefined) { - const dataHome = await realPath(e.dataHome); - return homeRoots.some((home) => scopeKey(dataHome, home.windows) === home.key); + return { interventions: credited.interventions, promptTokens: credited.promptTokens, daily: credited.daily }; +} + +/** + * A run's share of what is left of a bare snapshot entry, or undefined when + * nothing is left: that run and every later one of its ID were not reported. + */ +export type TakeReported = (run: T, left: T, runId: string) => { taken: T; left: T } | undefined; + +/** + * A reported snapshot as the run IDs of {@link filterEventsByScope} read it. + * Snapshots written before fallback runs had their own IDs are keyed by the + * bare `pid-…` ID, holding the sum of the runs of that ID in the log at that + * release's last report; compaction keeps or drops the runs of an ID together. + * So the runs of that ID in `events` (the scope's, as that filter returns them) + * consume the entry in order, each taking its share, from its totals in + * `current`, of what is left; once nothing is left, the later runs take none + * and are reported as new. The first run always takes one: the entry means + * that release reported it. A run's own entry is never replaced, and the bare + * entry is retired, so no later run of that ID reads it. Only an earlier + * release wrote bare entries, and only for runs it recorded, so a run whose + * first event carries a `dataHomeKey`, and every later run of its ID, takes + * none: the entry may be another scope's run under a reused PID-fallback ID. + * Shared snapshots also exclude path-keyed runs: that release already had + * per-scope snapshots. A tool's own session ID keys its one run as it is, so + * its entry needs no adoption. Returns `reported` itself when there is nothing + * to retire; otherwise the caller persists the result. + */ +export function adoptBareKeys( + reported: Record, + events: Iterable, + current: Record, + take: TakeReported, + source: 'scope' | 'shared' = 'scope', +): Record { + const legacyRuns = new Map(); + const recorded = new Set(); + const seen = new Set(); + for (const { sessionId: runId, dataHomeKey, dataHome } of events) { + if (seen.has(runId) || !runId.startsWith('pid-')) continue; + seen.add(runId); + const id = runId.slice(0, runId.lastIndexOf('@')); + if (recorded.has(id) || !Object.hasOwn(reported, id)) continue; + if (typeof dataHomeKey === 'string' || (source === 'shared' && typeof dataHome === 'string')) { + recorded.add(id); + continue; + } + legacyRuns.set(id, [...(legacyRuns.get(id) ?? []), runId]); + } + if (legacyRuns.size === 0) return reported; + const adopted = { ...reported }; + for (const [id, runs] of legacyRuns) { + let left = reported[id]; + let first = true; + for (const runId of runs) { + if (!Object.hasOwn(current, runId)) continue; + const share = take(current[runId], left, runId); + if (!share) { + if (first && !Object.hasOwn(adopted, runId)) adopted[runId] = left; + break; + } + if (!Object.hasOwn(adopted, runId)) adopted[runId] = share.taken; + left = share.left; + first = false; } - return !!root && !!e.cwd && isUnderScopeRoot(await realPath(e.cwd), root); - })); - return events.filter((_, i) => kept[i]); + delete adopted[id]; + } + return adopted; } /** @@ -456,25 +922,37 @@ export async function reportUsageToTeam( const dashboardEvents = await filterEventsByScope(await readEvents(), reportsConfig); const metrics = aggregateSessionMetrics(dashboardEvents); - const currentInterventions = new Map( - [...metrics].map(([sid, m]) => [sid, { interrupt: m.interrupt, toolReject: m.toolReject, correction: m.correction }]), + const currentInterventions = interventionCounts(metrics); + const currentDaily = aggregateDailySessions(dashboardEvents); + const writtenAt = await snapshotWrittenAt(reportsConfig); + const { + interventions: reportedInterventions, promptTokens: reportedPromptTokens, daily: reportedDailySessions, + } = await reportedBaselines(dashboardEvents, metrics, currentDaily, reportsConfig, true); + const dropped = droppedRollouts( + metrics, reportedPromptTokens, reportedInterventions, reportedDailySessions, writtenAt, metricsAsOf(dashboardEvents, writtenAt), ); - const reportedInterventions = await readReportedInterventions(reportsConfig); - const { delta: interventionDelta, nextReported } = computeInterventionDelta( - currentInterventions, - reportedInterventions, - ); - - const reportedPromptTokens = await readReportedPromptTokens(reportsConfig); + const effective = withDroppedRollouts(currentInterventions, currentDaily, dropped); const { delta: promptTokenDelta, nextReported: nextReportedPromptTokens } = computePromptTokenDelta( metrics, reportedPromptTokens, + dropped, + ); + const { delta: interventionDelta, nextReported } = computeInterventionDelta( + effective.interventions, + reportedInterventions, ); - const reportedDailySessions = await readReportedDailySessions(reportsConfig); const { delta: dailyDelta, nextReported: nextReportedDailySessions } = computeDailyStatsDelta( - aggregateDailySessions(dashboardEvents), + effective.daily, reportedDailySessions, ); + // This scope's sessions, and those of its snapshots no owner claims yet + // (a project whose data home is in its workspace, which the seed may miss). + if (reportsConfig) { + await recordSessionOwners([ + ...dashboardEvents.map((e) => e.sessionId), + ...await reportedBeyondShared(reportedPromptTokens, reportedInterventions, reportedDailySessions), + ], await dataHomeKey(getDataHome(reportsConfig))); + } const hasUsage = events.length > 0; const hasInterventions = hasInterventionDelta(interventionDelta); @@ -630,7 +1108,8 @@ export async function reportUsageToTeam( await writeReportedInterventions({ ...existingIv, ...nextReported }, reportsConfig); log.debug(`Reported intervention delta (${interventionDelta.sessions} new sessions) to team repo`); } - if (hasPromptTokens) { + // It also holds each rollout's totals, which any delta may have moved. + if (hasPromptTokens || hasInterventions || hasDaily) { const existingPt = await readReportedPromptTokens(reportsConfig); await writeReportedPromptTokens({ ...existingPt, ...nextReportedPromptTokens }, reportsConfig); log.debug(`Reported prompt/token delta (${promptTokenDelta.prompts} prompts) to team repo`); diff --git a/src/types.ts b/src/types.ts index da4f2e787..c24d47bd3 100644 --- a/src/types.ts +++ b/src/types.ts @@ -1214,6 +1214,19 @@ export interface SessionMetrics { prompts: number; /** Cumulative token usage across the logical session. */ tokens: TokenUsage; + /** + * Each rollout's own totals for a transcript-scoped session (Codex), keyed by + * its transcript path: a rollout's counters restart, so the session sums them. + * `since` is the rollout's first event. + */ + /** Its tokens come from one counter that spans its rollouts (Codex's thread-level counter). */ + tokensSpanRollouts?: true; + segments?: Record; since: string; + /** An event of the rollout ended in an error. */ + error: boolean; + }>; } export type DashboardSessionStatus = 'running' | 'waiting_for_input' | 'error' | 'idle' | 'stopped'; @@ -1232,9 +1245,14 @@ export interface DashboardEvent { /** Working directory of the session */ cwd?: string; /** - * Data home (`getDataHome`) of the scope that recorded the event, the key its - * report filters on (#785). Absent on events written before this field - * existed; the report then attributes them by `cwd`. + * `dataHomeKey()` of the data home of the scope that recorded the event, the + * key its report filters on (#785). Absent on events written before this + * field existed; the report then attributes them by `cwd`. + */ + dataHomeKey?: string; + /** + * The data home itself, as a path, which unreleased builds of #795 wrote in + * place of `dataHomeKey`. Read only: the report keys it with `dataHomeKey()`. */ dataHome?: string; /** @@ -1262,10 +1280,12 @@ export interface DashboardEvent { status?: DashboardSessionStatus; /** AI output captured from transcript at session stop (truncated to 500 chars) */ stoppedOutput?: string; - /** Path to Claude Code transcript file (from Stop hook STDIN) */ + /** The session's transcript (from the Stop, UserPromptSubmit and SessionEnd hook STDIN; never Copilot's) */ transcriptPath?: string; /** Resolved PID of the AI tool main process (for liveness monitoring) */ monitorPid?: number; + /** Last event timestamp the PID monitor observed, so a delayed exit targets that run. */ + processExitAfter?: string; /** Byte boundary captured at Copilot SessionStart; private log path is never stored. */ copilotRunStartOffset?: number; /** Opaque marker metadata retained for events written by older collector versions. */ @@ -1365,8 +1385,8 @@ export interface DashboardSession { // getUserHome() (dashboard-collector.ts getEventsPath, dashboard.ts), so HOME // isolation already works there. Removed (issue #374 P3). The dashboard is an // A2 machine-level singleton keyed by sessionId, not per-project; each event -// carries the data home of the scope that recorded it, which is what a scope's -// report filters on. +// carries the key of the data home of the scope that recorded it, which is what +// a scope's report filters on. export const DASHBOARD_DEFAULT_PORT = 3721; /** Sessions with no activity for this long (ms) are marked idle */ export const DASHBOARD_IDLE_TIMEOUT_MS = 5 * 60 * 1000;