Attribute nested Grok and Antigravity review-CLI token usage in cost reports

[atomantic]

Problem / Goal

The Usage page cost report (/devtools/usage, backed by data/usage.json) shows negligible Grok and Antigravity token usage even when those CLIs run substantial --review-with code-review passes (and as first-class CoS/toolkit providers). Quota meters can still look “used” because they scrape each CLI’s /usage panel; the persistent cost ledger is a different store and is what is wrong.

Goal: every Grok / Antigravity (and same-class nested CLI) session that ran in a PortOS workspace during a PortOS run is discovered from the harness’s local session files and attributed to that provider in data/usage.json — not discarded, not swallowed into the parent’s Claude/Codex bucket, and not left as chars/4 of the parent’s initial task blurb.

Context

Two independent systems share /devtools/usage. Mixing them up is how this looks fine on quota cards and empty on tokens:

• Quota cards — GET /api/usage/providers → server/services/providerUsage.js. Headless TUI scrape of agy /usage and grok /usage show (server/lib/tuiUsageScrape.js). Percent remaining/used. Already works (Add Antigravity + Grok subscription-usage adapters via headless TUI /usage scrape #2598). Out of scope.
• Cost / token table — GET /api/usage → server/services/usage.js (data/usage.json). Written by recordCompletedRunUsage (server/services/usageReconciler.js:515). This is the undercount. The page footer already admits estimated rows “understate real usage substantially” (client/src/pages/UsagePage.jsx:832-838).

How tokens get into usage.json today

recordCompletedRunUsage(metadata, output):

1. Estimate: tokensIn = ceil(promptLength / 4), tokensOut = ceil(stdout.length / 4), cache = 0, source: 'estimate' (usageReconciler.js:518-521, contextBudget.js). CoS promptLength is only task.description.length (agentRunTracking.js:51), not the built agent prompt.
2. If transcriptFamily(run) is 'claude' or 'codex', replace that with the CLI’s own per-message counts from disk (providerTranscriptUsage.js). Claude: ~/.claude/projects/<cwd-slug>/*.jsonl. Codex: ~/.codex/sessions/YYYY/MM/DD/rollout-*.jsonl. Includes cache tiers. This is Usage cost report understates real AI cost by orders of magnitude: input/cache tokens are never counted and most day buckets are dropped #3124.
3. Else keep the estimate. transcriptFamily is explicitly null for agy, grok, ollama, lmstudio, kimi (usageReconciler.js:169-184, pinned in usageReconciler.test.js).

Call sites: CoS completeAgentRun records on success and failure (agentRunTracking.js:162). Toolkit onRunCompleted records on success only (bootstrap.js:171-176).

Why nested Grok/Antigravity reviews are invisible

A CoS task that reviews with grok/agy does not spawn a grok/agy PortOS run.

task:ready → spawnAgentForTask → createAgentRun(PARENT provider)
  → parent CLI is told to run /do:pr --review-with grok,antigravity,…
  → lib/slashdo/lib/local-agent-review-loop.md Step 2
     grok --permission-mode bypassPermissions -p "$LOCAL_PROMPT"
     agy  --dangerously-skip-permissions --model "…" --print-timeout 30m -p "$LOCAL_PROMPT"
  → stdout → temp LOG_FILE the parent parses
  → no createAgentRun, no recordCompletedRunUsage for grok/agy
  → parent completes → recordCompletedRunUsage(PARENT metadata, parent stdout)

Prompt injection: server/services/agentPromptBuilder.js (~308–347, ~841). Follow-up review agents (spawnReviewLoopFollowUp in server/services/agentWorktreeCleanup.js:816) inherit the source task’s provider; they bash-launch reviewer CLIs rather than becoming grok/agy CoS agents.

Same-family nested Claude/Codex can piggyback (same transcript dir + window). Nested grok/agy never can. Cross-family nested Claude/Codex is also dropped: a grok parent that shells out to codex review writes Codex rollouts, but transcriptFamily(grok) is null so those files are never read.

What the harnesses already write (0 tokens to read)

Grok — ~/.grok/sessions/<urlencoded-cwd>/<session-id>/:

• summary.json — created_at, last_active_at, current_model_id, git_root_dir, num_messages
• updates.jsonl — session/update / _x.ai/session/update. Streaming chunks carry _meta.totalTokens (context-window occupancy — jumps when a tool result is added; not billed). On sessionUpdate: "turn_completed" the payload has real billed counts:

{
  "sessionUpdate": "turn_completed",
  "usage": {
    "inputTokens": 15258,
    "outputTokens": 1828,
    "totalTokens": 17086,
    "cachedReadTokens": 11264,
    "cacheCreationTokens": 0,
    "reasoningTokens": 848,
    "modelUsage": { "grok-4.5-build": { "inputTokens": 15258, "outputTokens": 1828 } }
  }
}

Those usage totals are cumulative for the session (same hazard as Codex total_token_usage). Summing every turn_completed inflates badly. Take the last snapshot in the run window minus a per-session high-water mark.

• chat_history.jsonl — fallback when no turn_completed (killed/interrupted run): user / assistant / reasoning / tool_result with model_id. chars/4 estimate.

PortOS pins grok headless to --output-format plain (server/lib/grok.js:91-92); stdout is not the source of truth.

Antigravity — ~/.gemini/antigravity-cli/:

• history.jsonl — { timestamp, workspace, conversationId } (cwd + time correlation)
• brain/<id>/.system_generated/logs/transcript.jsonl — steps (PLANNER_RESPONSE, tool calls, created_at); no token fields. chars/4 of textual step content.
• agy --output-format is stripped as unsupported (server/lib/aiToolkit/internal/antigravity.js)

Cursor / Kimi sit in the same transcriptFamily === null hole. Include them in the sibling-family scan only if a durable cwd-keyed session store is found during implementation; do not block grok/agy on that.

Backfill as written cannot repair this

usageBackfillWorker.js:41-44 skips !transcriptFamily(metadata) and already-usageReconciled runs. applyHistoricalUsageCorrections (usage.js:331-379) further requires !reconciledRuns[runId] and a pre-existing dailyActivity[day].byProvider[parentProviderId] bucket, then adds every measured record to that parent bucket. Sibling grok rows stuffed into the parent’s measured array would land on Claude. isMeasured (usageBackfillWorker.js:12-13) would also drop Antigravity’s transcript-backed estimates.

Related issues (not duplicates)

• Usage cost report understates real AI cost by orders of magnitude: input/cache tokens are never counted and most day buckets are dropped #3124 (closed) — measured Claude/Codex transcripts. This is the grok/agy/nested follow-on.
• Backfill historical usage buckets from provider transcripts (correlated per-run, no double-count) #3156 (closed) — historical backfill for transcript families.
• Add Antigravity + Grok subscription-usage adapters via headless TUI /usage scrape #2598 (closed) — quota-panel scrape for agy/grok. Different store.
• Federated usage sync transfers the whole digest map every 60s — add a capturedAt manifest + per-slot fetch #5759 (open) — federated usage sync shape. Different layer; do not fold this in.

Decisions (locked)

Do not re-litigate these. File a follow-up if a live install proves one wrong.

1. Do not wrap nested reviewers as PortOS CoS runs. Slashdo’s review loop is a child CLI of the parent agent. Read the CLIs’ own session files at parent completion — the same Usage cost report understates real AI cost by orders of magnitude: input/cache tokens are never counted and most day buckets are dropped #3124 shape.
2. At every PortOS run completion, scan every transcript family in that workspace + time window, not only the parent’s family. Attribute each session to its provider. A Claude parent that bash-launched grok records a grok row; a grok parent that bash-launched Codex records a Codex row. The per-message / per-session claim ledger stays the exclusivity mechanism.
3. Map a sibling family to the install’s matching enabled provider via familyForProvider (server/lib/providerFamilies.js: claude / codex / agy / grok). Prefer type === 'cli' over tui. If several cli records exist, prefer the one whose default/selected model matches modelUsage / model_id; else the first enabled cli. Skip the family if none match — do not invent an unknown grok bucket.
4. Grok parser: correlate by URL-encoded cwd directory + summary.json timestamps vs run [startTime, endTime] ± existing WINDOW_SLACK_MS. Prefer last turn_completed.usage in the window minus per-session high-water (Codex cumulative pattern). Map inputTokens → tokensIn, outputTokens + reasoningTokens → tokensOut, cachedReadTokens → cacheReadTokens, cacheCreationTokens → cacheWriteTokens, modelUsage → byModel. source: 'measured'. Do not sum _meta.totalTokens. Fallback: chars/4 of chat_history.jsonl (user+tool_result → in, assistant+reasoning → out), source: 'estimate'. Claim key = session id (plus prompt id when windowing turns).
5. Antigravity parser: correlate history.jsonl.workspace with cwdMatches (usageReconciler.js:118-122) and timestamp to the run window; chars/4 the matching brain transcript’s textual steps. Model = provider defaultModel (use a settings-change line if one names a model; do not regex-hunt the prompt). source: 'estimate'. Claim key = conversationId.
6. First-class grok/agy PortOS runs go through the same parsers via transcriptFamily returning 'grok' / 'agy'.
7. Backfill: sibling-family scan must run even on already-reconciled parent runs. applyHistoricalUsageCorrections must route each record to record.providerId (creating the day bucket if needed), subtract the parent estimate only from the parent provider, and accept sibling-add corrections that have no parent estimate to remove. isMeasured must not drop transcript-backed estimates. Historical repair stays the user-triggered POST /api/usage/backfill — no boot-time / silent backfill.
8. Keep the prompt/stdout estimate fallback when no session matches. A usage-accounting failure must never fail the run.
9. Do not change grok --output-format plain or agy argv as the primary source. Session files exist regardless of stdout format.

Proposed approach

1. server/lib/providerTranscriptUsage.js — add parseGrokSession and parseAgyTranscript (pure, truncated-line tolerant, byModel + countedKeys + exclude, same return shape as Claude/Codex plus source). Tests next to the existing parser tests. Pin both grok traps: occupancy _meta.totalTokens is not billed, and turn_completed.usage is cumulative (a two-turn fixture must not double).
2. server/services/usageReconciler.js
   • Extend transcriptFamily to 'grok' / 'agy' (and 'cursor' only if a store was found).
   • Split “reconcile parent family” from “collect sibling-family sessions in this window”.
   • Resolve sibling family → enabled provider as in decision 3 (listProviders() / familyForProvider; not getAllProviders()’s envelope).
   • reconcileRunUsage returns the parent record(s) plus sibling records. recordRunUsage already accepts an array and keys each entry by providerId.
3. recordCompletedRunUsage — unchanged caller contract; internally does the sibling scan. CoS and toolkit completion both flow through here.
4. usageBackfillWorker.js + applyHistoricalUsageCorrections — sibling pass on already-reconciled runs; per-record provider routing; do not require a pre-existing parent day bucket for a sibling-only add; do not drop source: 'estimate' sibling rows.
5. Claim ledger — grok session id / agy conversationId (and Codex/Claude message keys as today) so two overlapping PortOS runs in one worktree cannot both bill one nested review. Grok high-water sits next to the existing Codex high-water map.
6. Tests (boundary)
   • First-class grok run with turn_completed → grok row, source: 'measured', cache tiers populated, model from modelUsage.
   • Grok run with only chat_history.jsonl → source: 'estimate'.
   • Claude parent + grok session in the same cwd/window → two recordRunUsage entries (claude measured, grok measured/estimate). Nested grok tokens must not appear under Claude.
   • Overlapping same-cwd runs → nested grok session billed once.
   • Cumulative turn_completed not double-counted; _meta.totalTokens ignored.
   • transcriptFamily({ providerId: 'agy' }) === 'agy'.
   • Backfill sibling scan on an already-reconciled Claude run routes the grok row to the grok provider.
   • No matching session → existing prompt/stdout estimate, run still succeeds.
7. Usage page footer (UsagePage.jsx:832-838) — Grok rows can be Measured when turn_completed exists; Antigravity rows stay Estimated (transcript chars/4). Do not imply billed counts for agy.

Acceptance criteria

• A CoS or toolkit run whose provider is Claude/Codex and whose workspace contains a Grok session overlapping the run window records a Grok cost-report row; those tokens are not added to the Claude/Codex row.
• The same for an Antigravity history.jsonl + brain transcript overlapping that window (Estimated).
• A first-class Grok PortOS run with turn_completed.usage records source: 'measured' (including cache tiers and modelUsage); without it, chars/4 of chat_history.jsonl is Estimated.
• A first-class Antigravity PortOS run records tokens from its brain transcript, not from task.description + TUI stdout alone.
• _meta.totalTokens occupancy is never billed; multiple turn_completed snapshots in one session are not summed.
• Two overlapping runs in one cwd do not double-count one nested session.
• Sibling sessions are skipped (not unknown) when no enabled provider matches that family.
• POST /api/usage/backfill attributes historical nested grok/agy sessions to overlapping parent runs without rewriting already-measured Claude/Codex totals and without stuffing sibling tokens into the parent provider bucket.
• Quota cards (GET /api/usage/providers) are unchanged.
• Parser/reconcile/backfill tests above exist; a usage-accounting failure still cannot fail the agent run.
• Usage page footer names grok measured-vs-estimate and agy session estimates.

Out of scope

• Quota-panel scrape / Quota Burn (providerUsage.js, Add Antigravity + Grok subscription-usage adapters via headless TUI /usage scrape #2598).
• Federated usage sync transfer shape (Federated usage sync transfers the whole digest map every 60s — add a capturedAt manifest + per-slot fetch #5759).
• Turning nested slashdo reviewers into PortOS CoS agents.
• Changing grok --output-format or teaching agy a JSON usage stream as the primary source.
• Image/video Grok jobs, API executeApiRun SSE usage fields, Codex app-server thread/tokenUsage/updated.
• Cursor/Kimi parsers unless a durable cwd-keyed session store is found while doing grok/agy — then include them in the same sibling scan; otherwise file a follow-up with the store path.
• Boot-time or scheduled silent backfill.
• New usage.json fields for reasoningTokens / costUsdTicks — fold reasoning into tokensOut; PortOS keeps its own rate table.

Open questions

None — decisions above are locked.

[atomantic]

Implemented in #6035. Two of the locked decisions were adjusted because a live install disproved them — recording them here per the "file a follow-up if a live install proves one wrong" clause. Both were validated by counting shapes only; no session content was read into anything committed.

Decision 4 — turn_completed.usage is not always cumulative. Across 61 grok sessions with completed turns, 44 of the 46 multi-turn ones are non-monotonic in totalTokens, so on this grok version each block describes only its own prompt. Summing is correct there; last-minus-high-water would badly under-count. Rather than pick one shape, the parser detects it: 3+ turns non-decreasing in inputTokens, outputTokens and cachedReadTokens together is treated as cumulative and converted to per-turn deltas before summing (0 of 39 real per-prompt sessions with 3+ turns trip that test). Both shapes then flow through one code path, and both hazards are pinned in tests. The per-prompt-id claim ledger makes the Codex-style high-water map unnecessary — a grown file re-parses to the same per-prompt deltas and the already-billed prompt ids are excluded.

Decision 4 — token field mapping. Over 373 real turns, cachedReadTokens <= inputTokens and reasoningTokens <= outputTokens with zero counterexamples, and totalTokens === inputTokens + outputTokens in every one. Both are subsets. So the mapping is tokensIn = inputTokens - cachedReadTokens (Codex precedent, so cache reads price at the cache tier rather than full input) and tokensOut = outputTokens (reasoning already inside it). Mapping inputTokens straight to tokensIn and adding reasoningTokens on top, as written, would have over-billed both tiers.

Two smaller notes: Antigravity conversations are correlated on history.jsonl.workspace alone and then windowed at the step level via the brain transcript's created_at, rather than on the history entry's own timestamp — a conversation that starts before a run's window can still have steps inside it. And the claim key is the step index rather than the conversation id, which is strictly finer-grained.

Cursor/Kimi: no durable cwd-keyed session store was found while doing grok/agy, so they are not included and stay in the transcriptFamily === null hole.

[atomantic]

Shipped in #6035 (merged).

Done ✓

• Grok (~/.grok/sessions/) and Antigravity (~/.gemini/antigravity-cli/) session files are now discovered by cwd + run window and attributed to their own provider in data/usage.json — first-class runs, nested --review-with passes, and a backfill for existing rows.
• turn_completed.usage shape is detected rather than assumed: this Grok version writes per-prompt blocks on most sessions, so a cumulative stream is converted to per-turn deltas before summing. Both shapes and both double-count hazards are pinned in tests.
• Token mapping corrected against real turns — cachedReadTokens and reasoningTokens are subsets, so tokensIn = inputTokens - cachedReadTokens, tokensOut = outputTokens. The mapping originally specified here would have over-billed both tiers. (Details: Attribute nested Grok and Antigravity review-CLI token usage in cost reports #5831 (comment))
• Usage page footer updated to stop claiming these rows are estimates.

Moved to #6045 ▢

• Cursor and Kimi, which this issue gated on finding a durable cwd-keyed session store. None exists for either CLI today, so there is no data source to attribute from. Filed as blocked + reference-watch to be re-checked on a CLI upgrade.

Closing — the actionable scope is delivered.