Repository navigation
feat(chat): show the estimated cost of each assistant reply in the message footer - #1697
zenpasdan1-ai wants to merge 5 commits into
Conversation
…ssage footer Read usage.cost.total from assistant message frames and chat.history rows into ChatMessageInfo.CostUsd, carry it on ChatEntryMetadata, and render it after the usage summary in the assistant footer with the gateway Control UI precision. The value is attached to the latest assistant entry through a dedicated ChatConversationState.SnapshotAssistantCost path so the usage snapshot logic that openclaw#1473 reworks is left untouched. Cost is hidden when the gateway reports none. Related: openclaw#1433, openclaw#1473, openclaw#551 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Pin the chat.history cost ingress through the real parser (replacing the reflection-only ExtractChatCost test), add a legacy payload.text cost test, skip the redundant snapshot when a fresh entry reports a zero cost, make the duplicate-final test count publishes against an uncosted baseline, guard the footer contract test anchors, cite the upstream Control UI formatCost source, and correct the docs: the <$0.0001 floor applies below $0.00005 and the whole assistant footer stays gated by the tool-call visibility setting from openclaw#551. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
|
🦞👀 Pull request received. I will update this pull request when review starts. ClawSweeper review completeClawSweeper finished reviewing this revision. The review result is being finalized. |
clawsweeper
Bot
added
P3
|
Codex review: needs real behavior proof before merge. What this changesThis PR displays Gateway-reported per-message cost in native assistant reply footers. Example: An assistant history row reports usage.cost.total = 0.2561.
Review scores
ProductKind: Feature · Worth it: Yes · Fix scope: Complete Merge readiness⛔ Blocked before merge - 1 item remains Keep this PR open: it adds distinct functionality absent from current main, and the two previously reported replay defects are repaired. Priority: P3 Before merge
FindingsNone. Tests
Agent review detailsHow this fits togetherThe shared Gateway client parses live events and history into typed messages; native chat state attaches their metadata to timeline entries, and the footer formatter produces the displayed cost. flowchart LR
A[Gateway live events] --> C[Shared message parser]
B[Gateway chat history] --> C
C --> D[Chat provider and history loader]
D --> E[Conversation entry metadata]
E --> F[Cost formatter]
F --> G[Native assistant footer]
Technical reviewBest possible solution: Retain Gateway-owned per-message cost and render it once on the final assistant entry through the existing native footer. Do we have a high-confidence way to reproduce the issue? This is a feature request; source inspection confirms current main lacks per-message cost display, and supplied older-head media demonstrates the proposed single-reply presentation. Is this the best way to solve the issue? The existing parser, conversation metadata, history loader, and footer formatter are appropriate owners; the change avoids a new RPC or competing usage accumulator. AGENTS.md: found and applied where relevant. Codex review notes: model internal, reasoning medium; reviewed against 16299381a4df. Provenance checked
TestingProof path: shipped entry point. SecurityNone. EvidenceWhat I checked:
Likely related people:
Review metrics
LabelsLabel changes: No label changes. Label justifications:
Rank-up movesOptional improvements that raise the rating; they are not merge blockers.
Rating scale6/6 🦀 challenger crab · 5/6 🦞 diamond lobster · 4/6 🐚 platinum hermit · 3/6 🦐 gold shrimp · 2/6 🦪 silver shellfish · 1/6 🧂 unranked krab. Overall follows the weaker of proof and patch quality; ✨ marks media proof (a screenshot, video, or linked artifact) that directly shows the changed behavior. WorkflowClawSweeper edits this one comment on every review. Comment HistoryReview history (4 earlier review cycles)
Reviewed October 12, 2026, 9:25 AM ET / 13:25 UTC (Revision 5). |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
The test pinned method ordering and formatter fragments and would still pass if the cost were excluded from the rendered footer. The formatter and provider tests cover the behavior. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b00a02b987
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A chat.history assistant row with interleaved text and tool parts replays as several timeline entries. Its single usage cost was copied onto every one of them, so one charge rendered at each assistant run end. Attach it to the row's last text part only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Global triage: HOLD_FOR_AUTHOR. Take confidence 45%; recommendation confidence 99%; effort medium; risk medium. Reviewed exact head The current screenshot proves cold-start history rendering, but not a newly completed live reply. Accessibility proof is unrun, full Shared validation has one reported baseline-dependent failure, and exact-head checks are dispatch-only with one cancelled job. This PR also overlaps #1473 in the history/metadata path and must be refreshed after that dependency is resolved. Author action: assign the message cost to exactly one visible assistant replay entry, add a provider-level text/tool/text history regression that asserts a single cost before and after reload, and resolve the existing inline thread. Then collect current-head This is a triage classification, not a merge action. No branch changes, fixes, pushes, labels, CI reruns, or other remote mutations were made. |
|
Pushed bd205a9 on top of the reviewed head:
Validation on this head: Still open on my side: a live completion-to-footer capture on the current head and the accessibility proof pool. I will rebase after #1473 lands. 🤖 Generated with Claude Code |
clawsweeper
Bot
added
rating: 🦪 silver shellfish
|
Global triage: HOLD_FOR_AUTHOR. Take confidence 52%; recommendation confidence 99%; effort medium; risk medium. Reviewed exact head This exact head also has only dispatch/security checks, not canonical build and test CI. The body validation and screenshot are for Author action: attach the row cost to the final rendered assistant replay entry, including media, and add a provider-level text -> media regression asserting one visible cost. Then run exact-head required validation and collect current-head live-final, multipart reload, and accessibility proof before requesting review. This is a triage classification, not a merge action. No branch changes, fixes, pushes, labels, CI reruns, or other remote mutations were made. |
The cost carrier preferred a text part over a later media part. The footer renders on the run's final assistant entry, so a text then image row kept the cost on the earlier text entry and hid it after reload. Attach it to the last part that renders an assistant entry, text or media. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Pushed b91055a for the text then media case from the triage note.
Exact-head validation on b91055a: Not done yet: current-head live-final capture, multipart reload capture, and the accessibility proof pool. The screenshot in the body is still from b00a02b. 🤖 Generated with Claude Code |
|
Global triage: NEEDS_HUMAN_TEST. Take confidence 78%; recommendation confidence 99%; effort medium; risk medium. Reviewed exact head Author-reported exact-head local validation has the build and all 4,024 Tray tests passing. Shared tests report 4,271 passed, 36 skipped, and one host-specific The remaining gate is current-head behavioral proof. Collect This is a triage classification, not a merge action. No branch changes, fixes, pushes, labels, CI reruns, or other remote mutations were made. |
This branch has not been deployed
2 participants
Builds on #551 (Show chat usage in assistant timestamp footer).
Related: #1433 (Chat context footer accumulates overlapping request totals): cost is per message, never summed and never reconciled against
sessions.list; pinned byChatMessageReceived_CostStaysOnItsOwnAssistantMessage.Related: #1473 (fix(chat): correct context footer usage snapshots): touches the same files; this PR adds one new method and leaves
SnapshotAssistantUsageContributionLockedunmodified.Footer placement precedent: #1220 (Show response duration in the assistant-message model/speaker footer, closed not-planned).
Compatibility: does not call the
usage.costRPC (closed #1345); reads the cost field already present on assistant frames and history rows.What Problem This Solves
The Desktop chat shows tokens and context usage under each assistant reply but not what the reply cost, so operators have to open the gateway Control UI or the
/usagecommand to see the per-message cost the gateway already computes.User Impact
User impact: each assistant message footer now ends with the gateway-reported cost of that reply, for example
Assistant · 11:10 · qwen3.8-27b · 70.5K/131.1K (53%) · $0.588, when the gateway includesusage.cost.totalon the message. Nothing changes when the gateway reports no cost.The amount is the gateway's estimate from its configured per-token prices, not an invoice: for a local model it is a counter at whatever prices were configured, and subscription or OAuth providers may show a value that is never billed. No marker is added to the number; this matches the gateway Control UI, which renders the bare
$value.Why This Change Was Made
The gateway attaches
usage.cost.totalto assistant message frames (observed live on the pre-rebase worktree) and tochat.historyrows (observed on this head, see Real Behavior Proof). This PR reads that field intoChatMessageInfo.CostUsd, carries it onChatEntryMetadata, and renders it throughChatUsageFormatter.FormatCostwith the gateway Control UI precision (four decimals below $0.01, three below $1, otherwise two; amounts that would round to zero at four decimals (below $0.00005) render as<$0.0001; missing, zero, negative, or non-finite values are hidden). The footer itself keeps the #551 placement rule: it renders on the last bubble of each assistant run and only while tool calls are visible (Settings > Show tool calls).The cost is attached to the latest assistant entry through a new self-contained
ChatConversationState.SnapshotAssistantCostmethod instead of extending the usage-contribution path, so the usage snapshot code that #1473 reworks is left byte-for-byte unchanged. If #1473 lands first, the only expected touch is hunk ordering inOpenClawChatDataProvider(itshasUsageblock is deleted; theSnapshotAssistantCostcall stays); noChatConversationStateorChatEntryMetadataconflict is expected becauseCostUsdsits afterContextPercentin the record and every positional construction stops atContextPercent.Evidence
ParseChatHistoryPayload_AssistantRow_ReadsUsageCostandParseChatHistoryPayload_AssistantRow_RejectsInvalidUsageCostthrough the realchat.historyparser, three wire-levelChatEvent_*Cost*tests plusProcessRawMessage_LegacyAssistantMessage_ReadsPayloadUsageCostthroughProcessRawMessage), formatting (FormatCost_MatchesControlUiPrecision,FormatCost_ShowsFloorForCostBelowDisplayPrecision,FormatCost_HidesMissingZeroOrInvalidCost), the footer contract (ChatCostFooterContractTests), and the provider/state behavior (tenOpenClawChatDataProviderTestscases: merge onto a streaming entry, cost-only final, duplicate final with a snapshot-count baseline, zero cost on a fresh entry stays unset, cost stays on its own message, cost after a tool call, later cost replaces earlier cost with the identified-replay row pinned as dropped,sessions.updatedpreserves cost, history captures cost, history row without cost replaces the live cost).mainand the footer wording change): the footer showed the gateway-reported cost next to time, model, and usage against a local Gateway 2026.9.9 with a local model whose prices were configured per million tokens, so the value behaved as a counter and updated per reply. Those screenshots show the earlier~$x (virtual)wording and are not attached; the current-head capture is in Real Behavior Proof.Change Type
Scope
winnodeusage.cost.totalfrom existing frames and history rows; no new RPC)Required proof pools
windows-winui-interactive: the assistant message footer changed.Validation
Run on head b00a02b (Windows 11 Pro build 26300, .NET SDK 10.0.401), 2026-10-10, from a clean working tree. Environment:
$env:OPENCLAW_REPO_ROOT=<repo>; $env:OPENCLAW_TRAY_DATA_DIR=<isolated scratch directory outside the repo>. Full logs were kept outside the repo (validate-build.log,validate-shared.log,validate-tray.log,validate-cost-shared.log,validate-cost-tray.log)..\build.ps1: exit 0.Proof-pool validation passed: 7 named pools checked.,Documentation validation passed: 59 Markdown files checked., Build Summary: Shared, Cli, WinNodeCli, SetupEngine, WinUI all built (Debug, win-x64),All builds succeeded!.dotnet test .\tests\OpenClaw.Shared.Tests\OpenClaw.Shared.Tests.csproj(first run in this bin, so with restore): exit 1; total 4308, passed 4271, skipped 36, failed 1. The one failure isOpenClaw.Shared.Tests.TokenSanitizerTests.SanitizeLogMessage_RedactsCurrentUserAndMachineNames(Assert.Containsexpected<host>inuser=<user> machine=<user>, TokenSanitizerTests.cs line 283). It is environmental on the validation host, where USERNAME equals COMPUTERNAME (case-insensitively), so the machine name is redacted as<user>before the<host>rule applies; the same test fails identically on unmodifiedmainon this host, no file touched by this PR is involved, and it is reported here as a failure rather than hidden. The 36 skips are the pre-existing integration tests that skip without their external prerequisites.dotnet test .\tests\OpenClaw.Tray.Tests\OpenClaw.Tray.Tests.csproj(with restore): exit 0; total 4023, passed 4023, skipped 0, failed 0.dotnet test .\tests\OpenClaw.Shared.Tests\OpenClaw.Shared.Tests.csproj --filter "FullyQualifiedName~Cost" --logger "console;verbosity=normal": exit 0; 128 passed, 0 failed, 0 skipped. Added by this PR (11 cases):OpenClawGatewayClientTests.ChatEvent_ReadsAssistantMessageUsageCost,ChatEvent_PrefersMessageUsageCostOverPayloadCost,ChatEvent_FallsBackToPayloadUsageCost_WhenMessageHasNone,ProcessRawMessage_LegacyAssistantMessage_ReadsPayloadUsageCost,ParseChatHistoryPayload_AssistantRow_ReadsUsageCost,ParseChatHistoryPayload_AssistantRow_RejectsInvalidUsageCost(6 rows). The other 117 matches are pre-existing tests the filter also selects:GatewayUsageInfoTests(3),OpenClawGatewayClientTests.HandleRequestError_UnknownMethod_UsageCost_SetsUnsupportedFlag,OpenClawGatewayClientTests.ParseUsageCostPayload_UpdatesLegacyUsageTotals, and 112 data rows ofExecApprovalSecretRedactorTests.RedactMatchesMacOsToolsModeVectorswhose row text contains "cost".dotnet test .\tests\OpenClaw.Tray.Tests\OpenClaw.Tray.Tests.csproj --filter "FullyQualifiedName~Cost" --logger "console;verbosity=normal": exit 0; 37 passed, 0 failed, 0 skipped. Added by this PR (27 cases):ChatCostFooterContractTests.AssistantFooter_RendersCostThroughSharedFormatter;ChatUsageFormatterTests.FormatCost_MatchesControlUiPrecision(8 rows),FormatCost_HidesMissingZeroOrInvalidCost(5 rows),FormatCost_ShowsFloorForCostBelowDisplayPrecision;OpenClawChatDataProviderTests.ChatMessageReceived_AssistantFinal_MergesCostOntoStreamingEntry,ChatMessageReceived_CostOnlyFinal_AttachesCostToLatestAssistantEntry,ChatMessageReceived_DuplicateAssistantFinalWithCost_KeepsSingleCostAndUsage,ChatMessageReceived_ZeroCostOnFreshEntry_LeavesCostUnset,ChatMessageReceived_CostStaysOnItsOwnAssistantMessage,ChatMessageReceived_CostAfterToolCall_LandsOnFinalAssistantEntry,ChatMessageReceived_LaterAssistantFinalCost_ReplacesEarlierCost(3 rows),SessionsUpdated_PreservesAssistantCost,LoadHistoryAsync_CapturesAssistantCost,LoadHistoryAsync_HistoryRowWithoutCost_ReplacesLiveCost. The other 10 matches are pre-existing:Services.TrayDashboardSummaryBuilderTests(3) andUsageCostApplicationPolicyTests(7 incl. rows).b00a02b9 fix(chat): address review findings for cost footeron top of04e7c225 feat(chat): show the estimated cost of each assistant reply in the message footer, based on037c17dc(fix(chat): make Enter send from the multiline composer #1674).mainhas since advanced by one commit (16299381, fix: preserve node containment settings when opening Sandbox #1683) that touches none of these files;git merge-tree --write-tree origin/main HEADreports a clean merge. The validation above was not rerun on the merge result.Real Behavior Proof
Environment tested: Windows 11 Pro (build 26300), OpenClaw Gateway 2026.9.9 (WSL), local OpenAI-compatible model with per-million-token prices configured in the gateway so
usage.cost.totalis reported; Release build of this head (app-identityrelease, product version 2026.9.9-feat-message-cost-footer) copied over the installed Desktop app (real tray data, same gateway pairing).PR head or commit tested: b00a02b.
Exact steps or command run: close the Desktop, install the Release output of this head, start the app (new process), open a session whose assistant reply was produced before this process started, read the footer. That reply can only have come from
chat.history.Evidence after fix: footer under a stored assistant reply reads
Assistant · 12:32 · qwen3.8-27b · 24.1K/131.1K (18%) · $0.256(gateway store value 0.2561 for that message), rendered fromchat.historyin a freshly started app.Observed result: history rows on Gateway 2026.9.9 carry
usage.cost; the cost is restored from history on a cold start, and the three-decimal precision branch rendered as specified. Context usage segment unchanged.Screenshot or artifact links verified?
Yes(the image above resolves; it is hosted on a branch of the PR author's fork).Not verified or blocked:
transient same-model retry ... reason=timeoutwhile the model server reported the request complete), so no fresh assistant final was observed on this head; the live path (ChatEvent_ReadsAssistantMessageUsageCost,ChatMessageReceived_AssistantFinal_MergesCostOntoStreamingEntry) is covered by unit tests only on this head. A live capture was observed on the pre-rebase worktree with identical parsing code and the earlier wording..\scripts\run-proof-tests.ps1 -Project 'tests\OpenClaw.Tray.UITests\OpenClaw.Tray.UITests.csproj' -Filter 'Category=Accessibility' -ResultName 'winui-accessibility' -RuntimeIdentifier win-x64.<$0.0001floor are covered only by unit tests; no live provider on the test machine produces them.usage.cost.totalOrigin) were not exercised live.messagethenpayload) were not observed live; covered by the wire-level test only.usage.costas a bare number is intentionally ignored (pinned by a test row).FooterText.models.listwithincludeDetails: trueand surface the gatewaylocalflag to annotate local-model pricing, and honorusage.cost.totalOrigin == "provider-billed".Security Impact
No)No)No)No)No)Yes, explain the risk and mitigation: N/ACompatibility and Migration
Yes)No)No)ChatEntryMetadatais not JSON-serialized, so there is no persisted format change;ChatMessageInfo.CostUsdis a nullable addition and older gateways that omit the field produce the previous footer.Rubber-duck review
Rubber-duck review was run on the pre-publication plan and again on the branch; findings applied: removed the hard-coded local-provider list (it can never be complete), adopted the Control UI precision instead of fixed
F4, moved cost onto a dedicatedSnapshotAssistantCostpath soSnapshotAssistantUsageContributionLockedstays untouched for #1473, added the history-parser, legacy-payload, zero-cost, and duplicate snapshot-count tests, and corrected the floor and footer-gating wording in the docs. Decisions kept:$value andSessionInfo.Provideris operator-typed, so a hard-coded local-provider list can never be complete and would ship private provider ids. The "counter, not a bill" message lives indocs/CHAT_VISUAL_DESIGN.mdand in this PR body.4 / 3 / 2decimals by magnitude, mirroringformatCostinui/src/lib/format.tsof openclaw/openclaw (commit 0acc55cfa8a973f9d1764f7641f41e5a38e88abe); two deliberate divergences: zero is hidden instead of$0.00, and values below $0.00005 render as<$0.0001. The three existing$+F2sites in the tray are totals where two decimals are right, so no shared helper was introduced.SnapshotAssistantCostoverwrites a differing value, ignores an identical one, skips a reported zero on an entry that has no cost (renders the same as none), and never runs for identified duplicates because the queue classifier drops them first (pinned by the"m1"row ofChatMessageReceived_LaterAssistantFinalCost_ReplacesEarlierCost).chat.historyrebuild replaces live metadata for rows it covers, so a history row withoutusage.costclears a live cost (pinned byLoadHistoryAsync_HistoryRowWithoutCost_ReplacesLiveCost).message.usage.cost.totalbeforepayload.usage.cost.totalon live frames; it is the proven live order and the same objectchat.historyreads. Token usage uses the reverse precedence; left as is to keep the diff small.usage.costRPC, which this PR does not call.docs/TELEMETRY.md).Review Conversations
🤖 Generated with Claude Code