Skip to content

fix(desktop): correct WPP bridge token usage tracking #3

Closed

Description

@trmosala

Problem\n\nThe CookieMonster WPP bridge does not report token usage to OpenCode reliably.\n\nOpenCode already consumes standard OpenAI-compatible usage and persists it by session ID. The fault is in the bridge translation layer:\n\n- injected/content.js scrapes the WPP token pill immediately after network completion, before the UI is guaranteed to show the completed turn value.\n- openaiCompat.mjs treats that potentially stale cumulative value as authoritative total_tokens.\n- The prompt/completion split is synthetic.\n- If the pill is missing on a continued pinned thread, fallback accounting sees only the newly serialized delta and omits context retained by the WPP tab.\n- pageRecorder.js does not preserve network usage fields if WPP supplies them.\n\n## Expected behavior\n\nThe bridge should emit finite, nonnegative OpenAI-compatible usage for the completed provider step:\n\njson\n{\n \"prompt_tokens\": 123,\n \"completion_tokens\": 45,\n \"total_tokens\": 168\n}\n\n\nWhen available, it should also forward cache and reasoning details through prompt_tokens_details.cached_tokens and completion_tokens_details.reasoning_tokens.\n\nFor streaming responses, usage must appear before [DONE], after or alongside the finish-reason chunk.\n\n## Implementation\n\n- Add a failing regression test that simulates an unchanged pre-turn pill followed by a delayed post-turn update. Assert that the bridge never reports the unchanged pre-turn value.\n- Cover non-streaming and streaming responses. Assert streaming order: finish reason, usage-only chunk, [DONE].\n- Capture the token pill before prompt submission and associate it with the current worker run.\n- After network completion, poll the precise pill selector for a bounded period and accept a DOM measurement only when it differs from the pre-turn observation.\n- Treat an unchanged, missing, or unparseable post-turn pill as unavailable and use fallback accounting.\n- Remove the unrestricted whole-document scan from accounting, or retain it only for diagnostics.\n- Introduce an internal usage shape that preserves scope, source, and fidelity.\n- Extend pageRecorder.js to parse OpenAI-compatible usage from WPP SSE and JSON responses when present.\n- Preserve prompt, completion, total, cache-read, and reasoning values through serializeRecord, the controller, worker pool, and extensionBridge.mjs.\n- Normalize in this priority order: exact request-scoped network usage, confirmed post-turn context measurement, local estimate.\n- Keep prompt_tokens + completion_tokens === total_tokens and clamp invalid values.\n- Make continued-thread fallback context-aware by retaining the latest confirmed context measurement per pinned session or thread.\n- Clear retained usage when resetThread(sessionKey) runs or a worker is discarded.\n- Do not commit usage state for failed attempts. A fresh replay replaces the failed observation rather than adding it.\n- Add isolation tests proving parent and child OpenCode sessions cannot share usage state.\n- Add coverage for thread desynchronization, fresh replay, recovery turns, and worker replacement.\n- Keep native OpenCode accounting unchanged.\n\n## HITL validation required\n\nThe remaining semantic question must be answered against the live WPP UI before implementation locks in its accounting model:\n\nDoes the WPP token pill represent current conversation context occupancy, or lifetime or billable usage for the thread?\n\nCapture at least two consecutive turns from one WPP thread. For each turn, record:\n\n- Pill value immediately before submission.\n- Pill value when the network response completes.\n- Pill value after the UI settles.\n- Whether compaction, context truncation, retry, or recovery occurred.\n- Submitted prompt and response size, with sensitive content redacted.\n\nThe capture must establish whether the value can decrease after compaction or truncation and, where observable, whether it includes prior context, the current prompt, current completion, hidden instructions, and failed or recovery requests.\n\nRecord the evidence on this issue and select either scope: \"context\" or scope: \"lifetime\" before finalizing fallback and session-state behavior. Existing comments call the pill a cumulative conversation total. Treating it as current context occupancy remains a working assumption until this HITL check is complete.\n\n## Acceptance criteria\n\n- A stale pre-turn pill cannot be emitted as completed-turn usage.\n- Exact network usage wins over DOM and estimated usage.\n- Usage source, scope, and fidelity remain visible in bridge diagnostics and logs.\n- Continued pinned sessions account for retained context when exact usage is unavailable.\n- Usage state is isolated by OpenCode session ID and reset with WPP thread continuity.\n- Streamed usage reaches OpenCode before [DONE].\n- Native OpenCode code requires no modification.\n- Regression tests cover stale UI, missing pill, exact network usage, continuation, retries, recovery, and parent or child session isolation.

Activity

  1. added
    bugSomething isn't working
    HITLRequires human validation or interaction
    on Aug 21, 2026
  2. trmosala commented on Sep 22, 2026

    @trmosala
    OwnerAuthor

    Implemented and validated in ed32074af5.

    The bridge now:

    • records the pre-submit pill and accepts a DOM measurement only after it changes;
    • uses precise legacy/current WPP hooks (the current response-type-tag exposes N tokens via aria-label) and selects the newest assistant pill;
    • preserves exact request-scoped network usage plus cached/reasoning details when WPP supplies it;
    • resolves usage in the required order: exact network, confirmed context pill, estimate;
    • retains fallback context per OpenCode session only for a real continued turn, and clears/bypasses it for reset, failed replay, fresh replay, and worker replacement;
    • preserves prompt_tokens + completion_tokens === total_tokens;
    • emits optional streaming usage after the finish chunk and before [DONE];
    • records usage source, scope, fidelity, and before/network/settled observations in diagnostics and logs.

    Live macOS/WPP acceptance (2026-09-22)

    Validated with the locally built branded unsigned macOS app and visible WPP workers. Test content contained only the stated fixed-response probes.

    Turn Request / serialized prompt / response chars Before Network complete Settled Continuity
    1 132 / 283 / 21 unavailable 3,869 3,869 (0 ms) fresh
    2 257 / 264 / 21 3,869 3,869 stale 3,890 (202 ms) continued pinned thread
    large-context 24,275 / 24,302 / 11 3,972 9,970 9,970 continued pinned thread
    truncated 122 / 273 / 11 unavailable after New Chat 3,827 3,827 (1 ms) forced fresh replay
    post-restart replacement 233 / 436 / 17 unavailable 3,888 3,888 fresh worker/thread

    The 9,970 → 3,827 drop after replacing the large history with a one-message request establishes scope: "context", not lifetime/billable thread usage. The second turn proves the original fault: the network completed while the UI still showed 3,869; the bridge waited for and returned the settled 3,890 value. The app restart/worker replacement returned 3,888 without adding retained state.

    WPP did not expose structured usage in these live SSE responses, so the accepted live measurements were { source: "dom-pill", scope: "context", fidelity: "confirmed" }; every model response itself was captured through the page recorder with responseSource: "network". All live capture retry and recovery counts were zero. Retry, recovery, desync, failed replay, and worker-replacement accounting are covered by regression tests.

    A live stream with stream_options.include_usage: true emitted:

    1. finish_reason: "stop"
    2. usage-only chunk: 3,876 + 6 = 3,882
    3. [DONE]

    Validation

    • WPP bridge: 210 passed, 0 failed (533 assertions)
    • Focused usage/recovery/session tests: 70 passed, 0 failed
    • Desktop typecheck: passed
    • Push-time monorepo typecheck: 31 packages passed
    • Scoped lint: 0 errors (51 existing warnings)
    • Branded unsigned macOS build/package: passed; notarization intentionally skipped
    • Installed app app.asar matched the packaged app SHA-256: 1f143b926ea75ccb52058d52f61dbef0022f5b133fb64985b0c844e8452ae7c4
    • PKG SHA-256: 23b903fa61fb731fd1e623146b627d5868bb6ba6685d56280a21f4bd2346044a
    • Final installed app launched with visible workers; /bridge/health was healthy and /v1/models advertised 19 models.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    HITLRequires human validation or interactionbugSomething isn't working

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions