Summary
Reported run cost was ~15% below the amount actually billed by the provider. The shortfall is not uniform: accounting reconciled almost exactly across the Claude Sonnet 5 enhance phase, and the entire gap accumulated across the two phases that ran on Claude Opus 5 (Stage 1 detection and Stage 2 verification).
Evidence
Reconciliation was performed twice against the provider's own usage endpoint during a single run:
| Checkpoint |
Discrepancy vs billed |
| After the enhance phase (Claude Sonnet 5) |
0.12% |
| Full run (adds Stage 1 + Stage 2 on Claude Opus 5) |
14.6% under |
Those two reasoning-model phases accounted for ~91% of reported run cost, implying roughly a 19% undercount on them specifically. Token counts were recorded and appeared plausible; only the derived cost was short.
What I can and cannot attribute it to
Both models in this run perform extended thinking by default, and reasoning tokens bill as output tokens — so the mere presence of thinking does not explain why one phase reconciled and the others did not. That rules out the first hypothesis I reached for. Two candidates survive the observation:
- Reasoning volume rather than reasoning presence. If the gateway's OpenAI-shaped
usage.completion_tokens omits reasoning tokens (OpenRouter surfaces them separately for some models), then any undercount scales with how much a model actually reasons. Stage 1 and Stage 2 put open-ended exploitability questions to Claude Opus 5; the enhance phase asks Claude Sonnet 5 for a short classification. That would concentrate the error exactly where it appeared.
- Per-turn usage capture in multi-turn loops. Stage 2 issued roughly 8.5 API calls per unit. If usage is recorded only from the final response of a tool-use loop, intermediate turns are lost. Counting against this: the enhance phase is also an iterative tool-use loop and reconciled accurately — so if this is the cause, something differs between how the two loops account for usage.
I did not instrument either path, so please treat the measurement as the finding and both mechanisms as leads rather than diagnoses.
Why it matters
ARCHITECTURE.md §6 already documents "no spend ceiling — --limit caps units, not dollars." A meter that under-reports by roughly a fifth on the model class most likely to be chosen for detection compounds that hazard: the user's only feedback channel is itself optimistic, and consistently in the same direction.
Suggested fix
Capture whatever reasoning/thinking token field the provider returns — for OpenRouter that means reading the reasoning-token field alongside completion_tokens — and include it in TokenTracker. Where a provider does not expose one, mark the reported figure as a lower bound rather than presenting it as exact. Recording usage on every turn of a tool-use loop, not only the last, would close the second candidate. A reconciliation test against a provider usage endpoint on one small run would keep this from regressing silently.
Observed on one full-pipeline run at production scale: a private TypeScript monorepo (Angular front end, NestJS back end), 1,351 analysis units from 866 source files, run with --verify and no --limit, ~6.5 hours wall clock. The engine was invoked directly (python -m openant scan …, Python 3.13); the Go CLI was not built, so none of this involves the Go↔Python envelope. Stage 1 detection and Stage 2 verification ran on Claude Opus 5; application context, enhancement and reporting on Claude Sonnet 5 — all seven phases routed through OpenRouter (anthropic/claude-opus-5, anthropic/claude-sonnet-5) via a locally-added openrouter provider adapter, on a build based on upstream 2ed78f6. Cost figures are expressed as proportions of the run total; absolute amounts are omitted deliberately, as is the identity of the scanned repository.
Summary
Reported run cost was ~15% below the amount actually billed by the provider. The shortfall is not uniform: accounting reconciled almost exactly across the Claude Sonnet 5 enhance phase, and the entire gap accumulated across the two phases that ran on Claude Opus 5 (Stage 1 detection and Stage 2 verification).
Evidence
Reconciliation was performed twice against the provider's own usage endpoint during a single run:
Those two reasoning-model phases accounted for ~91% of reported run cost, implying roughly a 19% undercount on them specifically. Token counts were recorded and appeared plausible; only the derived cost was short.
What I can and cannot attribute it to
Both models in this run perform extended thinking by default, and reasoning tokens bill as output tokens — so the mere presence of thinking does not explain why one phase reconciled and the others did not. That rules out the first hypothesis I reached for. Two candidates survive the observation:
usage.completion_tokensomits reasoning tokens (OpenRouter surfaces them separately for some models), then any undercount scales with how much a model actually reasons. Stage 1 and Stage 2 put open-ended exploitability questions to Claude Opus 5; the enhance phase asks Claude Sonnet 5 for a short classification. That would concentrate the error exactly where it appeared.I did not instrument either path, so please treat the measurement as the finding and both mechanisms as leads rather than diagnoses.
Why it matters
ARCHITECTURE.md§6 already documents "no spend ceiling —--limitcaps units, not dollars." A meter that under-reports by roughly a fifth on the model class most likely to be chosen for detection compounds that hazard: the user's only feedback channel is itself optimistic, and consistently in the same direction.Suggested fix
Capture whatever reasoning/thinking token field the provider returns — for OpenRouter that means reading the reasoning-token field alongside
completion_tokens— and include it inTokenTracker. Where a provider does not expose one, mark the reported figure as a lower bound rather than presenting it as exact. Recording usage on every turn of a tool-use loop, not only the last, would close the second candidate. A reconciliation test against a provider usage endpoint on one small run would keep this from regressing silently.Observed on one full-pipeline run at production scale: a private TypeScript monorepo (Angular front end, NestJS back end), 1,351 analysis units from 866 source files, run with
--verifyand no--limit, ~6.5 hours wall clock. The engine was invoked directly (python -m openant scan …, Python 3.13); the Go CLI was not built, so none of this involves the Go↔Python envelope. Stage 1 detection and Stage 2 verification ran on Claude Opus 5; application context, enhancement and reporting on Claude Sonnet 5 — all seven phases routed through OpenRouter (anthropic/claude-opus-5,anthropic/claude-sonnet-5) via a locally-addedopenrouterprovider adapter, on a build based on upstream2ed78f6. Cost figures are expressed as proportions of the run total; absolute amounts are omitted deliberately, as is the identity of the scanned repository.