Skip to content

A missing config/models.json record silently prices a model at zero #216

Description

@NahumKorda

Summary

Running a current-generation model whose slug is absent from config/models.json produces a full scan in which every phase reports a cost of zero. Token counts are recorded correctly; only the monetary total is blind. The model dispatches normally.

Evidence

An entire pipeline run reported cost_usd: 0.0000 for all seven phases because the registry contained no record for anthropic/claude-opus-5 or anthropic/claude-sonnet-5 under the openrouter provider. Adding records restored accurate reporting, reconciling to 0.12% against billed spend on the Claude Sonnet 5 phase.

Why it matters

The registry's own header comment states the design intent:

price is null for any model that must not be auto-dispatched (retired/unknown) so it never resolves to a silent $0 in cost accounting.

A missing record defeats that guard entirely — the model is dispatched and priced at zero, which is the exact outcome the null-price convention was introduced to prevent. The guard covers models the registry knows about and deliberately refuses to price, but not models it has never heard of, and the second case is the common one as new models ship.

Suggested fix

Two independent parts:

  1. Add records for the current-generation slugs (anthropic/claude-opus-5, anthropic/claude-sonnet-5, and anthropic/claude-fable-5 were all missing). Rather than embed rates that will drift, they are worth taking from the live catalogue at the time of the change (GET /api/v1/models on the gateway exposes per-token rates; the registry wants per-million).
  2. Make an unpriced-but-dispatched model as loud as a null-priced one — emit a per-run warning and mark the reported cost as incomplete, instead of silently summing zeros.

Separately worth considering: records whose rate is a time-limited introductory price go stale silently. An optional expires field would make staleness detectable.


Observed on one full-pipeline run at production scale: a private TypeScript monorepo (Angular front end, NestJS back end), 1,351 analysis units from 866 source files, run with --verify and no --limit, ~6.5 hours wall clock. The engine was invoked directly (python -m openant scan …, Python 3.13); the Go CLI was not built, so none of this involves the Go↔Python envelope. Stage 1 detection and Stage 2 verification ran on Claude Opus 5; application context, enhancement and reporting on Claude Sonnet 5 — all seven phases routed through OpenRouter (anthropic/claude-opus-5, anthropic/claude-sonnet-5) via a locally-added openrouter provider adapter, on a build based on upstream 2ed78f6. Cost figures are expressed as proportions of the run total; absolute amounts are omitted deliberately, as is the identity of the scanned repository.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions