Skip to content

test: pin the client provider mirrors against their server originals (#5673) - #5812

Merged
atomantic merged 2 commits into
mainfrom
claim/issue-5673
Sep 2, 2026
Merged

test: pin the client provider mirrors against their server originals (#5673)#5812
atomantic merged 2 commits into
mainfrom
claim/issue-5673

Conversation

@atomantic

Copy link
Copy Markdown
Owner

Summary

client/src/utils/providers.js hand-mirrors three server modules
(providerModels.js, localModelHeuristics.js, providerGateways.js) plus the
vendored toolkit's TOOL_USE_RE, and every one of those mirrors was pinned by a
prose "keep in lockstep" comment. Each side carried its own hardcoded
expectations, so a row added to one copy alone left both suites green — the same
hole stageRunner.mirror.test.js was written to close for the context-window
table.

It isn't cosmetic in either direction: an effort level the picker offers but the
server's ladder doesn't accept gets clamped or dropped, and a tool-use family the
server recognises but the client doesn't puts a "no known tool use" warning on a
model that works fine.

Three parity suites now pin them. All read the client file as TEXT with
readFileSync and never import it, so the client's dependency tree stays out of
the server CI job — which is what makes the "deliberately NOT imported" carve-out
in providerGateways.parity.test.js unnecessary.

  • server/lib/providerModels.mirror.test.js (new) — the six effort ladders,
    EFFORT_RANK, CODEX_ULTRA_MODELS, CONFIGURED_DEFAULT_SENTINELS and the
    Antigravity effort-suffix split. All ten are byte-identical today.
  • server/lib/localModelHeuristics.mirror.test.js (new) — the embedding /
    vision / tool-use capability regexes. The server spells them as a commented
    array of alternatives and the client as one inline literal, so declaration TEXT
    can't be diffed: a new compareRegexDeclaration primitive compares what each
    pattern MATCHES instead. Forcing one side to adopt the other's typesetting
    would be churn for no behavioural gain. The same suite pins the toolkit's third
    TOOL_USE_RE copy, which is the array form on both sides, as text.
  • server/lib/providerGateways.parity.test.js — gains the client registry as
    a third leg, scoped to the id / label / apiKeyEnv / legacyMarker fields
    the browser actually carries. It omits baseURL and legacyApiKeyField (it
    never dials the gateway and never handles the key), so toEqual against the
    server rows can't be used.

The readers fail closed: an array element that isn't a string literal, an
escape that can't be decoded by inspection ('\x2e' is ., which matches
anything), a gateway row carrying a spread or a computed value, or a recognisable
shape sitting inside a wrapped/conditional assignment all return null rather
than a partial read — a guard that reports a diverged mirror as intact is worse
than no guard.

Also refreshes the mirror pointers that had already rotted to
server/lib/stageRunner.js, a path that has not existed since the module moved
to server/services/, and points each "keep in lockstep" comment at the suite
that now enforces it.

Test plan

  • cd server && npm test — 1836 files / 37310 tests green.
  • cd client && npm test — 855 files / 10719 tests green; npm run lint clean.
  • Mutation-checked each leg: adding an alternative to the server's TOOL_USE_RE,
    dropping max from the client's CURSOR_EFFORT_LEVELS, and renaming a client
    gateway label each fail exactly the corresponding parity test, with a message
    naming the declaration and stating which copy is authoritative.
  • Fail-closed probes assert the readers return null (not a partial read) for a
    spread element, an \x/\n escape, a duplicate or computed row field, and a
    decoy array inside a wrapped assignment.

Note: services/voice/fineTuning.test.js flaked twice across full-suite runs on
this branch (grace-window timers) and passes in isolation and on a clean re-run;
it is untouched by this diff.

Closes #5673

https://claude.ai/code/session_01GMxEz43s3YCLaVZV9KmVwE

…5673)

`client/src/utils/providers.js` hand-mirrors three server modules plus the
vendored toolkit, and every one of those mirrors was pinned by prose alone —
each side carried its own hardcoded expectations, so a row added to one copy
left both suites green. That is user-visible in both directions: an effort
level the picker offers but the server's ladder rejects gets clamped or
dropped, and a tool-use family the server recognises but the client doesn't
puts a "no known tool use" warning on a model that works fine.

Three parity suites now read the client file as TEXT (never import it, so no
client deps leak into the server CI job):

- `providerModels.mirror.test.js` pins the six effort ladders, `EFFORT_RANK`,
  `CODEX_ULTRA_MODELS`, `CONFIGURED_DEFAULT_SENTINELS` and the Antigravity
  effort-suffix split.
- `localModelHeuristics.mirror.test.js` pins the embedding / vision / tool-use
  capability regexes. The server spells them as a commented array of
  alternatives and the client as one inline literal, so declaration text can't
  be diffed — the new `compareRegexDeclaration` primitive compares what each
  pattern MATCHES instead. The same suite pins the toolkit's third
  `TOOL_USE_RE` copy as text.
- `providerGateways.parity.test.js` gains the client registry as a third leg,
  scoped to the `id`/`label`/`apiKeyEnv`/`legacyMarker` fields the browser
  actually carries (it omits `baseURL`/`legacyApiKeyField`, so `toEqual`
  against the server rows can't be used).

Also refreshes the mirror pointers that had already rotted to
`server/lib/stageRunner.js`, a path that has not existed since the module
moved to `server/services/`.

Claude-Session: https://claude.ai/code/session_01GMxEz43s3YCLaVZV9KmVwE
…n't account for (#5673)

Review pass on the new parity guards. Each reader scanned its input for the
parts it recognised and ignored the rest, so a mirror that had genuinely
changed could still be reported intact — the worst possible outcome for a
guard test. All three now consume their input to the end and return null on
any leftover:

- the alternative reader accepts only a comma-separated run of string
  literals, so `[...SHARED, 'extra']` or a named fragment can no longer
  silently vanish from the comparison;
- escapes other than `\\` and `\'` are refused rather than copied through as
  their own text — `'\x2e'` is `.`, which matches anything;
- both the capability-regex and gateway-table readers are anchored to the
  WHOLE declaration, so a decoy array or row list left inside a wrapped or
  conditional assignment no longer reads as the exported value;
- gateway rows reject spreads, computed values and duplicate keys.

Each rule gets a probe asserting the reader returns null rather than a
partial read.

Claude-Session: https://claude.ai/code/session_01GMxEz43s3YCLaVZV9KmVwE
@atomantic
atomantic merged commit 516b1a0 into main Sep 2, 2026
7 checks passed
@atomantic
atomantic deleted the claim/issue-5673 branch September 2, 2026 05:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Pin the client provider mirrors against their server originals instead of trusting "keep in lockstep" comments

1 participant