test: pin the client provider mirrors against their server originals (#5673) - #5812
Merged
Conversation
…5673) `client/src/utils/providers.js` hand-mirrors three server modules plus the vendored toolkit, and every one of those mirrors was pinned by prose alone — each side carried its own hardcoded expectations, so a row added to one copy left both suites green. That is user-visible in both directions: an effort level the picker offers but the server's ladder rejects gets clamped or dropped, and a tool-use family the server recognises but the client doesn't puts a "no known tool use" warning on a model that works fine. Three parity suites now read the client file as TEXT (never import it, so no client deps leak into the server CI job): - `providerModels.mirror.test.js` pins the six effort ladders, `EFFORT_RANK`, `CODEX_ULTRA_MODELS`, `CONFIGURED_DEFAULT_SENTINELS` and the Antigravity effort-suffix split. - `localModelHeuristics.mirror.test.js` pins the embedding / vision / tool-use capability regexes. The server spells them as a commented array of alternatives and the client as one inline literal, so declaration text can't be diffed — the new `compareRegexDeclaration` primitive compares what each pattern MATCHES instead. The same suite pins the toolkit's third `TOOL_USE_RE` copy as text. - `providerGateways.parity.test.js` gains the client registry as a third leg, scoped to the `id`/`label`/`apiKeyEnv`/`legacyMarker` fields the browser actually carries (it omits `baseURL`/`legacyApiKeyField`, so `toEqual` against the server rows can't be used). Also refreshes the mirror pointers that had already rotted to `server/lib/stageRunner.js`, a path that has not existed since the module moved to `server/services/`. Claude-Session: https://claude.ai/code/session_01GMxEz43s3YCLaVZV9KmVwE
…n't account for (#5673) Review pass on the new parity guards. Each reader scanned its input for the parts it recognised and ignored the rest, so a mirror that had genuinely changed could still be reported intact — the worst possible outcome for a guard test. All three now consume their input to the end and return null on any leftover: - the alternative reader accepts only a comma-separated run of string literals, so `[...SHARED, 'extra']` or a named fragment can no longer silently vanish from the comparison; - escapes other than `\\` and `\'` are refused rather than copied through as their own text — `'\x2e'` is `.`, which matches anything; - both the capability-regex and gateway-table readers are anchored to the WHOLE declaration, so a decoy array or row list left inside a wrapped or conditional assignment no longer reads as the exported value; - gateway rows reject spreads, computed values and duplicate keys. Each rule gets a probe asserting the reader returns null rather than a partial read. Claude-Session: https://claude.ai/code/session_01GMxEz43s3YCLaVZV9KmVwE
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
client/src/utils/providers.jshand-mirrors three server modules(
providerModels.js,localModelHeuristics.js,providerGateways.js) plus thevendored toolkit's
TOOL_USE_RE, and every one of those mirrors was pinned by aprose "keep in lockstep" comment. Each side carried its own hardcoded
expectations, so a row added to one copy alone left both suites green — the same
hole
stageRunner.mirror.test.jswas written to close for the context-windowtable.
It isn't cosmetic in either direction: an effort level the picker offers but the
server's ladder doesn't accept gets clamped or dropped, and a tool-use family the
server recognises but the client doesn't puts a "no known tool use" warning on a
model that works fine.
Three parity suites now pin them. All read the client file as TEXT with
readFileSyncand never import it, so the client's dependency tree stays out ofthe server CI job — which is what makes the "deliberately NOT imported" carve-out
in
providerGateways.parity.test.jsunnecessary.server/lib/providerModels.mirror.test.js(new) — the six effort ladders,EFFORT_RANK,CODEX_ULTRA_MODELS,CONFIGURED_DEFAULT_SENTINELSand theAntigravity effort-suffix split. All ten are byte-identical today.
server/lib/localModelHeuristics.mirror.test.js(new) — the embedding /vision / tool-use capability regexes. The server spells them as a commented
array of alternatives and the client as one inline literal, so declaration TEXT
can't be diffed: a new
compareRegexDeclarationprimitive compares what eachpattern MATCHES instead. Forcing one side to adopt the other's typesetting
would be churn for no behavioural gain. The same suite pins the toolkit's third
TOOL_USE_REcopy, which is the array form on both sides, as text.server/lib/providerGateways.parity.test.js— gains the client registry asa third leg, scoped to the
id/label/apiKeyEnv/legacyMarkerfieldsthe browser actually carries. It omits
baseURLandlegacyApiKeyField(itnever dials the gateway and never handles the key), so
toEqualagainst theserver rows can't be used.
The readers fail closed: an array element that isn't a string literal, an
escape that can't be decoded by inspection (
'\x2e'is., which matchesanything), a gateway row carrying a spread or a computed value, or a recognisable
shape sitting inside a wrapped/conditional assignment all return
nullratherthan a partial read — a guard that reports a diverged mirror as intact is worse
than no guard.
Also refreshes the mirror pointers that had already rotted to
server/lib/stageRunner.js, a path that has not existed since the module movedto
server/services/, and points each "keep in lockstep" comment at the suitethat now enforces it.
Test plan
cd server && npm test— 1836 files / 37310 tests green.cd client && npm test— 855 files / 10719 tests green;npm run lintclean.TOOL_USE_RE,dropping
maxfrom the client'sCURSOR_EFFORT_LEVELS, and renaming a clientgateway label each fail exactly the corresponding parity test, with a message
naming the declaration and stating which copy is authoritative.
null(not a partial read) for aspread element, an
\x/\nescape, a duplicate or computed row field, and adecoy array inside a wrapped assignment.
Note:
services/voice/fineTuning.test.jsflaked twice across full-suite runs onthis branch (grace-window timers) and passes in isolation and on a clean re-run;
it is untouched by this diff.
Closes #5673
https://claude.ai/code/session_01GMxEz43s3YCLaVZV9KmVwE