Repository navigation
Conversation
A measurement phase: the default endpoint family stays `modern`, so no plugin behaviour changes and no characterization row flips. - Name the default in one constant, AUTO_ENDPOINT_FAMILY in src/utils/plane-model.ts, which is what `auto` resolves to, and pin it in test/endpoint-family.test.ts. - Extend the keyless matrix to three URL shapes (legacy, root, modern) over four protocols and two planes, add LEGACY_DISPATCH, and add one keyless free-model probe with its own pure classifier. - Add minimal keyed probes (pure test/support/keyed-probes.ts and its e2e): one tiny request per plane, protocol and shape, sent with the plane's own key and the plugin's identity from one shared constant, PROBE_IDENTITY. Only the cells the default sends to are asserted; the rest are recorded. - Gate the live job by tier with OPENCODE_E2E_KEYED (none, probes, full; unset means full). CI arms `probes` on a pull request that touches what the probes send, `full` nightly and on dispatch, and passes the secrets only when the tier is not `none`. - Write a dated endpoint-family record from both live suites to the run summary and an `endpoint-families` artifact, and add §13 to docs/redirect-planes.md with every cell pending the first CI run. The live part runs in CI only; no live result is invented here. A failing keyed modern cell in CI is the evidence for a later `fix:` that flips the default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
|
This PR didn't cause it. No fix exists yet. Regenerating the table needs Generated by Claude Code |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
What the review found, and what changed: - The keyed flip rule read a default cell's retired-model answer as a retired model without checking the same model on the other shapes. planeOutcome now reads each default cell beside the legacy and root cells of the same plane and protocol. - A refused key counted as flip evidence even when every shape refused it. It is now "check the CI secret" when a sibling refused the key and none accepted it, and "nothing else measured the key" only when no sibling refused it; a path-gone, rate-limited or busy sibling no longer blocks the CI-secret reason. - Breaks no flip could fix (accepted on the root shape only, or Go's Google, which no flip moves) were called flip evidence, and the "protocol" and "gone" reasons named causes nothing measured. A wrong-protocol or path-gone cell is now decided by the cell that endpoint: legacy dispatches to: "refused" (a flip would not serve it) when that cell refused readably, and always for Go's Google; "unmeasured" (re-run) when it measured nothing. Neither is flip evidence. - An account answer counted as the flip target accepting a protocol or a model, though the account's gate answers before either is read. It now counts only beside a refused key or a gone path, and a retired model next to an account answer no longer fails the plane. - The keyless free probe failed on a busy upstream (503/529) that the keyed classifier treats as routed. It now moves on past a busy candidate, and its dated record no longer pairs a derived upstream-busy with the last retired candidate's model and status. - A busy upstream followed by no answer failed the keyed plane, against the verdict definitions in redirect-planes.md section 13. Section 13 now says what the code does (no answer on the last attempt is transport-error unless an earlier attempt was a 429), and a test pins that it fails as unreadable, not as flip evidence. - Comments and docs said P-R2 had already measured the families while section 13 is pending its first CI run; said models.opencode.ai declares the root shape, which the generated catalog contradicts; said legacy cells are asserted nowhere live; and said a keyed auth-refused means the plugin's own turns would be refused, though a default Go turn sends a different key. All corrected. - CONTRIBUTING's key and mirror bullets now cover the keyed probes. - The plan no longer claims P-R2 opens with a green pin, gives the conditional flip a phase, and says the keyed probes decide a plane from the default cells read beside their siblings, not alone, and that an unreadable answer is never flip evidence. - Tests now cover the tier wiring of the four keyed live suites, no probe following a redirect, the push-to-none arm and the publish and upload steps of the CI wiring, the free probe moving on past a busy upstream, and the record being rewritten after every cell. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
|
This phase's first reviews ran on the general model because the dedicated reviewer was unavailable at the time. It has now had the dedicated review: three rounds through three lenses (correctness, repository rules, test matrix). Rounds
Four pairs of upheld findings were the same defect reported by two lenses. Rounds 2 and 3 mostly reviewed the previous round's classifier fixes. Fixed
Declined
Check, unit suite, mock suite, build and bundle gate all pass on the commit. Commit: 3ccd70c Generated by Claude Code |
Ported from the stack's base (d13cbe2 on #105) so this pull request's own `catalog freshness` goes green now, rather than waiting for the base to be merged up. It no-ops once the merge from P-R1 brings the same change. It takes `src/catalog-data.ts` exactly as the nightly refresh produced it (#111). The counts that restate the catalog's size follow it, on this branch's own wording of those passages: - the Zen shim count in catalog.test.ts (82 -> 81); - redirect-planes.md: Zen served 82 -> 81, the no-SDK Zen cell 25 -> 24, and "0 of 82" -> "0 of 81"; - protocol-routing-and-merge.md: the per-protocol shares now read "of 81", with completions at 24; - both READMEs: the no-SDK row is 24, and the active total is 81. "127 listed" is unchanged, because a model the vendor retires stays listed as `status: "deprecated"`. (cherry picked from commit d13cbe2) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
…r1' into claude/vigilant-mendel-bctz9n-p-r2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
…t-mendel-bctz9n-p-r2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
…r1' into claude/vigilant-mendel-bctz9n-p-r2 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
main's coverage gate (#142) checks bases; this adds its per-request form over the cells either endpoint setting dispatches to, with the bundled catalog's models and each protocol's own path. The root shape is the converse: nothing dispatches there, so a mock that routed it could hide a turn sent to the wrong family, and every root cell must answer no_route. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
…ins report P-R2 measures before any code moves, so each lesson becomes a record, not an assertion: - a 429 FreeUsageLimitError is the free tier's usage cap, its own verdict: never retried, never read as served, a skip (L3); main's live-run verdicts read it as the tier closing, not a busy gateway; - the System One family's three shapes on the Zen plane, keyless (L13); - keyed variants of each plane's default completions cell: a minted x-opencode-request (L4), x-opencode-project as global, a folder name and a 40-hex digest (L5), and reasoning_effort "none" beside no effort (L10); - every keyed 429's Retry-After (L9). The record formatter, section 13 and the docs follow; P6 and P-R3 decide. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
P-R1's derivation said the matrix did not probe the root family yet; on P-R2 it does, so a default turn reaching a root cell would match it rather than fall through as unmatched. The case now says so and asserts that no default cell is a root one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
The pins handed the route the rc channel's own pi-ai default window and output tokens, which reads as the plugin restating them. P-R1 removed that (D17, D26) and its plane-model tests use stand-in values; the family a turn is sent to does not depend on them, so these pins follow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
…h the P-R2 restack The probe classifiers now read a refusal of type ModelDeprecated as a withdrawn model at any status. The per-request mock-coverage port requires each dispatched cell to reach its own protocol's endpoint. The usage-cap verdict change now reaches the walk and the prose that describe it. - both-kept-1-F1 / resolutions-1-PR2-1 (fixed): one shared predicate, isModelDeprecated (status >= 400 and a word-bounded ModelDeprecated), read right after the redirect arm and before the 404, 410 and 400 arms in classifyKeylessProbe, classifyFreeProbe and classifyKeyedProbe. A retired free model now moves the free probe on instead of failing it. A keyed 404 or 400 of that type is no longer path-gone or path-exists. Classifier rows (400, 404 and 410 gone, a ModelDeprecatedish negative, a 2xx naming the word) and a runFreeProbe move-on row were added. Each arm, the status guard and the word boundary were mutation-checked, and each mutant failed its rows. - both-kept-1-F2 / resolutions-1-PR2-2 (fixed): a dispatched cell must be recorded with its own protocol and plane, and not as a method refusal; the root converse is unchanged. Three mutants turn the case red: narrowing the Google pattern, dropping the Messages pattern, and relabelling the completions endpoint. Under the old form the Google mutant left the case green. - both-kept-1-F4 / rules-docs-1-R2-RD-1 (3) (fixed): verdict.ts gains freeUsageCapped and freeWalkGap. The keyed free-tier walk skips with a reason that names the usage cap, instead of reporting a session gate it did not observe. probe-verdict rows were added, and the cap and busy arms were mutation-checked. - resolutions-1-PR2-3 / rules-docs-1-R2-RD-1 (1, 2) (fixed): the accessDisabled JSDoc reads a 429 as load or the usage cap. docs/testing.md rule 1 names the cap as a closed tier and a busy gateway as any other 429. - both-kept-1-F3 / rules-docs-1-R2-RD-2 (fixed): the plan's tier table and matrix line and ci.yml's tier comment name the System One record, the six keyed variants and Retry-After, and the free probe's usage-cap skip. - rules-docs-1-R2-RD-3 (fixed): the plan credits P-R1 with the turn pre-filter and P-R2 with the probe classifiers, in both its record and its restack row. The section 13 legends and CONTRIBUTING name the type. - rules-docs-1-R2-RD-4, rules-docs-1-R2-RD-5 (declined here): both are inherited from main and touch only the README pair, which this test-only phase does not change. They are recorded as a follow-up for a docs PR to main or the lowest open branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
…rose findings - both-kept-2-PR2-R3-1, rules-docs-2-R2-RD-N1: the plan's restack bullet now states the tested rule, that the free walk skips on the cap when ANY candidate met it and none met a busy gateway (busy, then capped, then closed), not when every candidate did. - resolutions-2-R2-1: freeWalkGap keeps its "any" rule; the capped skip's reason now comes from freeWalkCappedReason, which also names how many candidates answered 403 FreeTierError, so a mixed walk still prints the session gate. The probe-verdict row for it was written first and failed without the helper. The freeUsageCapped JSDoc and the test comment say "a walk that met the cap", matching the rule. - both-kept-2-PR2-R3-2: the combined Google User-Agent item is closed offline, not left open: the fetch patch replaces the User-Agent on the request every protocol sends, and the mock pins Google's wire value to OPENCODE_UA on both planes. redirect-planes.md section 9 says so, and the plan's restack bullet records it, reopening only if P10a moves the agent into call options. - both-kept-2-PR2-R3-3, rules-docs-2-R2-RD-N2: the stray "* *" in the keyed-probes module doc is a paragraph break again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0141fKp5k6hiaJcP5ovhGtSD
|
Restacked onto main 1.8.0. This branch now carries Conflicts and how they were resolvedSource
Test support
Tests and e2e
CI
Docs
Follow-ups added in this restack
Dropped
Review rounds
Round 1 found:
Round 2 found:
For the round-2 fixes:
Live re-measures still owed
The dedicated reviewer was unavailable, so every review and verify pass in this run used the general model. That covers all three rounds of the resolutions, both-kept and rules-docs reviews, and both verify passes. Generated by Claude Code |
Phase P-R2 of docs/improvement-plan.md, stacked on #113.
P-R2 is a measurement phase. The default endpoint family stays
modern, so the plugin's behaviour does not change and noS0_TODAYor characterization row flips. The title carries no!and the commit has no breaking footer.What changed
AUTO_ENDPOINT_FAMILY = "modern"is whatautoresolves to. Before this, the choice sat inside a ternary. A later flip is a one-word edit, and the constant is pinned.src/utils/plane-model.ts,test/endpoint-family.test.ts/zen(/go)/v1, a newroot/inference(/go)/v1, and modern/inference(/go)/{openai,anthropic,google}/…. That is 24 cells over 4 protocols and 2 planes. It addsLEGACY_DISPATCH(the cellsendpoint: legacyreally sends to) and one keyless free-model probe with its own pure classifier.classifyKeylessProbeis unchanged.test/support/keyless-matrix.ts,test/e2e/keyless-matrix.e2e.tsPROBE_IDENTITY, and an offline test pins it against the identity a real turn sends. Only the cells the default sends to (TODAY_DISPATCH) are asserted. Every other cell is recorded.test/support/keyed-probes.ts,test/e2e/keyed-probes.e2e.ts,test/keyed-probes.test.tsOPENCODE_E2E_KEYEDtakesnone,probesorfull. Unset meansfull, so local runs behave as before. CI picks the tier per event:probesfor a pull request touchingsrc/host/**,protocols.ts,plane-model.ts,identity.ts,request-shape.tsor the probes' own inputs;fullnightly and on dispatch;noneotherwise. Secrets are passed only when the tier is notnone. The full keyed suites read their keys througharmedKey(tier, "full", …)..github/workflows/ci.yml,test/e2e/*.e2e.ts$OPENCODE_E2E_RECORD_DIR. The job publishes it to the run summary and to anendpoint-familiesartifact.test/support/endpoint-record.ts,test/endpoint-record.test.tsdocs/redirect-planes.md§13 uses the same columns, and every cell reads "pending the first CI run". A unit test holds the doc's tables to the formatter's columns and row order. The tier scope, the skip reasons and the plan's status row are updated.docs/redirect-planes.md,docs/improvement-plan.md,CONTRIBUTING.md,AGENTS.mdDecision recorded in the plan:
endpoint: legacyis recorded, not guarded, in the live job. Unit tests pin its mapping of every protocol on both planes. If a keyed modern cell fails in CI, that is the evidence for a laterfix:that flips the default.Behaviour changes
Plugin: none. The changes below are to the test suite and CI only:
PROBE_IDENTITYintest/support/keyed-probes.ts. What they send live is unchanged: user-agentOPENCODE_UA,x-opencode-client,x-opencode-project, and the probe session id under both session headers. They deliberately leave outx-opencode-request, the per-request metric id.probeskeyed tier on a pull request that touchessrc/utils/request-shape.ts, because every Messages probe sends itsanthropic-version. This is in addition tosrc/host/**,protocols.ts,plane-model.ts,identity.tsand the probes' own inputs.PLUGIN_HEADERStoOPENCODE_UA. That user agent reaches the suite's keyless free-tier and unmetered cases on every tier, so on every pull request and every push to main. This PR's own e2e run is therefore the first live observation of the new user agent on keyless traffic, before the nightly run.docs/credentials.mdrecords that this user agent did not change the free-tier verdict.Review
Each round ran three reviewers (correctness, tests, rules) with none missing. Every upheld finding was fixed in the next round.
not_found_error; apipefailhazard in the tier steprequest-shape.tsRound-4 mutation checks were each run, watched fail, then restored:
PROBE_IDENTITYbound to the patch UA: 4 failed.UNAVAILABLEand "model is overloaded" alternatives: 2 failed.UNAVAILABLEmatch: 1 failed.Verification
pnpm run checkvp test runvp test run --config vitest.mock.config.tspnpm run buildscripts/check.tscheck passed, including "lib/ carries the current build, including the served client bundle"vp test run --coveragegit diff, the 6 new files andlib/sk-probe-0123456789abcdefThe first five Node 26 rows were re-run on the final tree right before the commit. The live keyed and keyless results are not in this PR: they come from this branch's CI run, and §13 stays "pending" until then.
Known red check:
catalog freshnessfails on every branch until the vendor-catalog refresh lands on main. It does not come from this change.Restacked onto main 1.8.0
The branch now carries
main1.8.0 through its parent, #113 (P-R1), which mergedmain. P-R2 merged P-R1; it did not mergemainitself.Scope after the restack
test:measure the endpoint families against the live gateway, on both harness channels.AUTO_ENDPOINT_FAMILY(modern) is the one valueautoresolves to.test/endpoint-family.test.tspins the bases of the default and oflegacyfor every plane × protocol, and the cells each setting reaches (TODAY_DISPATCH,LEGACY_DISPATCH). Since P-R1 these are the same on every host cell. No behaviour change, and no characterization row flips.OPENCODE_E2E_KEYED(none/probes/full). They add the keyed variants of the default completions cell:x-opencode-request(L4),x-opencode-projectas global, folder or 40-hex (L5), andreasoning_effort: "none"(L10). They also record every 429'sRetry-After(L9).429 FreeUsageLimitErroris nowfree-usage-limitedand is never retried (L3). It is not a rate limit, and it is notfree-tier-gated, the name already used for the 403 session gate.main's test(mock): the mock must serve every base the plugin can produce #142 mock-coverage gate, ported to work per request.endpoint-familiesartifact, for §13.fix:) pastes the first CI run into §13. It flips the default only if that run shows flip evidence.Dropped, because main or the parent now does it
ModelDeprecatedas a P-R2 gone rule. P-R1 brought it withlive-probe.ts's gone rule. After the merge,keyless-matrix.ts'sisModelGoneMessagecarries it, so there is nothing left for P-R2 to add./inference/v1" (L3). P-R2 already probed the root shape as one of its three. The only new URL is System One's root.free-tier-gatedfor the usage cap. That name already means a 403FreeTierError(a pass). The cap isfree-usage-limited(a skip).test/mock/protocol-compat.mock.tspins Google's wire value toOPENCODE_UAon both planes.docs/redirect-planes.md§9 now says so.Decisions applied
rcandalpha, are named once. The dispatch set and the binary e2e cover both channels.patches/is gone from the CI seam regex.baseUrlFortakes no pi argument, andlegacyreads the fixed table. Test fixtures restate none of pi-ai's defaults, sotest/endpoint-family.test.tsnow uses stand-in window and output values.Live re-measures CI must confirm
probestier, because the PR touchessrc/utils/plane-model.tsandci.yml. Paste its keyless and keyed tables into §13 (P-R2b). If a modern cell shows flip evidence, flipAUTO_ENDPOINT_FAMILY, together with the copy,TODAY_DISPATCHand the re-tagged wire facets.free-tier-gatedorfree-usage-limited);/inference(/go)/v1cells;x-opencode-requestpresent vs absent (L4);x-opencode-projectas global, folder or 40-hex (L5);Retry-Afteron Go's 429s (L9);reasoning_effortnonevs absent (L10)./zen/v1cells still answer for plane models under the current gate.KNOWN_DEFECT).HOST_LIMITATION).Generated by Claude Code