AsyncAPI 2.0.0-3.1.0 support: declare the range, fix three ingestion defects - #2
Merged
Conversation
Records the brainstormed design for declaring and hardening the AsyncAPI
version range Offbook supports, alongside three ingestion defects found by
research in the path that serves it:
- a valid 3.x multi-format payload (`{schemaFormat, schema}`) silently
collapses the channel validator to accept everything, including null
- a legal draft-07 tuple payload schema crashes registry construction under
the 2020-12 stamp, uncaught, out of bootProject
- 2.x MQTT bindings are unvalidated, so `qos: 9` reaches a Channel typed 0|1|2
Decisions: harden the existing dual path rather than convert on ingest;
Offbook enforces its own supported-version gate; payload validation moves to
draft-07 (the dialect both majors declare and the parser emits) paired with a
dialect-mismatch diagnostic; binding policing is a value guard plus a key-set
check sourced from the official binding schema. Closes D-005's deferred
2020-12 obligation as superseded, retaining its tripwire.
Parser floor moves to ^3.6.0: 3.4.0 and 3.5.0 resolve a specs package
containing the 3.1.0 schema, pass their own dynamic version gate, then die
inside the Spectral ruleset. Every behavioral claim was verified empirically
against the installed toolchain; results are tabulated in the doc.
Not yet implemented; D-018 and R-037..R-039 are forward references.
Ten TDD tasks over docs/superpowers/specs/2026-07-30-asyncapi-version-support-design.md: the supported-version contract and preflight gate, the payload schema boundary (multi-format unwrap, draft-07 dialect, mismatch diagnostic, compile containment, multi-message anyOf), mqtt binding integrity, spec-version provenance, two new fixtures, and the R-028 gate extension. Every spec literal in the plan was parse-verified against @asyncapi/parser 3.6.0 first. Two spec claims did not survive planning and are recorded as deviations: new diagnostics reuse the existing closed `spec-load` kind with a tagged detail prefix rather than adding six kinds to a union that DiagnosticSummary.byKind must zero-fill, and `offbook doctor` cannot report spec-version because its checks are network-free while the spec lives in a remote repo.
Refuse any spec outside the tested set (2.0.0-2.6.0, 3.0.0, 3.1.0) with a branded error naming the version, the range, and the convert remedy, checked parser-free before parse(). Raise the parser floor to ^3.6.0: 3.4.0/3.5.0 pass their own dynamic version gate on a 3.1.0 doc and then fail inside Spectral. Records D-018 and R-037..R-039.
A valid AsyncAPI 3.x payload authored as {schemaFormat, schema} produced a
channel schema with no validation keywords, so it accepted everything including
null and bare strings: a silent false negative on a supported version. Extract
the inner schema, and pin it with a multi-format.yaml fixture whose garbage
payloads must be rejected on both the send and receive paths.
Two inaccuracies found while executing Tasks 1 and 2: - Touching fixtures/asyncapi/ couples to the R-027 jsf-fidelity tripwire, which asserts SPIKE_FIXTURES covers the fixture listing exactly and pins per-fixture draw counts. Adding a fixture (Task 2, Task 8) or adding a channel to one (Task 3) makes the full suite red until both files are updated. Recorded as a global constraint plus explicit steps in Tasks 3 and 8, with instructions to re-measure rather than assume and to never relax failures: 0. - Task 1 Step 1 specified the #src/ alias for a same-directory test import; D-013 keeps those relative. Corrected, with the contrast against the genuine upward reach in Step 5 spelled out.
Both spec majors define the Schema Object as a draft-07 superset and the parser emits draft-07, so stamping 2020-12 was the root cause of a legal tuple schema crashing ajv.compile() and of additionalItems being silently ignored. Switch to Ajv's draft-07 build and stamp the dialect explicitly, so channel.schema (which GET /topics hands out) is self-describing. Verified the external-ref correctness bar and the D-005 sibling tripwire behave identically: the external pattern is still enforced, the sibling minLength still is not. Both are reworded from "known limitation" to dialect-correct behavior, pinned so a future dialect change is loud. Reverses the 2020-12 statement in contracts.md §5 (canonical) and closes D-005's deferred 2020-12 obligation as superseded. Deviation from the plan: the tuple regression case lives as an INLINE test spec rather than as a composition.yaml channel. Implementation measured that json-schema-faker 0.6.2 cannot draw a valid draft-07 tuple at all (it emits objects with numeric keys and fails the Ajv recheck 10/10 seeds, unchanged by additionalItems or minItems), so a tuple fixture would fail the R-027 faker-floor spike and flip D-008's measured verdict. That is a separate question from the dialect, so it is recorded in D-018 and left open rather than resolved here.
Steps 1 and 1b (a tuple channel in composition.yaml, plus its re-measured spike expectation) were not taken: json-schema-faker 0.6.2 cannot draw a valid draft-07 tuple, so a tuple fixture fails the R-027 faker floor and would flip D-008's verdict. The case is inline instead. Recorded so the plan matches what shipped.
Task 3 reversed the canonical dialect statement and closed D-005 as superseded, but fixtures/asyncapi/README.md and REQUIREMENTS.md R-004 still assert the old known-limitation story and now contradict contracts.md. Neither was listed in any step. Added as Task 10 Step 5b with a grep to catch remaining stale claims.
…-039) Registry-time findings cannot be recomputed from Channel, so SpecRegistry gains diagnostics(), merged by the composition root beside the computed ones. First consumer: an mqtt CHANNEL binding, which is illegal per the binding spec at every version yet parses clean, is now reported as ignored instead of silently falling back to the default qos. The kind union stays closed: these findings reuse `spec-load` with a machine-greppable `detail` tag prefix, and DiagnosticSummary.byKind keeps exactly its four zero-filled keys. Widening the interface also required `diagnostics: () => []` on the twelve SpecRegistry fakes in control-plane, engine (dispatch/index/instances/reset), scenarios (index/loader) and test/cli-dispatch.test.ts.
Draft-07 ignores prefixItems, unevaluatedProperties, dependentRequired and friends rather than erroring, which would under-validate silently. Name every offending keyword per channel. $defs and definitions are excluded: both work under draft-07 and shared/common.yaml uses $defs.
ajv.compile() threw out of buildRegistry uncaught, surfacing as a raw Ajv error from bootProject with no channel named. Wrap it: the channel still enters the catalog, because discovery is a v1 floor, but every payload reports one explicit offbook:schema-compile-failed violation rather than validating green, and the failure is surfaced as a spec-load diagnostic naming the topic.
Only messages()[0] was compiled, so a v2 message.oneOf union or a v3 multi-message operation reported every non-first variant as a contract violation. anyOf keeps Channel.schema singular, leaving the faker and /topics untouched.
AsyncAPI 2.x maps mqtt to an empty schema, so qos: 9 and a misspelled retian parsed with zero errors and the bad qos reached a Channel typed 0 | 1 | 2. Guard the two values offbook consumes, falling through the precedence chain on a bad one, and report unknown keys against the official binding schema's key set plus MQTT-5-only fields as unhonored. Adds a 2.0.0 fixture pinning the range floor.
Read the asyncapi field in the same parser-free pass as info.version and record it as spec-version in specs.lock and on SpecInfo, so GET /v1/specs answers which spec major each service is on. doctor deliberately does not report it: its spec checks are network-free while the spec lives in a remote repo.
Extend the R-028 validation gate over multi-format.yaml and v2-oldest.yaml so the false-negative class is covered by a v1 gate, flip the three requirements to tested with traces, and state the supported range for adopters. The wiring guide gains the 2.x perspective caveat: a spec authored from the client's point of view loads with every direction inverted and no tool can detect it. Also finishes the D-005 sweep left by Task 3: the fixtures README row and R-004's prose still told the old known-limitation story, which contradicted the canonical draft-07 statement in contracts.md. AGENTS.md's status line is refreshed for the three new requirements.
Amends the 2026-07-30 design on three mechanisms, from PR #2 review findings 1-3: - the preflight fires only on a version it positively read and positively recognizes as untested, so malformed YAML, an empty file, and a fetched HTML error page reach the parser and surface its own diagnostics instead of a version error advising `asyncapi convert` - offbook's supported set is kept aligned with the parser's by an install-time drift test rather than a runtime post-parse recheck, which would be a permanently uncovered throw while the sets agree - @asyncapi/specs becomes a devDependency and the mqtt binding key set a hand-authored constant, extending to bindings the reasoning D-018 applied to SUPPORTED_SPEC_VERSIONS; the transcription includes the vendor-extension pattern the derived set was dropping Records two defects found while designing: an unquoted `asyncapi: 2.6` makes the parser die with a raw TypeError stack, and the derived key set was rejecting spec-legal `x-` extensions.
Biome 1.9's noRestrictedImports.paths matches whole module specifiers, so a bare "@asyncapi/specs" entry does not flag the deep JSON import the design is preventing (verified both ways; the exact path does flag). A lint rule would be a no-op against the realistic regression. Uses the repo's existing idiom instead (test/transport-isolation.test.ts for the aedes family, src/ingestion/index.test.ts for G12): walk src/**/*.ts and regex the import edge, which catches every subpath. biome.json is now untouched by this design.
Four tasks over the 2026-08-01 design: the preflight firing rule and message, the @asyncapi/specs devDependency move with hand-authored binding constants, the upstream-drift gate, and the requirement / decision / doc updates. Each task carries its own mutation check and ends on a green full gate set.
The preflight ran before parser.parse() and treated readSpecVersion's undefined as 'unsupported version', so malformed YAML, an empty file, and a fetched HTML error page all died with a version error advising `asyncapi convert`, which is the wrong remedy for all three. Spec-load failure aborts `up` in the foreground, so that message was the whole error an adopter saw (R-036). It now fires only on a version it positively read and positively recognizes as untested; everything else falls through to the parser, whose diagnostics already distinguish a YAML syntax error from a non-AsyncAPI document. An unquoted `asyncapi: 2.6` is still caught here, ahead of the parser's raw getSemver TypeError. The message names the tested set instead of '2.0.0 through 3.1.0', which would have told a 2.7.0 user they were supported while refusing them.
src/registry/ imported @asyncapi/specs/bindings/mqtt/0.2.0/operation.json to derive the legal mqtt operation-binding key set, but that package is a transitive dependency of @asyncapi/parser, not a declared one, so the import resolved only through flat hoisting. A parser upgrade that bumps or drops the dep, or an install under a non-hoisting layout, breaks the build. The subpath is also an unversioned internal file: the package publishes no exports map. The key set is now a hand-authored constant, extending to bindings the reasoning D-018 applied to SUPPORTED_SPEC_VERSIONS, with the package demoted to a devDependency for the drift gate that follows. Transcribing the schema by hand also surfaced what deriving it hid: only `properties` was being read, so the schema's patternProperties allowance for ^x- vendor extensions was dropped and a spec-legal x-vendor-thing was reported as an unknown key.
Three checks: the supported version set equals what @asyncapi/specs exposes, the mqtt operation-binding constants equal that schema's properties and patternProperties, and nothing under src/ imports @asyncapi/specs at any subpath. The first is what closes the gap the preflight leaves open. offbook refuses only versions it positively recognizes as untested, so a version the parser accepts and offbook does not would slip through; catching that at the dependency bump beats a runtime recheck, which would be a permanently uncovered throw while the two sets agree. The third is a source-text check rather than a lint rule because Biome's noRestrictedImports matches whole specifiers and cannot express a package at any subpath, which is the exact shape of the import it replaces.
test/upstream-drift.test.ts's "@asyncapi/specs stays a devDependency"
check only matched `from "@asyncapi/specs..."`. Its comment claimed
the same idiom as test/transport-isolation.test.ts, but that gate
also matches `require("...")` — this one didn't. Since a Biome
noRestrictedImports rule can't express "this package at any subpath"
(rejected for that reason, D-019), this regex is the sole enforcement
of the devDependency boundary, and Bun's global `require` resolves
deep JSON subpaths like @asyncapi/specs/bindings/mqtt/0.2.0/operation.json
even under "type": "module" — so a require() import under src/ would
have evaded the gate silently.
Hoist the pattern to a module-level const (matching
transport-isolation's shape) and extend it to match require(...) with
incidental whitespace, alongside the existing from "..." form.
R-037 said the preflight refuses any spec outside the tested set, which was too broad: it now refuses only a version it can read and has not tested, and defers the rest to the parser. R-039 said the key set comes from the official schema, which is now a hand-authored transcription behind a drift gate, honoring the vendor-extension pattern the derived set dropped. D-019 amends D-018 on the three mechanisms and records the unquoted `asyncapi: 2.6` parser TypeError found while designing.
readSpecVersion collapsed both an absent `asyncapi` field and a
present-but-null one (bare `asyncapi:`) to undefined. The absent case is
correctly not offbook's call — it defers to the parser's own diagnosis.
But the null case IS a positive read that the R-037 preflight should
judge, and letting it fall through instead sent it straight into
@asyncapi/parser, which TypeErrors on it inside getSemver
(`patchWithRc.split`) with a raw internal stack instead of the branded,
actionable refusal.
Now only a genuinely absent field yields undefined; any present value,
null included, is read via String() (kept deliberately unnarrowed: the
same parser TypeError also fires on '', true, and {}, so narrowing to
string/number would silently reopen the bug on those three inputs).
… go quiet
Four gaps in test/upstream-drift.test.ts, the gate that keeps offbook's
hand-authored version/binding constants aligned with @asyncapi/specs:
- The version-set comparison imports the ROOT @asyncapi/specs (our
devDependency), assuming that is the same copy @asyncapi/parser
resolves at runtime. They are deduped today, but nested copies are a
real phenomenon in this tree
(node_modules/@asyncapi/parser/node_modules/ajv-formats exists). If a
future parser bump ever carries its own nested @asyncapi/specs, the
comparison would silently run against the wrong copy at exactly the
moment a bump exists to catch something. Now asserts that nested copy
does not exist, before the comparison, so that scenario fails loud
instead.
- The mqtt binding-key check reads bindings/mqtt/0.2.0/operation.json,
a versioned (hence immutable) upstream artifact — it can never itself
detect drift, since that file will never change. Real drift arrives
as a NEW binding version directory, which may add legal keys that
offbook's key set doesn't yet know about, drawing a spurious
binding-unknown-key warning until the constant catches up. Now also
pins the binding version SET (readdirSync sorted equals
["0.1.0", "0.2.0"]), so a new version directory goes red at the bump.
- The devDependency-boundary regex matched `from "…"` and
`require("…")` but not `await import("…")`, even though dynamic
import is a live idiom in this codebase (src/engine/dispatch.ts).
Added an import(...) alternative with the same whitespace tolerance
as require(...). Verified by temporarily adding a matching dynamic
import inside src/cli/runfile.ts: the gate failed and named the file,
then was reverted byte-for-byte (confirmed via `git diff --stat`) and
the gate went green again. Indirection through a variable
(import(someVar)) still evades this by design — that would need real
static analysis, not a source-text regex.
- Two comments overstated the gate's own guarantees: the remedy comment
implied verifying a newly-flagged version against the parser always
succeeds (D-018's own history says otherwise — 3.1.0's schema was
present under parser 3.4.0 and the parser still died on it), and the
"excluded from the export map" comment named only the 2.0.0 release
candidates when every 1.x schema is excluded too. Both reworded.
…egex The transport-isolation gate (only src/broker/ may import an MQTT/WS transport package — a hard project constraint) required exactly one space after `from` and gave no whitespace tolerance inside `require(…)`, making it the weaker of the two structural import-guard gates in this repo: test/upstream-drift.test.ts already tolerates `from\s+` and `require\(\s*…\s*\)`. Harmonized upward to the same tolerance; the set of matched packages is unchanged.
D-019's "discovered during implementation" note credited String()
normalization with catching the unquoted `asyncapi: 2.6` TypeError, but
left an unrecorded gap: a present-but-null asyncapi field still
collapsed to undefined and fell through to the same parser TypeError
until this fix wave closed it (see the preceding model fix). Recorded
the gap and its closure: any positively-present asyncapi value, null
included, is now read, because the parser's internal TypeError also
covers null, empty string, boolean, and object values, not just the
unquoted-float case.
Also added one comment, at the first of four `as Error` casts in
src/registry/index.test.ts, noting why the cast is needed:
Promise.catch unions its handler's return with the promise's own
resolve type, so the cast is compile-time only and does not touch the
runtime value. The other three occurrences follow the same pattern
without repeating the explanation.
REQUIREMENTS.md's R-039 wording ("drift-tested against @asyncapi/specs")
was checked against the hardened drift gate and left unchanged: it was
accurate before this wave and remains accurate now.
The comment cited src/engine/dispatch.ts as the live example of the idiom the new import() branch catches, but dispatch.ts:70 passes a variable, which the very next sentence says evades the guard. It is still the right evidence that dynamic import is idiomatic here, so the comment now says what it actually demonstrates, and names require.resolve alongside variable indirection as the forms that stay out of reach of a source-text regex.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Declares and tests the AsyncAPI version range offbook supports, and fixes three defects in the ingestion path that serves it. Design:
docs/superpowers/specs/2026-07-30-asyncapi-version-support-design.md. Plan:docs/superpowers/plans/2026-07-30-asyncapi-version-support.md. Decision: D-018. Requirements: R-037, R-038, R-039.Why
Research turned up three defects, none of which any test was catching.
{schemaFormat, schema}, andBaseModel.json()returns that wrapper verbatim, so the channel schema ended up with no validation keywords. Such a channel accepted everything: wrong types, unrelated objects, a bare string,null. No error, no diagnostic, green forever. This is precisely the false-negative class R-028 exists to prevent.{type: array, items: [...]}is valid in both majors, butajv.compile()threw because offbook stamped 2020-12 over a schema the parser emits as draft-07. Nothing caught the throw, so it surfaced as a raw Ajv error out ofbootProjectnaming no channel.mqttto an empty schema, so{qos: 9, retian: true}parsed with zero errors andqos: 9reached aChanneltyped0 | 1 | 2.What changed
Supported range: 2.0.0 through 2.6.0, 3.0.0, 3.1.0. 1.x is refused with an actionable message pointing at
asyncapi convert. The set is an explicit tested constant, notObject.keys(specs.schemas), because a schema being present does not mean the ruleset handles it. The parser floor moves to^3.6.0: 3.4.0 and 3.5.0 pass their own version gate on a 3.1.0 document and then die inside Spectral withError running Nimma.Payload validation moves to draft-07, the dialect both majors declare and the parser actually emits, stamped explicitly. This is the root-cause fix for defect 2 and for the
additionalItemssilent drop, and it needs no schema rewriting so R1 stays intact. Post-draft-07 keywords cannot be honored, so they are diagnosed rather than ignored. This reverses a canonical statement incontracts.md§5 and closes D-005's deferred 2020-12 obligation as superseded, with its tripwire retained and reworded.Also: an uncompilable schema now yields a violation per payload instead of a crash, and never validates green (discovery survives, false confidence does not). Operations declaring several messages validate as
anyOf, fixing v2message.oneOfunions. Bindingqos/retainare guarded and fall through the existing precedence chain on a bad value, with unknown keys checked against@asyncapi/specs' own binding schema. The spec version is recorded inspecs.lockand onSpecInfo.Findings worth reviewing
json-schema-faker 0.6.2 cannot draw a valid draft-07 tuple. It emits objects with numeric keys and fails the Ajv recheck 10/10 seeds, unchanged by
additionalItemsorminItems. Verified pre-existing, not caused by the dialect switch (identical draws under no stamp, 2020-12, and draft-07;prefixItemsdraws fine). Consequence: a tuple-shapedtoClientpayload has no usable L1 fake and falls to F5 drop-and-surface. The tuple regression case therefore lives as an inline test spec rather than a fixture, because a tuple fixture would fail the R-027 faker floor and flip D-008's measured verdict. Whether the L1 floor should gain a keyed-fallback re-draw is recorded in D-018 and left open.Two deviations from the design, both narrowing rather than expanding scope. New diagnostics reuse the existing closed
spec-loadkind with a taggeddetailprefix instead of adding six values to a union thatDiagnosticSummary.byKindmust zero-fill. Andoffbook doctordoes not reportspec-version, because its spec checks are network-free while the spec lives in a remote repo.One behaviour change on specs that already load: an operation with multiple messages previously validated against the first only, and now accepts any declared variant.
Verification
Full CI gate set green:
check-docs,lint,typecheck,demo-app:build, andbun test(423 pass, 0 fail) all exit 0.The R-028 validation gate is extended over both new fixtures with end-to-end rejection assertions through the composed stack. Those assertions were confirmed load-bearing by mutation: reverting the unwrap, blinding the qos binding read, and inverting the v2 direction map each turn the relevant gate test red. That last mutation exposed a wrong assertion in the plan (a
toClientpublish over/v1/publishis a mock emission and is still schema-validated, so the violation kind fired either way); it now asserts the responsedirectionfield, which the mutation confirms is load-bearing.Fixture additions are registered with the R-027 spike tripwire using measured values, never assumed ones.