DAN-855: the conformance suite's documented size was wrong in all three places it appeared - #6
Merged
Danny-Devs merged 3 commits intoAug 16, 2026
Conversation
…ee places it appeared `spec/vectors/` holds 167 vectors across 14 files. The repo said otherwise three different ways: README.md claimed 133, and docs/specification.md claimed 161 in two separate places. All three are now the real number. Two of those are cosmetic. The third is not. §13 "Conformance" criterion 5 is the NORMATIVE definition of what makes an implementation s402-conformant, so an implementer in Go, Python or Rust was told they conformed at 161 vectors — leaving six shipped vectors outside the definition of conformance. That clause now binds conformance to *every* shipped vector and carries the count as a version-anchored parenthetical rather than as the operative term. Also fixed: the README told readers to find the vectors by cloning the repo and running `ls test/conformance/vectors/`. That directory is generated by scripts/prepare-publish.sh at publish time and is git-ignored, so following the instructions in a fresh clone produced "No such file or directory". Clones are now pointed at `spec/vectors/`, and the relationship between the two paths is stated so the npm path stays right for package consumers. A count written into prose is a derived value maintained by hand — it rots silently on every vector added, and nothing fails when it does. Three independent wrong numbers is what that looks like once it has been running a while. So the fix is not only the numbers: test/spec-doc-counts.test.ts counts spec/vectors/ and compares it against every claim in both documents. The test is deliberately built to refuse a vacuous pass. If the wording changes so that no claim matches, it fails rather than silently verifying nothing — a check that cannot fail is not a check. It was watched to fail against four mutations before being kept: a wrong count in the spec, a wrong count in the README, a reworded claim, and a vector added without a doc update. It reads `spec/vectors/` and not `typescript/test/conformance/vectors/` on purpose. The latter is git-ignored build output; a check keyed there passes on any machine that has ever packed and is red on its first CI run. Also adds the README's missing "who this is for / who this is not for" and a falsification section naming four observable conditions that would retire the project — per the README convention's third question, which the doc set had nowhere to put. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
…was claiming a CI check that never existed `Status: Accepted` has only ever meant DECIDED. It has never said BUILT — so an ADR that shipped and an ADR that was ratified and quietly never built were byte-identical on the page. All twelve records now carry an `Implementation:` field, each determined against the code rather than against the prose. The ratio this makes countable: 7 of 12 shipped, 1 upheld, 2 in-progress, 2 not-started. ADR-006 was the only record already carrying an `Implementation:` field, and it was false. It stated in the present tense that scripts/compute-scheme-digests.mjs "runs in CI and writes spec/scheme-digests.json", failing any PR that changed a scheme doc without updating the digests. Neither file appears in any commit on any branch, and CI's five jobs contain no such gate. So the project's single answer to "was this built?" was a claim that read as verified precisely because it sat in the slot reserved for verified things. The paragraph is now marked as an unbuilt plan. Two findings fell out of doing this honestly: ADR-004's extensions framework ships as the `s402/extensions` subpath in package.json while its Status still reads Proposed — built without the decision ever being accepted. Status is left untouched: flipping a governance field is a decision, not a cleanup. ADR-010's S16 is half-built. Searching for the label found one narrative mention; searching for the MECHANISM found the protocol version genuinely bound into the signed envelope and required at envelope.ts:268. Its scheme-digest half is blocked on ADR-006, which is not-started. Grepping for a rule's name finds where someone wrote the name; grepping for its mechanism finds where it is enforced. Where the evidence is a search rather than a proof, the field says so — the negatives are recorded as "no witness found", not as "proven absent". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tranger meets it Sui has deprecated JSON-RPC on public fullnodes. Every core method returns -32601 on testnet AND mainnet, verified directly on both rather than taken from a report. mcp-demo/src/server.ts and src/agent.ts both construct a SuiClient over that transport, and docs/guide/tutorial.md's "Making It Real" snippet tells readers to do the same. This is a hole in the done-bar item I had just called satisfied. Item 1 is "a stranger can run the one demo in 10 minutes", and I had verified the README's CLAIMS — paths, counts — without ever running the demo. The grant-ready standard I authored says to enumerate every shipped surface; I applied that to test suites and missed the runnable artifact, which is the surface the stranger test is actually about. The fix here is honesty, not repair: both surfaces now state the blocker above the instructions, so nobody follows a dead path. The migration to gRPC/GraphQL is tracked separately and is not attempted inside a documentation pass. Both notices are to be removed by that work — a stale blocker warning is its own defect, so the removal is written into that ticket's done-bar. Worth recording what did NOT break: typescript/package.json has no Sui dependency and never had one (ADR-002, invariant S7, enforced by the S7 Chain-Agnostic Boundary CI job). The entire protocol layer — encoding, validation, conformance vectors, all 1108 tests — runs offline and is untouched. The blast radius is exactly one demo and one snippet. That containment is the pure-protocol decision paying out, and it is the strongest evidence for S7 the repo has produced so far. Raised cross-lane by agent-b g11 from the SweeFi pass; verified independently before acting. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This was referenced Aug 16, 2026
Danny-Devs
deleted the
agent-a/DAN-855-spec-vector-count-and-stranger-test
branch
August 16, 2026 23:19
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this fixes
spec/vectors/holds 167 vectors across 14 files. The repo stated that three different ways, and all three were wrong:README.mddocs/specification.md:11docs/specification.md§13 criterion 5The third one is the reason this is a bug rather than a typo. §13 Conformance defines what makes an implementation s402-conformant, so an implementer in Go, Python or Rust reading the spec was told they conformed at 161 vectors — leaving six shipped vectors outside the definition of conformance. That clause now binds conformance to every shipped vector, and carries the count as a version-anchored parenthetical rather than as the operative term.
Second fix: the README's clone instructions did not work
Under Conformance Testing, the README told readers to clone the repo and run
ls test/conformance/vectors/. That directory is generated byscripts/prepare-publish.shat publish time and is git-ignored (typescript/.gitignore:7), so following the instructions in a fresh clone producedNo such file or directory.Clones are now pointed at
spec/vectors/, and the relationship between the two paths is stated explicitly so the npm path stays correct for package consumers.test/conformance/vectors/is untouched — it is publish output that ships inpackage.json'sfiles, and removing it would strip the contract from the npm package.Why a test and not just three edited numbers
A count written into prose is a derived value maintained by hand. It rots silently on every vector added, and nothing fails when it does — three independent wrong numbers is what that looks like after a while. Editing them only resets the clock.
typescript/test/spec-doc-counts.test.tscountsspec/vectors/and compares it against every"<N> vectors across <M> files"claim in both documents.It is built to refuse a vacuous pass. If the wording changes so no claim matches, it fails rather than silently verifying nothing — a check that cannot fail is not a check. It was watched to fail against four mutations before being kept:
It reads
spec/vectors/and nottypescript/test/conformance/vectors/deliberately: the latter is git-ignored build output, so a check keyed there passes on any machine that has ever packed and is red on its first CI run.conformance.test.ts:44resolves the same tracked path for the same reason.Third: the README's missing third question
Per the README convention, a README answers what this is, who it is for, and what would prove it should stop. The last had nowhere to live. Added:
exactshould use x402.Verification
Run on this branch, rebased onto
main@1693d1d(post-DAN-860), clean tree. Both suites, because the TypeScript-only command reports green on a repo whose Python suite is red:pnpm typecheckpnpm vitest runpnpm builduv run --extra dev pytestNo conformance vector was edited, and the merge that landed DAN-860 did not change
spec/vectors/— the count is 167/14 before and after.🤖 Generated with Claude Code