feat(postman): regenerate the collection from the live spec, and check it - #31
Merged
Merged
Conversation
…k it The collection shipped with the v0.1.0 SDKs and was never touched again: 6 requests against an API that now has 26 operations, and — as far as anyone can tell — never executed. Preparing it for the Postman Public API Network meant running it for the first time, and the browse request would not even leave curl, because the spec example it was built from carries an unencoded space. Now generated from https://api.greencalculus.com/openapi.json: 28 requests in 5 folders, every operation the API has, request bodies taken from the spec's own worked examples. TEN OF THEM NEED NO KEY, AND THEY SORT FIRST IN EVERY FOLDER. That is the reason to publish a collection at all — import it and the corpus browse, the change feed, coverage, the absence register and a publisher's whole source feed answer before you have signed up for anything. Auth per request is set from MEASURED behaviour, not from what the spec declares: the two disagreed on six endpoints until gc-api-gateway#118, and a request marked "needs a key" that actually runs keyless is a request nobody pastes. check-collection.mjs sends every request and fails on any 4xx. Keyless ones go with no Authorization header at all, because that is the claim being tested; a 403 is tolerated on a keyed request (an entitlement, not a broken request) and never on a keyless one. The three Account writes are skipped unless GC_CHECK_MUTATIONS=1 — a check should not edit the account it is checking. Wired into CI keyless-only, so it needs no secret: 28 requests, 0 failing. Publishing to the Public API Network still needs a human — it requires a Postman account for the workspace. The five steps are in postman/README.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NRueWxopDXHoWY2dPvsmLG
The collection was generated from the live OpenAPI document once, by hand, in the commit before this one. Generation is an event, not a property the file keeps: from the moment it was committed, the collection and the spec became two independently editable copies of the same 28 worked examples, each with its own gate proving it answers 200, and nothing proving they still agree. That is gc-api-gateway#118's failure one level up. #118 found cursor=2026.182 in the spec — older than the change feed's own record_begins, so the documented example was one the API is built to reject. It was fixed in the spec; the collection's copy of that value was fixed separately. Next time a pin ages out, whichever artefact is edited first is right and the other ships the broken default — and the collection is the one strangers press Send on before they read anything. check-collection.mjs cannot see this. It proves every request ANSWERS; it has no opinion on whether the spec asks for that request. The two questions are different and a stale pin separates them: it answers 200 and is still wrong. check-collection-parity.mjs re-reads the live spec and fails on coverage drift in either direction, on a collection value that contradicts a declared spec example (path, query or request body), and on a query parameter the spec does not document at all. Verified by mutation, not by passing: a stale path pin, an undocumented operation, an undocumented query parameter, a drifted request body and a dropped operation each exit 1; the real collection exits 0. Two things it deliberately does NOT fail on. Postman-disabled parameters, which are never sent — they sit in the request to show a reader the option exists. And values the spec declares no example for: it names the parameter but pins nothing, so the collection had to invent one. Those are printed loudly instead, because failing on them would push authors to delete the example rather than pin it. There are eight today, and as_of=2026.175 is seven of them — an archive pin sixteen versions behind current, governed by nothing. It still resolves (matched: true, reproducible: true). Pin it in the spec and this gate starts enforcing it. Auth is not checked here on purpose. The collection marks a request keyless from MEASURED behaviour, not the spec's security block; the two disagreed on six endpoints until #118, and measurement was right. That claim is tested where it belongs, by check-collection.mjs sending the keyless ones with no header at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeaxFttn3GPPVq1yj3Jjuk
Collaborator
Author
|
Added Why, given the collection is generated: generation is an event, not a property the file keeps. From the moment it was committed, the collection and the spec became two independently editable copies of the same 28 examples — each with a gate proving it answers 200, and nothing proving they still agree. There is no generator in the repo to re-run, so nothing stops the next edit to one from leaving the other behind. That is #118's failure one level up: a stale version pin answers 200 and is still wrong, so Verified by mutation, not by passing. Five divergence classes, each exits 1; the real collection exits 0:
Two things it deliberately does not fail on. Postman-disabled parameters ( There are eight today, and Auth marking is not checked here on purpose: the collection sets it from measured behaviour, not the spec's |
…side it
Two gaps the parity gate's own docblock names but does not close.
THE GENERATOR ONLY EXISTED IN A SCRATCH DIRECTORY. "Generated from the live
spec" was true of an afternoon, not of the file — the next person to regenerate
would have written a different script and got a different collection. generate.py
is that script, in the repo, reproducible: run against the live spec today it
reproduces the committed file byte for byte (--dry-run says "no change").
It keeps the property that matters and a stock converter loses: AUTH IS MEASURED,
NOT READ. Each request is sent once with no Authorization header; 401 means keyed,
anything else means open. The live spec still declares only 4 open endpoints while
10 answer keyless — that is fixed and awaiting deploy in gc-api-gateway#118 — so a
generator that trusted `security` today would have quietly demoted six requests
and taken the reason to publish a collection with them. POSTs are probed with {}
rather than the real body: we are testing the door, not running somebody's
calculation.
THE LISTING COPY WAS A STRING LITERAL IN THE GENERATOR. It is now description.md.
The old text was a readme — it opened on a category line any competitor could
write, asserted provenance without showing any, and buried "press Send right now,
no signup" in the third paragraph. The new one opens on the claim, shows a real
response (0.13096 kg CO2e per kWh, cell 'UK electricity'!E25, its OGL v3.0
licence and its versioned proof link, all pulled live), and puts the keyless ten
in a heading. Only the description changed in the collection — the 28 requests
are byte-identical.
All three gates pass: 28 requests answering, parity with the spec, generator
reproducible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NRueWxopDXHoWY2dPvsmLG
The parity gate's README says the collection "is generated" and lists the gate that keeps it true; it did not say how to regenerate it, because until the previous commit there was no committed way to. Adds the two commands, and the rule that matters for anyone rewriting the listing copy: description.md is the source, and editing the description in Postman instead survives exactly until the next regeneration. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NRueWxopDXHoWY2dPvsmLG
… than duplicates Postman matches an imported collection against info._postman_id. The generated file carried none, so every import minted a fresh one and dropped a second copy of "GreenCalculus API" into the workspace beside the first — in a workspace being prepared for publication, that is the kind of mess that ships. The id is now derived from the collection's own URL namespace, so it is identical on every machine and every regeneration, and a re-import offers Replace. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NRueWxopDXHoWY2dPvsmLG
Its header was written when the collection had been generated once, by hand, and said so. cbef85f put generate.py in the repo, which makes "once, by hand" wrong and reads as if the gate were arguing against a state of affairs that no longer holds. The reason survives the generator intact, and is worth stating precisely because a generator invites the opposite assumption: generate.py rebuilds the collection, but nothing makes anyone run it. The spec moves on its own deploy cycle, and a request can be edited in Postman and exported back over the top. Between one run and the next the two documents drift freely, and a stale pin answers 200 the whole time. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LeaxFttn3GPPVq1yj3Jjuk
The README described publishing as a thing still to do. It is done: the workspace is public, the domain is DNS-verified at the apex, and Guided Auth is configured and verified against api.greencalculus.com. Two findings recorded beside the URLs, because both cost time today and neither is written down anywhere else. The published documentation serves noindex,nofollow with no SEO toggle in the publish flow, while other public documenter pages carry no robots meta at all — so a Postman listing is a directory entry, not an indexable page, until support explains that. And the workspace summary and team tagline both truncate silently at 140 characters, which published "…and data versio" on the first attempt. Also the re-import rule the stable _postman_id buys: regenerate, import, choose Replace. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NRueWxopDXHoWY2dPvsmLG
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The Postman collection shipped with the v0.1.0 SDKs and was never touched again: 6 requests against an API that now has 26 operations, and never executed. Preparing it for the Postman Public API Network meant running it for the first time — the browse request would not even leave curl, because the spec example it was built from carries an unencoded space.
What changed
https://api.greencalculus.com/openapi.json— 28 requests in 5 folders, every operation, bodies from the spec's own worked examples. It cannot fall behind the API the way a hand-maintained collection does.check-collection.mjssends every request and fails on any 4xx. Keyless ones go with noAuthorizationheader at all, because that is the claim being tested. A 403 is tolerated on a keyed request (an entitlement) and never on a keyless one. The threeAccountwrites are skipped unlessGC_CHECK_MUTATIONS=1— a check should not edit the account it is checking.Still needs a human
Publishing to the Public API Network requires a Postman account to own the workspace. The five steps are in
postman/README.md. Worth doing in the same sitting:public-apishas a "Call this API" column that wants the Run in Postman button, and our entry there (public-apis#6984) has been open since 21 August.🤖 Generated with Claude Code
https://claude.ai/code/session_01NRueWxopDXHoWY2dPvsmLG