Repository navigation
chore+docs: pin 0.23.0, regenerate the contract reference, and name the release the pages were waiting on - #168
Merged
Merged
Conversation
`supportedCliVersion` and `supportedCliInstallSpec` move to 0.23.0. Nothing else in cli-version.json moves, and that was checked rather than assumed: 0.23.0 bundles 0.7.1 through 0.7.6 with 0.7.6 still in PREVIEW_VERSIONS, so the stable schema is still 0.7.5; `fluid init --quickstart` still emits 0.7.5; and the SDK and custom-scaffold pins are still the latest on PyPI (0.10.0 and 0.4.1), since this release was `data-product-forge` alone. The version sweep then named 21 lines. They split two ways. Fourteen are claims about what a reader should run, or sample output, and move to 0.23.0. Each is substantiated by a check in this same run: `fluid --version` on 0.23.0 prints `FLUID Forge CLI v0.23.0`, which is what `cli/version.md` now shows; the command-coverage check passes 69 subcommands; and the fence-flag oracle passes 1335 invocations. Seven record which release a page's own content was verified against — the `Docs Baseline` banners on the four provider pages and two walkthroughs, and `cli/README.md`'s note that its examples use a particular release. None of those pages was re-verified or re-recorded here, so moving the number would claim work nobody did. They keep the release that produced them and gain `<!-- cli-version: historical -->`, which is what that marker is for: a line whose version is deliberately not the pin. That leaves roughly 424 other `0.18.1` mentions across the tree that the sweep does not match — "as of 0.18.1 ...", recorded transcripts, sample `fluid_version` fields. Re-baselining those is the "track 0.23.0" pass this repo has done before, once per release, and it is a PR of its own rather than a rider on the pin bump. `scripts/forge-cli-releases.json` is refreshed from the forge-cli checkout, so `reviewedThroughRelease` is 0.23.0 and the release-marker sweep's freshness half stays armed for the next release. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…lker minProperties
`reference/contract-0.7.6-preview.md` has been rendered from the 0.7.6
schema bundled in 0.18.1 since the pin stopped moving, so it listed
neither `engine: flink` nor `trigger.type: continuous` nor
`sink.catalogAuth`, four releases late. With the pin on 0.23.0 the
generator runs again: the delta goes from 48 added rows to 93 and from
12 changed to 17, and now carries all of the above plus the seven fields
0.23.0 added.
The generator refused the first run:
ERROR: fluid-schema-0.7.6.json $.$defs.column.properties.quality.items:
keyword 'minProperties' is not one the reference generator renders.
That is the walker working as designed — it hard-fails on an unknown
keyword rather than dropping the constraint silently. `minProperties: 1`
is new in this release (absent from 0.22.0's bundled 0.7.6 and from
`main` before forge-cli #753), and it sits on a column's quality rule,
where every key is optional and an empty `{}` would otherwise validate
as a rule that says nothing. The walker now renders it as the object
counterpart of `minItems`, so the row reads `each item: min properties 1`
in the same idiom as the rest of the Allowed values column.
`contract-0.7.5.md` moves too, by four lines: its provenance line now
names 0.23.0 with the new schema sha, and two Lake Formation
descriptions gained text in this release.
Confirmed in the regenerated output rather than assumed: the
column-`quality` description renders `fluid odcs validate`, not the
`fluid generate odcs --validate` that does not exist, so forge-cli's fix
to that schema text carried through verbatim as expected.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four places told a reader the work was coming in "the next release after 0.22.0" because the number was not settled when they were written. It is 0.23.0, and they say so: the exported-artifact warning and the DuckDB drift-defect tip in `upgrading.md`, the two extra protected variables in `project-dotenv-trust.md`, and the seven preview fields in `preview-fields.md`. Each also loses the sentence explaining that the number was unsettled, which has nothing left to explain. `preview-fields.md` needed more than a number. Its shape was built around the stale pin, and moving the pin is what lets it be an index again: - The seven 0.23.0 fields join the page's own "What the preview adds" table, with `First in CLI` 0.23.0 and the path spelled the way the generated delta spells it, so the table is a usable index into that page. Their detail section loses its duplicate table and keeps the four subsections that hold what a row cannot — the `fluidEmail` caveat, pinning a contract id, declaring the wire format, and what setting a column's `quality` replaces. - `sink.catalogAuth`, `builds[].engine: flink` and `trigger.type: continuous` were prose paragraphs *because* the delta could not list them. It can now, so they are rows too. The paragraph about `sink.catalogAuth` also claimed it had no row in the delta, which stopped being true the moment the pin moved; that claim is gone with the paragraph. - The pointer to the delta named only its `consumes` and `exposes` sections. The seven fields land under `metadata`, `description` and `builds` as well, so it now names the grouping rather than two of its five groups. The overlay-by-identity paragraph stays prose: it is a behaviour, not a field, and a field table is the wrong place for it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The post-release pass for forge-cli
0.23.0: the pin, the regeneration it unblocks, and the four pages that said "the next release after0.22.0". Three commits, one per concern.1. The pin
supportedCliVersionandsupportedCliInstallSpecmove to0.23.0. Nothing else incli-version.jsonmoves, and that was checked rather than assumed, since you asked:supportedFluidContractVersion0.7.5FluidSchemaManager.PREVIEW_VERSIONSon 0.23.0 is['0.7.6']quickstartScaffoldVersion0.7.5fluid init --quickstarton 0.23.0 emits 0.7.5 — unchangedsupportedSdkVersion0.10.0data-product-forge-sdksupportedCustomScaffoldVersion0.4.1data-product-forge-custom-scaffoldBUNDLED_VERSIONSon 0.23.0 is0.7.1 … 0.7.6, same set as 0.18.1, so the contract-pins check passes untouched.The version sweep named 21 lines, and they are not all the same thing
Fourteen are claims about what a reader should run today, or sample output. They move to
0.23.0, and each is substantiated by a check in the same run:fluid --versionon 0.23.0 printsFLUID Forge CLI v0.23.0(which is whatcli/version.mdnow shows), the coverage check passes 69 subcommands, and the flag oracle passes 1335 fenced invocations.Seven are a different claim: they record which release a page's own content was verified or recorded against — the
Docs Baselinebanners onproviders/{aws,gcp,local,snowflake}.mdandwalkthrough/{export-orchestration,universal-pipeline}.md, pluscli/README.md's note that its examples use a particular release. I did not re-verify or re-record any of those pages. Moving the number would have claimed work nobody did, so they keep the release that produced them and gain<!-- cli-version: historical -->— a line whose version is deliberately not the pin, which is exactly what that marker is documented for.Say if you would rather they moved to
0.23.0anyway; it is a one-line change per page, but it is a claim I cannot stand behind from here.The 424 the sweep does not see
There are about 445
0.18.1mentions across roughly 145 files outsideRELEASE_NOTES_*. The sweep matches 21 of them, because it only matches baseline phrasings on purpose. The rest are "as of0.18.1…", recorded transcripts, samplefluid_versionfields and the like — many historically true, some genuinely stale now that 0.19 through 0.23 have shipped.Re-baselining those is the "track 0.23.0" pass this repo has done once per release before (
docs: track 0.18.1 — every page says what 0.18.1 does, #141). It is a PR of its own, not a rider on the pin bump. Flagged as a follow-up with the count so it is a decision rather than an oversight.scripts/forge-cli-releases.jsonis refreshed from the forge-cli checkout, soreviewedThroughReleaseis0.23.0and the release-marker sweep's freshness half stays armed for the next release.2. The regeneration — and one thing it refused to do first
reference/contract-0.7.6-preview.mdhas been rendered from the 0.7.6 schema bundled in 0.18.1 ever since the pin stopped moving, which is why it listed neitherengine: flinknortrigger.type: continuousnorsink.catalogAuth. The delta now goes from 48 added rows to 93, and from 12 changed to 17.The first run failed, and it was right to:
That is the walker's fail-closed design doing its job — it refuses an unknown keyword rather than dropping the constraint silently.
minProperties: 1is new in this release: absent from 0.22.0's bundled 0.7.6 and frommainbefore #753, present on the #753 branch. It sits on a column's quality rule, where every key is optional and an empty{}would otherwise validate as a rule that says nothing. The walker now renders it as the object counterpart ofminItems, so the row readseach item: min properties 1in the same idiom as the rest of that column. That is the only change to the generator.The hard-fail is the right behaviour, and it has a standing cost. Rendering a constraint the
walker does not understand, or quietly dropping it, would publish a field reference that
understates what
fluid validateenforces — which is worse than a failed job. The cost is thatthe generator lives in this repo while the schema lives in forge-cli, so any forge-cli release
that introduces a schema keyword blocks its own docs regeneration until someone adds support
here. That is a follow-up below, not a thing to fix in passing.
contract-0.7.5.mdmoved by four lines too: its provenance line now names 0.23.0 with the new schema sha, and two Lake Formation descriptions gained text in this release.Confirmed in the output, not assumed
fluid odcs validatecarried through. The regenerated page contains it once and containsfluid generate odcszero times, so the schema-text fix you made ine1b62a53renders verbatim as predicted. I also diffed the #753 branch schema against the released 0.23.0 one: that one description is the only difference between them, which is the tidiest possible confirmation.min properties 1renders onexposes[].contract.schema[].quality.metadata.team.members[].emailrenders withformat emailand an empty description, because the schema gives it none — which is precisely why thefluidEmailcaveat earns its own subsection rather than a table cell.3. The four slots, and what else the pin unlocked
upgrading.md× 2 (the exported-artifact warning, the DuckDB drift-defect tip),project-dotenv-trust.md× 1,preview-fields.md× 1 — all now naming0.23.0, each also losing the sentence that explained the number was unsettled.preview-fields.mdneeded more than a number swap, because its shape was built around the stale pin:First in CLI0.23.0, each path spelled the way the generated delta spells it (soexposes[].contract.schema[].quality, not "qualityon a column"). Their detail section drops its now-duplicate table and keeps the four subsections that hold what a row cannot.sink.catalogAuth,builds[].engine: flinkandtrigger.type: continuouswere prose paragraphs because the delta could not list them. It can now, so they are rows. Thesink.catalogAuthparagraph also asserted it had no row in the delta, which stopped being true the moment the pin moved — gone with the paragraph.consumesandexposessections; the seven land undermetadata,descriptionandbuildsas well, so it names the grouping instead of two of five groups.The overlay-by-identity paragraph stays prose: it is a behaviour, not a field.
Premise notes
The root
descriptiontext did not change, and did not need to. You said it changed and that "the old sentence about an ODCS export publishingpurposealone is gone". Diffed byte for byte: that description is identical between the #753 branch and released 0.23.0. It is also already correct, because the sentence is written as history — "While this was a string, an export of a valid contract could only ever carrydescription.purpose" — and is followed by the sentence saying the object now survives a FLUID → ODPS → FLUID cycle and reaches both exports. Nothing to fix; the only schema-text change between the branch and the release is thefluid odcs validateone.0.23.0 is on PyPI, and my first install said it was not.
pip install data-product-forge==0.23.0failed with a version list stopping at 0.22.0. That was a stale cached index page, not reality: pypi.org's JSON API reportsinfo.version0.23.0, the/0.23.0/jsonendpoint is 200, and the simple index lists both the wheel and the sdist.--no-cache-dirinstalled it. Noting it because your "verified against the pypi.org index rather than local pip" is exactly the distinction that mattered, and because a CI runner with a warm index could hit the same thing — the workflow'spip installhas no--no-cache-dir, though a fresh runner has no cache to be stale.Verified locally
Reproduced from the workflow files, on python3.12 with
data-product-forge[local]==0.23.0:check_cli_docs.py --version-onlycheck_cli_docs.py(all 7 checks)check_providers.pygen_contract_reference.py --checknpm run docs:buildnode scripts/check-dist-links.mjsThe release-marker sweep now reports
forge-cli newest release 0.23.0, reviewed through 0.23.0.Heading anchors are still the one link class no gate here covers, so I read them out of the built HTML again. The retitled section's anchor is
four-of-the-0-23-0-fields-need-more-than-a-table-row; nothing linked to the old one. The four subsection anchors are unchanged, and the four cross-page anchors the new rows use all resolve:authenticate-a-sink-to-a-rest-catalog,flink-jobs-engine-flink,continuous-builds-trigger-type-continuous,read-this-first-if-you-consume-the-exported-artifacts.Follow-ups
Four, none of them acted on here.
Re-verify the seven
Docs Baselinepages against 0.23.0, then move the banner. This isthe other half of the decision above. A banner frozen at
0.18.1behind a historical markeris honest today and misinforming eventually, and the only thing that resolves it is actually
re-running those four provider pages and two walkthroughs — plus
cli/README.md's examples —against 0.23.0 and re-recording what changed. Real work, deliberately not smuggled in here.
Add schema-keyword support to
gen_contract_reference.pyas part of each release. Asabove: the walker refuses an unknown keyword by design, the generator is in this repo, and the
schema is in forge-cli — so a release that tightens the schema stops its own docs. 0.23.0's
minPropertiesis the first instance and took one line plus a comment, but the pattern willrepeat, and the failure lands on whoever next bumps the pin rather than on the release. Worth
either a line in forge-cli's release checklist pointing at this file, or a periodic pass over
the keywords the bundled schemas use versus
KNOWN_KEYWORDS.The "track 0.23.0" pass over the roughly 424 remaining
0.18.1mentions the sweep doesnot match — "as of
0.18.1…", recorded transcripts, samplefluid_versionfields. Theprecedent is docs: track 0.18.1 — every page says what 0.18.1 does, with release notes and an upgrade guide for 0.15.1–0.18.1 #141,
docs: track 0.18.1 — every page says what 0.18.1 does, with release notes and an upgrade guide, which is the same pass done once before, as its own PR. Overlapsfollow-up 1 and could absorb it.
docs/cli/reference edits for--envonfluid odcs exportandfluid odps-bitol export, which 0.23.0 added. These are safe in fenced blocks now that thepin is 0.23.0; the flag oracle would have rejected them against 0.18.1, which is why docs: the seven new 0.7.6 preview fields, and the exported ODCS/ODPS artifact changes (forge-cli #753) #167 left
them in prose.