feat(artifact)!: load versioned manifests as a set

[lokewate]

Summary

An artifact source publishes manifest_v1.json, manifest_v2.json, … and artifact.Set keeps one Registry per version. A published manifest and the files it lists never change: a change is a new file plus a new manifest that points the same ID at it. A flow records Set.Current() when it starts and resolves every artifact through Set.Registry(ctx, version), so a deploy never changes the templates of a running flow.

This is the first of two PRs. The second threads the version through the workflow engine and taskflow, and is stacked on this branch.

Type of Change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• Documentation update
• Refactoring (no functional changes)
• Performance improvement
• Other (please describe):

Changes Made

• LoadSet probes manifest_v1.json upward until the loader reports one missing. Any other load error fails the call, so a transient fault cannot hide newer manifests. The highest version is current unless WithPinnedVersion names an older one. With a pin, only manifests up to it are read, so a broken newer manifest cannot block a rollback.
• Set.Registry loads a version above the highest it has on demand. Replicas load their sets at different times, and a flow pinned by a newer replica must still run on an older one. A probe fetches only up to the version asked for and holds no lock that lookups wait on. After a probe that fails or finds nothing, it waits 10s before probing again.
• Manifest rules. RegisterFromConfig rejects two rows with the same id, kind and version. LoadSet rejects an explicitly versioned row whose path differs between manifests. GetExplicit serves such a row outside any flow, which static data needs. On a miss it probes for newer manifests with the same back-off, because a form in a flow a newer replica started can ask for a version only a newer manifest lists. Latest returns a manifest's unversioned row even when the manifest also lists explicit versions.
• Set.Validate fetches every artifact a manifest lists and runs the Check for its kind. It reports every missing file, malformed artifact and dangling reference at once. Each core adapter gains a Check, collected in corechecks.Checks():
  • a task template must name a listed workflow and render config;
  • a workflow's TASK nodes must name a listed task or step template;
  • a split that runs one template for every branch must name a listed workflow;
  • a render config's sections must name listed templates.
• adapter/kinds holds the core kind constants, so adapters that point at each other share them.
• WithManifestVersion / ManifestVersionFrom carry a version across a boundary that knows nothing of artifacts, such as uiprojector's template provider.
• cmd/manifestcheck checks bundles on disk for an artifacts repo's CI. It checks contiguous versions, LoadSet rules and Validate for every manifest, reading and checking each file once. With -base, it also checks that no published manifest or artifact was changed or deleted.
• Breaking: ManifestFilename gives way to ManifestPath(v), LoadManifest takes a version, and workflowdef.LoadVersion is removed.

Testing

• I have tested this change locally
• I have added tests that prove my fix is effective or that my feature works
• I have tested edge cases
• All existing tests pass

Run with go test -race ./artifact/.... The tests cover:

• pin rollback past a broken newer manifest, and pins above the highest or below zero;
• on-demand loading, a probe that stops at the requested version, and back-off after both empty and failed probes;
• a probe not blocking lookups of loaded versions, and concurrent Registry calls;
• duplicate rows, explicit-version path changes, GetExplicit, and Latest preferring the unversioned row;
• Validate reporting several problems together, and each adapter Check;
• manifestcheck for gaps, dangling references, and immutability edits and deletions.

manifestcheck passes on every bundle of one-trade-artifacts (cda, cda-v2, customs, customs-v2, fcau, npqs, sltb, tnsw) with each manifest.json renamed to manifest_v1.json. That run includes the duplicate-row fix in OpenNSW/one-trade-artifacts (linked below).

Checklist

• My code follows the project's style guidelines
• I have performed a self-review of my code
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• My changes generate no new warnings
• I have checked that there are no merge conflicts

Related Issues

N/A

Screenshots/Demo

N/A

Additional Notes

• Reference checks cover what core can see. Plugin and extension properties can also name artifacts. Only the host knows those shapes, so the host adds checks: feat(artifacts)!: run each flow on the manifest version it started on nsw-srilanka#660 checks notification template_ids.
• A TASK node may name either a task template or a step template. A single workflow file does not say whether it is a case's top-level workflow or a task's own workflow.
• S3 needs s3:ListBucket. Without it, S3 reports a missing key as access denied, and LoadSet fails instead of ending the probe.

🤖 Generated with Claude Code

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

---

_{Comment @coderabbitai help to get the list of available commands.}