Skip to content

✨ Workflow models: ingest Tekton from a cluster and written specs (LLM-assisted, inferred) - #17713

Merged
clubanderson merged 6 commits into
v5from
workflowmodel/v5-p6-ingest-cluster-spec
Nov 11, 2026
Merged

clubanderson merged 6 commits into
v5from
workflowmodel/v5-p6-ingest-cluster-spec

Conversation

@clubanderson

Copy link
Copy Markdown
Member

RFC phase 6 of the workflow-models RFC (#17620), on v5: two more ways into a draft lifecycle model — Tekton from a live cluster context and a written process spec, the latter optionally LLM-assisted with every output element confidence: inferred.

What

A. Tekton from a cluster — pkg/workflowmodel/ingest/cluster

  • The repo has no client-go (everything shells out to kubectl), so the package uses a Runner seam (Kubectl{Kubeconfig} in production, a table-driven fake in tests) instead of a fake dynamic client.
  • Fetch lists tekton.dev Pipelines/Tasks/PipelineRuns (newest N per pipeline, default 3, max 20) and triggers.tekton.dev EventListeners/Triggers/TriggerTemplates/TriggerBindings, serialises to cluster/<ns>/<kind>/<name>.yaml and feeds the existing file-based Tekton ingester — one parser, citations at those virtual paths.
  • Last run per pipeline → runs[] in the response + workflow.hivecommons.dev/observed-run/<pipeline> annotation + CodeObservedRun coverage finding. Observed, never an inferred lifecycle element.
  • Errors: ErrNoCRD 422, ErrForbidden 403, ErrNoContext 404, ErrNamespaceRequired 400, ErrNoRunner 422, ErrTooLarge 413. Caps: 25 namespaces, 20 runs/pipeline, 500 objects, 8 MiB, 60 s/kubectl call.

B. Written spec — pkg/workflowmodel/ingest/spec

  • Split → heading sections with line ranges; refs spec:<heading>#L<a>-L<b>.
  • Stage 1 (deterministic, always runs, always compiles): step lists → states in order; hint table (triage/intake/request/review/approve/merge/publish/deploy…) → columns; if/when/after/unless → guards; role nouns → roles; a step naming a role → a gate it releases.
  • Stage 2 (optional LLM): prompt = spec + section refs + skeleton + reflection-derived schema + fact vocabulary + compiler refusal rules (prompt.go, schema.go). Accepted only after json.Unmarshal with DisallowUnknownFields → Mark → Compile. One retry with diagnostics appended (MaxAttempts = 2), then fallback to the skeleton (spec-llm-fell-back).
  • Mark forces: apiVersion/kind/name, meta.version "1", meta.source spec:<origin>, meta.provenance ingested-unreviewed, every state/event/gate origin: ingested + confidence: inferred, citations validated against section refs (unknown → whole-doc ref + spec-llm-citation-replaced), pipelines/sources/agents stripped. Annotation workflow.hivecommons.dev/llm-assisted: "LLM-assisted (model <id>)" when the gateway's answer was used.

C. Dashboard

  • New pkg/dashboard/api_workflows_ingest_sources.go, registered by one line in registerWorkflowRoutes:
    • GET /api/workflows/ingest/capabilities → {cluster, clusterReason, contexts[], spec: {llm, model}}
    • POST /api/workflows/ingest/cluster {context?, namespace | allNamespaces, selector?, runsPerPipeline?, repository?, save?, name?} → {model, compiled, diagnostics, profile, files, pipelines, source: "cluster:<ctx>/<ns>", runs[], namespaces[], entry?}
    • POST /api/workflows/ingest/spec {text | repo+path | url, useLLM?, repository?, save?, name?} → {model, compiled, diagnostics, profile, source: "spec:<origin>", llm: {used, model, attempts, accepted, error?}, sections[], entry?}
    • Owner role, audit workflow_ingest, 201 + entry on save. useLLM defaults to "gateway configured"; requesting it without one → 422. No kube access → 422. URL fetch goes through the existing SSRF guard (https, public host).
  • LLM access reuses the reviewer-gateway TurnRunner (newWorkflowSpecRunner, 8 000-token budget) — no provider is called directly. New deps WorkflowRepoFile, WorkflowClusterRunner, WorkflowSpecLLM wired from cmd/hive.
  • Additive openapi blocks for the three routes + WorkflowIngestCapabilities, WorkflowClusterIngestResult, WorkflowSpecIngestResult.
  • UI: new static-src/js/config-workflows-ingest-sources.js + css/workflows-ingest.css; one guarded hook at the end of renderWorkflowIngest. Two <details> panels: From a cluster (context select from capabilities or text input, namespace, selector, all-namespaces, observed runs) and From a written spec (paste / URL / repo file; Use AI assistance only when a gateway is reported; LLM note + section list). Reuses the repo-ingest result block, diagnostics and diagram. Tokens only, no toLocaleString, no forced downloads.

D. Docs

  • docs/design/workflow-models.md — "Ingest from cluster and from written specs (RFC phase 6)".
  • changelog.d/added-workflowmodel-ingest-cluster-spec.md.

The LLM boundary

The gateway is reached only through spec.Completer, only from spec.Ingest, only to draft. Nothing in derivation, compile, the engine, or the other ingesters imports a gateway. Its output cannot be confirmed, cannot cite anything but the spec, cannot add pipelines, cannot bypass Compile; the worst it can do is be ignored. Tests cover accepted / retry-then-accepted / bad JSON / non-compiling / citation replacement / fallback.

Tests & coverage

  • ingest/cluster 95.9%, ingest/spec 98.6% (fake Runner / mock Completer; no network).
  • Dashboard handler tests (capabilities, cluster preview+save, cluster error mapping, spec deterministic, spec with LLM, spec from repo/URL), node UI test, TestWorkflowSpecRunnerUsesLargerBudget.
  • go vet, go test ./pkg/workflowmodel/... ./pkg/dashboard/ ./pkg/dashboard/webstatic/ ./cmd/hive/ green apart from the known macOS-only TestConfigExportReportsPersistedPath / TestKnowledgeDrainLegacyMemory; ratchets unchanged; index.html -check clean; changelog guard OK.

Gaps

  • kubectl Runner seam rather than client-go's fake dynamic client (no client-go in go.mod; adding it was out of scope).
  • No live-cluster or live-gateway integration test; both are mocked.
  • Deterministic extraction is heuristic by design — the LLM stage and the human review are what make it a model.

Refs #17620

@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 11, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign nicholasjackson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Nov 11, 2026
@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#17716 ✨ benchmarks: per-PR evidence export + opt-in publish to hivecommons/benchmarks same files dashboard/openapi.json
src/pkg/dashboard/static/index.html
#17706 ✨ Workflow models: visual editor with live compile, YAML, diff, simulate and export CONFLICT dashboard/openapi.json
src/docs/design/workflow-models.md
src/pkg/dashboard/api_workflows.go (conflict)
src/pkg/dashboard/static-src/js/config-workflows-library.js
src/pkg/dashboard/static-src/manifest.txt (conflict)
src/pkg/dashboard/static/index.html (conflict)
#17714 ✨ Workflow models: signed bundles, hub marketplace catalogue with nightly refresh, spoke import/publish CONFLICT dashboard/openapi.json (conflict)
src/docs/design/workflow-models.md (conflict)
src/pkg/dashboard/api_workflows.go (conflict)
src/pkg/dashboard/static-src/js/config-workflows-library.js
src/pkg/dashboard/static-src/manifest.txt (conflict)
src/pkg/dashboard/static/index.html (conflict)

@hivecommons-hive

hivecommons-hive Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Why:

  • the branch conflicts with the base and needs a rebase

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

@github-actions

github-actions Bot commented Nov 11, 2026 •

Copy link
Copy Markdown
Contributor

Mutation testing

⏱️ The 900s mutation budget ran out, so 2 package(s) were not fully mutated and the results below do not cover them:

  • pkg/workflowmodel/ingest/cluster: skipped (not started)
  • pkg/workflowmodel/ingest/spec: skipped (not started)

❌ gremlins did not produce a report for 1 package(s), so no verdict can be derived from this run (see the job log):

  • pkg/dashboard: gremlins exited 137

3 mutant(s) in changed files are not reached by any test (NOT COVERED). A maintainer may apply the mutation/acknowledged label to accept this.

File Line Mutation
src/cmd/hive/workflow_ingest.go 59 CONDITIONALS_NEGATION
src/cmd/hive/workflow_ingest.go 63 CONDITIONALS_NEGATION
src/cmd/hive/workflow_ingest.go 63 CONDITIONALS_NEGATION

Verdict: error (#17349).

@clubanderson
clubanderson force-pushed the workflowmodel/v5-p6-ingest-cluster-spec branch from 7cab8fc to ba69f28 Compare November 11, 2026 17:48
@clubanderson clubanderson reopened this Nov 11, 2026
Add pkg/workflowmodel/ingest/cluster: lists tekton.dev Pipelines, Tasks,
PipelineRuns (newest N per pipeline) and Tekton Triggers objects from a
kube context via kubectl, serialises them as cluster/<ns>/<kind>/<name>.yaml
and feeds the existing file-based Tekton ingester so there is one parser.
Last run status per pipeline is recorded as an observed annotation and
coverage finding, never as an inferred lifecycle element. Clear errors for
missing CRDs, RBAC denials, unknown contexts and missing namespaces; caps
on namespaces, runs, objects and bytes.

Refs #17620

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Add pkg/workflowmodel/ingest/spec. Markdown/plain text is split into
heading sections with line ranges; a deterministic stage turns step lists,
conditional sentences and role nouns into a compiling skeleton (states,
columns, chained transitions, manual guards, gates, roles). An optional
LLM stage asks a Completer to refine the skeleton with the Model schema
(reflected from the Go types), the fact vocabulary and the compiler's
refusal rules; the answer is parsed with DisallowUnknownFields, normalised
by Mark (inferred confidence on every state/event/gate, ingested origin,
spec:<heading>#L<a>-L<b> citations validated against real sections,
pipelines/sources/agents stripped, provenance ingested-unreviewed) and
compiled, with one retry quoting the diagnostics before falling back to
the skeleton. This is the only LLM seam in the workflow-model stack.

Refs #17620

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Add POST /api/workflows/ingest/cluster, POST /api/workflows/ingest/spec
and GET /api/workflows/ingest/capabilities in a new handler file,
registered by one line from registerWorkflowRoutes. The cluster route
runs the kubectl-backed fetcher and the existing Tekton ingester; the
spec route runs the deterministic extractor and, when a model gateway is
configured and useLLM is on, the gateway-backed refinement through the
reviewer TurnRunner with a larger completion budget. Owner role, audit
workflow_ingest, same response shape as repo ingest plus source, llm
and sections. 422 when kube access or a gateway is missing.

Wire WorkflowRepoFile, WorkflowClusterRunner and WorkflowSpecLLM deps
from cmd/hive; document the three routes and their schemas additively
in dashboard/openapi.json.

Refs #17620

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Two collapsible panels appended to the Ingest view by a single guarded
hook in renderWorkflowIngest: From a cluster (context select from
kubectl's contexts or a text field, namespace, selector, all-namespaces,
observed-run list) and From a written spec (paste, URL or repository
file; a Use AI assistance checkbox shown only when the capabilities
route reports a gateway, LLM note and section list in the result). Both
reuse the repository ingest's result block, diagnostics and diagram.
Pure renderers over module state, token-only CSS in its own sheet, node
test for gating, modes, request validation and result rendering.

Refs #17620

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
Append the RFC phase 6 section to the design doc — data flow for both
sources, the kubectl seam, the observed-run rule, the LLM boundary and
what enforces it (strict unmarshal, Mark, Compile, one retry, fallback),
prompt outline, failure modes and caps — and add the changelog fragment.

Refs #17620

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
govet's inline check flags the reflect.Ptr alias.

Refs #17620

Signed-off-by: Andrew Anderson <andy@clubanderson.com>
@clubanderson
clubanderson force-pushed the workflowmodel/v5-p6-ingest-cluster-spec branch from ba69f28 to 6a6c71c Compare November 11, 2026 18:20
@kubestellar-prow kubestellar-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Nov 11, 2026
@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Nov 11, 2026
@clubanderson
clubanderson merged commit a3997ca into v5 Nov 11, 2026
84 of 87 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the workflowmodel/v5-p6-ingest-cluster-spec branch November 11, 2026 18:33
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conflicts-with-open-pr Open PR has a merge conflict with another open PR dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant