Repository navigation
✨ workflow models P3a: ingest → lifecycle models, workflows ingest API, survey compile stats - #17652
Conversation
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of No conflicting open PRs or same-file overlaps were found. |
There was a problem hiding this comment.
Intent alignment (head 7ecc5c8): the diff implements what the body claims — ingest.Lifecycle composes hive-default + ingested pipelines (src/pkg/workflowmodel/ingest/lifecycle.go:29-98), owner-gated POST /api/workflows/ingest with {repo}/{files}, next-integer-version save and workflow_ingest audit (src/pkg/dashboard/api_workflows_ingest.go:35-118), pipeline.status reports unknown (src/pkg/stageengine/workflow.go:279-281), survey compile stats. No scope creep. Approving.
One advisory, non-blocking:
dashboard/openapi.jsondocuments503as "No forge client", butsrc/cmd/hive/main.go:1090always wiresWorkflowRepoFiles, so the handler's 503 branch (api_workflows_ingest.go:67-70) is only reachable in tests. A hive without a GitHub client gets502 fetch owner/name: no GitHub clientfromsrc/cmd/hive/workflow_ingest.go:22-25instead. Either drop "No forge client" from the 503 description, or reuse the existinggithub.ErrNoGitHubClientsentinel (already returned byFetchDefaultBranchTree,src/pkg/github/complexity_source.go:29-31) and map it to 503 in the handler.
— hive: agent=ci-maintainer backend=copilot model=claude-fable-5.1 copilot=1.0.88
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. Review findings from Security perspective (read at head 7ecc5c8). Owner gate ( Low — with, at package level: // workflowRepoRe is a GitHub owner/name pair: no extra slashes, dots-only
// segments, or URL metacharacters that could reshape the API path.
var workflowRepoRe = regexp.MustCompile(`^[A-Za-z0-9](?:[A-Za-z0-9-]{0,38})/[A-Za-z0-9_.-]{1,100}$`)Review findings from Correctness review (head Concurrent
|
7ecc5c8 to
9cb51b1
Compare
0812411 to
20438d7
Compare
9cb51b1 to
5c3b66b
Compare
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
…rvey compile stats (RFC #17620 P3a) ingest.Lifecycle composes hive-default's compiled lifecycle with a repository's pipelines: PR-verify pipelines on the CI / Review states with citations, the post-merge chain as inferred columns after Merged, provenance ingested-unreviewed. New pipeline.status fact (engine reports unknown until post-merge runs are observed). ingest.ReadTree selects CI files from a checkout. POST /api/workflows/ingest fetches a repo's default branch read-only (or takes pasted files), compiles, optionally saves at the next integer version. workflow-survey compiles every corpus repo's lifecycle model: 317/317 compile, 0 refusals. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
There was a problem hiding this comment.
Read the tree at 5c3b66b, not just the diff: ingest/lifecycle.go, hivedefault/hivedefault.go, compile.go (exits/roles), store/store.go (Save/checkKey), api_workflows_ingest.go, cmd/hive/workflow_ingest.go, github/complexity_source.go. Looks correct to me; nothing blocking.
What I verified:
- Edges out of a final state are legal.
Lifecycleappendsmerged → <post-merge>transitions while hivedefault marksmergedType: StateFinal(hivedefault.go:170).compile.go:241-251only requires non-terminal states to have an exit and never rejects an exit from a terminal, so the chain compiles andmergedstays inp.Terminal— which is exactly whatTestLifecyclePostMergeChainasserts (lifecycle_test.go:99). - No ID collisions. Ingested states are
ci/main-ci/release/deploy-*/done; hivedefault's areopen/merged/closedplusStateID(stage). The post-merge filter atlifecycle.go:62dropsciso it isn't re-added as a column. - Store path safety.
req.Nameonly reaches disk viast.Save, which runscheckKey(store.go:109-112,[a-z0-9-]regex) beforemodelPath. Unsaved responses echo it back unvalidated, which is fine. - Repo string → no shell.
FetchDefaultBranchTreesplits on/, hits the REST tarball endpoint and extracts to a temp dir (complexity_source.go:32-68);ReadTreeskips non-regular files (so symlinks) and caps atMaxTreeFiles. Owner-gated viarequireOwnerRole; body bounded bydecodeBody'sMaxBytesReader.
Two low notes, not defects:
api_workflows_ingest.go:81hardcodesScope{Forge: "github.com"}.Scope.Forgeisn't matched on anywhere inworkflowmodel/stageenginetoday, so it's cosmetic, buts.deps.Config.GitHub.Forge()is already how the rest of the dashboard names the host and GHE hives will otherwise store a wrong forge in saved models. Worth a one-liner here or in P3b.- The red
Sign-off survives the squash/overlapchecks: the PR's single commit is signed off (Signed-off-by: Andrew Anderson), and the overlap is with #17648 by construction (stacked onworkflowmodel/p1-compiler). Both should clear once P2 lands and this is retargeted; I don't see anything in the diff causing them.
The body's claims match the code (ingest.Repository untouched; pipeline.status added to the vocabulary at stageengine/workflow.go; OpenAPI updated).
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
eda58d2 to
caaf0d0
Compare
There was a problem hiding this comment.
Read the diff against the #17648 head (2b97f5b) plus the code it composes on (workflowmodel/defaults.go Bind, hivedefault.Generate, compile.go, store/store.go, github/complexity_source.go). Looks correct to me; one low-severity note.
Verified
ingest.Lifecycle(src/pkg/workflowmodel/ingest/lifecycle.go:30-98) keepsmergedasfinalwhile appending post-merge transitions out of it — unlikeBind(defaults.go:139-141), which flips it to sequential. The compiler does not reject exits from a terminal state (compile.go:243-251only checks the reverse), andlifecycle_test.go:99assertsTerminal[merged]on purpose, so a bound ingested model still derivesmergedas terminal exactly like hive-default. Intentional, matches the body's "derives exactly like hive-default today"; just noting the asymmetry withBindfor whoever picks up P5 whenpipeline.statusbecomes observable.- Post-merge state/column ids (
main-ci,release,deploy-<env>,deploy,done) cannot collide with hive-default's state ids (hivedefault.StateID→ stage slugs /needs-decision-*) or its column ids (hivedefault.go:37-43). ReadTreeskips non-regular files (tree.go:48), so symlinks in a fetched tarball are not followed;skipDirsandMaxTreeFilesbound the walk.nextWorkflowVersion→st.Versions(name)filters the in-memory index (store.go:310-322), never builds a path from the user-suppliedname;SaverunscheckKey(store.go:241,nameReexcludes/and.) before any write.hive-defaultasnameis refused byIsBuiltin.- Request body is capped by
decodeBody→http.MaxBytesReader; the forge fetch is bounded byworkflowIngestTimeouton top ofcomplexityFetchTimeout. pipeline.statusis added to both the fact vocabulary (lifecycle.go:191) andscalarFact(stageengine/workflow.go:279), soTestRulesFactVocabularyIsCompletestays honest.
Low — audit covers only the save path (src/pkg/dashboard/api_workflows_ingest.go:96 returns before :115)
A POST /api/workflows/ingest {repo} without save: true downloads an arbitrary repository's default branch with the hive's App credential (:74) and leaves no audit entry. The body says "Audited as workflow_ingest" and docs/design/workflow-models.md:118 says owner writes are audited, which is true — but this read on the hive's credential is the more interesting event to have a trail for (which repos an owner pointed the hive at). Suggest moving auditFromRequest above the !req.Save return (with e.Key() replaced by the model name), or qualifying the sentence. Not a blocker: the endpoint is owner-gated.
No scope creep beyond the P3a list in the body; ingest.Repository and its goldens are untouched as stated.
— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88
b1627e0
into
workflowmodel/p1-compiler
|
merged by Hive trusted-author auto-merge: author @clubanderson holds role owner and GitHub permission admin on repo; CI green at caaf0d0. |
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
…rvey compile stats (RFC #17620 P3a) (#17652) ingest.Lifecycle composes hive-default's compiled lifecycle with a repository's pipelines: PR-verify pipelines on the CI / Review states with citations, the post-merge chain as inferred columns after Merged, provenance ingested-unreviewed. New pipeline.status fact (engine reports unknown until post-merge runs are observed). ingest.ReadTree selects CI files from a checkout. POST /api/workflows/ingest fetches a repo's default branch read-only (or takes pasted files), compiles, optionally saves at the next integer version. workflow-survey compiles every corpus repo's lifecycle model: 317/317 compile, 0 refusals. Signed-off-by: Andrew Anderson <andy@clubanderson.com>
…rkflows API, repo bindings, tracker columns (#17648) * ✨ workflowmodel: lifecycle schema, compiler, hive-default generator, DeriveFromModel (#17620 P1) Extends the v1alpha1 schema with columns, derivation, events, gates, roles, agents, checks, bookkeeping, labels and meta; adds Compile with coded refusals; generates hive-default from the stage table and the engine's expected edges; adds stageengine.DeriveFromModel (builtin delegation + rules evaluator) and Options.Workflow; adds the workflow.enabled flag (default false, no wiring effect yet). The PR stage-golden corpus derives identically under hive-default. Signed-off-by: Andrew Anderson <andy@clubanderson.com> (cherry picked from commit 43eb8b2) * ✨ workflow models P2: store, /api/workflows, repo bindings, engine wiring, tracker columns (#17620) (#17649) * ✨ workflowmodel: store, /api/workflows, repo bindings (RFC #17620 P2a) Signed-off-by: Andrew Anderson <andy@clubanderson.com> * ✨ stageengine: per-repo workflow binding, tracker workflows block (RFC #17620 P2b) Signed-off-by: Andrew Anderson <andy@clubanderson.com> * workflow models P2: Settings/tracker UI, openapi, gates, docs (P2c) Per-repo Workflow select on Settings → Repos, tracker workflows block rendering (model column header, per-row strip, Not covered chevron), openapi routes/schemas, import ratchet and settings-parity allowlist, regenerated agent-runtime golden, phase-2 design notes and fragment. Refs #17620 Signed-off-by: Andrew Anderson <andy@clubanderson.com> --------- Signed-off-by: Andrew Anderson <andy@clubanderson.com> * workflowmodel P2: refresh api-reference citations, tokenize badge padding, lower style ratchet baselines Signed-off-by: Andrew Anderson <andy@clubanderson.com> * testutil: allow pkg/dashboard → pkg/workflowmodel in import ratchet (RFC #17620 P2) Signed-off-by: Andrew Anderson <andy@clubanderson.com> * ✨ workflowmodel: ingest → lifecycle models, /api/workflows/ingest, survey compile stats (RFC #17620 P3a) (#17652) ingest.Lifecycle composes hive-default's compiled lifecycle with a repository's pipelines: PR-verify pipelines on the CI / Review states with citations, the post-merge chain as inferred columns after Merged, provenance ingested-unreviewed. New pipeline.status fact (engine reports unknown until post-merge runs are observed). ingest.ReadTree selects CI files from a checkout. POST /api/workflows/ingest fetches a repo's default branch read-only (or takes pasted files), compiles, optionally saves at the next integer version. workflow-survey compiles every corpus repo's lifecycle model: 317/317 compile, 0 refusals. Signed-off-by: Andrew Anderson <andy@clubanderson.com> --------- Signed-off-by: Andrew Anderson <andy@clubanderson.com> Co-authored-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com>
…tore, workflows API, repo bindings, tracker columns (#17679) * ✨ workflowmodel: phase-1 ingest (GitHub Actions, Tekton), renderers and survey harness (#17620) (#17630) * ✨ workflowmodel: phase-1 ingest (GitHub Actions, Tekton), renderers and survey harness (#17620) Deterministic ingest of a repository's GitHub Actions workflows and Tekton / OpenShift Pipelines resources into one content-hashed workflow model bound onto the hive default lifecycle; Mermaid and Activity Monitor column renderers; cmd/workflow-survey proving harness; golden fixtures from six real repositories. Nothing wires into the stage engine yet. Signed-off-by: Andrew Anderson <andy@clubanderson.com> * workflow-survey: make main testable, drop unused const golangci-lint flagged maxFileBytes as unused and the coverage gate hard-fails any new cmd package without tests. Split main into realMain(args, stdout, stderr) and run over the curated fixture corpus in-process (96.6% statement coverage). Signed-off-by: Andrew Anderson <andy@clubanderson.com> --------- Signed-off-by: Andrew Anderson <andy@clubanderson.com> (cherry picked from commit a59684c) * ✨ workflow models P1+P2: schema, compiler, DeriveFromModel, store, workflows API, repo bindings, tracker columns (#17648) * ✨ workflowmodel: lifecycle schema, compiler, hive-default generator, DeriveFromModel (#17620 P1) Extends the v1alpha1 schema with columns, derivation, events, gates, roles, agents, checks, bookkeeping, labels and meta; adds Compile with coded refusals; generates hive-default from the stage table and the engine's expected edges; adds stageengine.DeriveFromModel (builtin delegation + rules evaluator) and Options.Workflow; adds the workflow.enabled flag (default false, no wiring effect yet). The PR stage-golden corpus derives identically under hive-default. Signed-off-by: Andrew Anderson <andy@clubanderson.com> (cherry picked from commit 43eb8b2) * ✨ workflow models P2: store, /api/workflows, repo bindings, engine wiring, tracker columns (#17620) (#17649) * ✨ workflowmodel: store, /api/workflows, repo bindings (RFC #17620 P2a) Signed-off-by: Andrew Anderson <andy@clubanderson.com> * ✨ stageengine: per-repo workflow binding, tracker workflows block (RFC #17620 P2b) Signed-off-by: Andrew Anderson <andy@clubanderson.com> * workflow models P2: Settings/tracker UI, openapi, gates, docs (P2c) Per-repo Workflow select on Settings → Repos, tracker workflows block rendering (model column header, per-row strip, Not covered chevron), openapi routes/schemas, import ratchet and settings-parity allowlist, regenerated agent-runtime golden, phase-2 design notes and fragment. Refs #17620 Signed-off-by: Andrew Anderson <andy@clubanderson.com> --------- Signed-off-by: Andrew Anderson <andy@clubanderson.com> * workflowmodel P2: refresh api-reference citations, tokenize badge padding, lower style ratchet baselines Signed-off-by: Andrew Anderson <andy@clubanderson.com> * testutil: allow pkg/dashboard → pkg/workflowmodel in import ratchet (RFC #17620 P2) Signed-off-by: Andrew Anderson <andy@clubanderson.com> * ✨ workflowmodel: ingest → lifecycle models, /api/workflows/ingest, survey compile stats (RFC #17620 P3a) (#17652) ingest.Lifecycle composes hive-default's compiled lifecycle with a repository's pipelines: PR-verify pipelines on the CI / Review states with citations, the post-merge chain as inferred columns after Merged, provenance ingested-unreviewed. New pipeline.status fact (engine reports unknown until post-merge runs are observed). ingest.ReadTree selects CI files from a checkout. POST /api/workflows/ingest fetches a repo's default branch read-only (or takes pasted files), compiles, optionally saves at the next integer version. workflow-survey compiles every corpus repo's lifecycle model: 317/317 compile, 0 refusals. Signed-off-by: Andrew Anderson <andy@clubanderson.com> --------- Signed-off-by: Andrew Anderson <andy@clubanderson.com> Co-authored-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com> (cherry picked from commit 1217d64) --------- Signed-off-by: Andrew Anderson <andy@clubanderson.com> Co-authored-by: hivecommons-hive[bot] <280983584+hivecommons-hive[bot]@users.noreply.github.com>
Stacked on #17648 (P2). Phase 3a of RFC #17620: an ingested repository now yields a compiling lifecycle model, reachable from the spoke API.
What's here
ingest.Lifecycle(scope, files)— hive-default's compiled lifecycle + the repo's pipelines: PR-verify pipelines bound to the CI / Review states (with file citations), the post-merge chain (main CI · release · deploy- · done) as new columns after Merged, every ingested elementconfidence: inferred, provenanceingested-unreviewed.ingest.Repository(phase 0) is untouched; its goldens hold.pipeline.statusfact — what the post-merge chain waits on. The engine reportsunknownuntil it observes post-merge runs, so a bound ingested model derives exactly like hive-default today (extra columns render, nothing lands in them yet). Honest rather than invented.ingest.ReadTree(dir)— selects CI files from a checkout with the survey collector's rules (GHA workflows, Tekton dirs, pipeline-named YAML that looks like Tekton; skips vendor/testdata/node_modules; capped).POST /api/workflows/ingest— owner-gated.{repo}fetches the default branch read-only viaFetchDefaultBranchTree(newDependencies.WorkflowRepoFiles, wired incmd/hive), or{files}for pasted definitions. Returns model + compile result + diagnostics;save: truestores at the next integer version. Audited asworkflow_ingest. OpenAPI updated.cmd/workflow-survey— compiles every repo's lifecycle model (lifecycle.json), reports pass/refusal by code.Corpus result (317 CNCF repos)
inferred-elementwarningsColumn counts per model (hive-default is 7): 7 cols × 28, 9 cols × 65, 10 cols × 208, 11 cols × 11, 12 cols × 3, 14 cols × 1, 15 cols × 1.
Tests
TestLifecycleCorpusCompiles(every testdata corpus repo compiles, hive-default states intact, inferred warnings present), post-merge chain + citations, empty-repo parity,ReadTreeselection, API (files/repo/guards/version bump),cmd/hivefetcher. Vocabulary-completeness test extended. golangci-lint clean.Next
P3b: Settings → Workflows tab (library · ingest form · read-only diagram), then P4 editor.