Skip to content

✨ Add integration-aware orchestration above preparation and native queues - #15885

Merged
clubanderson merged 10 commits into
v5from
feat/integration-orchestration
Oct 30, 2026
Merged

clubanderson merged 10 commits into
v5from
feat/integration-orchestration

Conversation

@clubanderson

@clubanderson clubanderson commented Oct 30, 2026 •

Copy link
Copy Markdown
Member

Integration-aware orchestration — default OFF

Bounded capability 3 above the existing preparation/direct authority and GitHub native queue; tracks #15884. This is implementation, not a second merge engine. No deployment, production configuration/activation, gate weakening, or 5,000/day claim.

P3 — durable candidates and actual observations

  • Versioned, cross-process-fenced, fsynced/replayed evidence ledger: exact ordered full-head membership, base, authoritative entry/group, actual tested commit/tree, workflow/producer/policy, run/attempt/generation and invalidation lineage. Late generations cannot revive superseded proof; A+B is never copied to B. Exact documented native prefix promotion retains its original identity.
  • Real signed webhooks and bounded rotating REST repair; read-only native queue/tree/ref provenance and pinned workflow/App/suite/check/job binding. Direct checkout-artifact consumer verifies actual merge parents/tree. Neither producer nor native workflow/provider activation is installed here; unsupported/missing evidence remains unknown.
  • Actual assigned-runner execution accounting clips windows, counts canceled work, separates reruns, excludes queued/skipped work and avoids shared-job double counting. Completeness, unfinished execution, merged denominator and unclassified work are explicit; unknown totals are null.

P4 — actual pre-code coordination

  • Internal and contributor dispatch resolve canonical work into existing approved DAG/ownership, validated prerequisite conditions, versioned anticipated scopes and live claim generations. Common prerequisite ownership is reused, not duplicated.
  • Existing live claims drive shadow same-file/semantic advice, including disjoint-file contract interactions and owner false-positive feedback, without taking advisory reservations. Actual dispatch diagnostics are visible in the existing Features inspector; these boot-local diagnostics are explicitly not durable merge proof.
  • Existing overlap probes feed advisory scope drift. Bounded renewable claim/lease reservations expire, reconcile generation changes and use holder/generation-fenced release. No global file lock or intake freeze.

P5 — real opt-in admission/controller

  • Independent off|advisory|enforce configuration, exact repository/base enrollment and validated bounds. OFF preserves dispatch/observation behavior; ADVISORY cannot block/reorder. ENFORCE controls new scoped authoring admissions and new Hive-owned native enqueues through existing claim/admission and journaled mutation paths.
  • Fixed active/job/runner-minute budgets include explicit outstanding allocations; unknown/stale telemetry freezes growth. Existing priority/own-work bands retain bounded intent aging. Confirmed combined failures reduce future Hive admission only; exact-head retry bounds and the existing owner-first repair route carry exact candidate context. Author-head enrichment requires exact signed suite/check identity, not a guessed details URL or reused rerun status.
  • No native queue jumping, unsupported GitHub build-window control, shared/safety-job cancellation, bypassed CI/review/trust/DCO/coverage/soak/hold/current-base gates, or automatic live pilot/concurrency growth. Rollback releases only this layer's reservations and retains evidence, claims and native queue state.

Entry points and composition

  • Existing Settings → Features controls/inspector; four documented owner API routes under /api/integration-orchestration and /api/config/integration-orchestration.
  • src/docs/integration-orchestration.md documents configuration, producer contracts, controller limits and rollback. Existing vanilla SPA is generated from its manifest; no new tab/framework.
  • Landed preparation snapshot/readState/priority from 🐛 Bound review-slot handoff for active base preparation heads #15886 is reused. Standalone 🐛 Show current preparation backlog without fabricating a ready train #15887 owns tracker/preparation UI; its source landed atcccbf34a4e86e7f7cde03be0d610a84ac0201aae, and this inspector consumes its unchanged cached DTO. Source landing is not a runtime rollout claim.

Validation and honest boundaries

One independent code-reviewer pass found six real defects, corrected before the initial implementation commit. Subsequent implementation/corrections are not claimed to have received another independent pass. Per AGENTS.md, no local Go tests/build/vet/lint or citation guards; changed Go formatted. The canonical citation fixer was used only after CI reported drift.

Terminal exact-head CI receipt

Final source head: 7e69ee2b6ff6ef8606c0a2f5074addce732fc2f0. All applicable exact-head workflows are terminal nonfailing, including full ARM64/main image and contributor build proof: 63 passed, 15 scope-skipped, 0 failed, 0 pending. No Playwright failure was hidden; its workflow also succeeded.

Actual caused CI failures were corrected, not retried blindly: config golden/tooltips/path parity/OpenAPI/canonical citation drift, scheduler sig placement, and the new package's89.9%-versus90 coverage gap. Targeted CI-profile-derived accounting tests, equal-source-time replay/progress, and reordered/partial runner-execution regressions preserve the original floor and gates.

The bounded P3/P4/P5 implementation is complete and committed/pushed. PR deliberately remains draft/open: no feature merge authorization, production configuration/activation, deployment, or runtime/throughput proof is inferred from green CI.

Bounded historical listing overflow deliberately leaves a cohort incomplete; journal caps stop new observations/admission explicitly. There is no scale-grade archive/telemetry, custom batching/speculative-group construction, ML, batch bisection, publication/runtime rollout or P6 empirical throughput proof. Primary paper abstracts challenge WIP/fairness/invalidation economics; they are not Hive production benchmarks.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
Signed-off-by: Andy Anderson <andy@clubanderson.com>
…mission

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the DCO. labels Oct 30, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign nicholasjackson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Oct 30, 2026
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#15877 ✨ feature: show abandoned counts on contributor cards and status CONFLICT src/docs/api-reference/dashboard-misc.md
#15888 ✨ ioscan: accept omp as a tool-less classifier CLI backend same files src/pkg/dashboard/static/index.html
#15871 🐛 fix: stage tracker gives immediate Let-hive-decide feedback and says why a human is wanted same files src/pkg/dashboard/static/index.html
#15849 ✨ Show online ClankeR IDs on the Contributors count pill CONFLICT src/docs/api-reference/dashboard-misc.md
#15892 🐛 fix: preserve merge notifications across reloads and show countdown same files src/pkg/dashboard/static/index.html
#15865 ✨ feature: carry abandoned counts through the hub leaderboard same files src/cmd/hive/main.go
#15859 ✨ feature: add abandoned counts to contributor leaderboards same files src/pkg/dashboard/static/index.html
#15853 ✨ Complete native GitHub queue CI and policy integration same files src/pkg/dashboard/static-src/js/config-features.js
src/pkg/dashboard/static/index.html
#15890 🐛 dashboard: point operator-reference help marks at the section sub-pages same files src/pkg/dashboard/static-src/js/config-features.js
src/pkg/dashboard/static/index.html

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 30, 2026
…ounting

Signed-off-by: Andy Anderson <andy@clubanderson.com>
Signed-off-by: Andy Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 30, 2026
…rovenance

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@github-actions github-actions Bot added the conflicts-with-open-pr Open PR has a merge conflict with another open PR label Oct 30, 2026
…pt context

Signed-off-by: Andy Anderson <andy@clubanderson.com>
…ng gaps

Signed-off-by: Andy Anderson <andy@clubanderson.com>
…ries

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@clubanderson

Copy link
Copy Markdown
Member Author

Completed bounded P3/P4/P5 implementation receipt — source only, no activation or merge authorization.

Exact head: 7e69ee2b6ff6ef8606c0a2f5074addce732fc2f0 (all tracked work committed/pushed with DCO).
Observed 2026-10-30T22:58:12Z: MERGEABLE/CLEAN, draft/open; 63 pass, 15 scope-skipped, 0 fail, 0 pending. The formerly pending full ARM64/main image job actually finished successfully; this receipt does not substitute minimal required checks or stale-head green.

Actual functionality: P3 exact ordered tested-subject/provenance ledger, replay/invalidation and real webhook/bounded workflow-job/candidate/merged-receipt reconciliation; P4 canonical approved-plan/live-claim coordination in both actual authoring lanes, semantic/file advice and drift/false-positive visibility without locks; P5 opt-in scoped atomic admission, fixed execution/fan-out allocations, bounded failure shrink/retry and exact-context owner-first repair. Existing Features/owner API surfaces expose real modes, source/runtime/writer uncertainty and accounting. Rollback retains claims/native state/audit and releases only this layer's reservations.

Safety: default OFF; advisory cannot block/reorder; no new merge engine, copied statuses, native queue jumping, unsupported build-window controls, shared-job cancellation, gate/hold weakening, or live pilot/concurrency changes. Landed #15886 seam/priority and #15887 cached DTO are composed, not duplicated. One independent review pass found six real defects fixed before initial commit; later work is not falsely described as another independent pass.

Honest external/operating limits: Direct producer installation and native trusted workflow/provider activation remain separately owned; absence stays unknown. Deployment prerequisites need an authoritative producer. Historical-page overflow and journal caps remain explicit bounded blockers, not scale-grade telemetry/archive. No custom batching/speculative-group construction, ML, bisection, production rollout, or P6/5,000-per-day proof. Primary abstracts inform challenges, not throughput evidence.

Configuration/API/producer contracts and rollback: src/docs/integration-orchestration.md; existing Settings → Features. Per AGENTS no local tests/build/vet/lint/guards; changed Go gofmt-clean. Canonical citation fixes were only after actual CI drift. Actual CI compile/ratchet/coverage failures were surgically corrected, not ignored or blindly rerun.

@clubanderson
clubanderson marked this pull request as ready for review October 30, 2026 23:01
@kubestellar-prow kubestellar-prow Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 30, 2026
@clubanderson
clubanderson merged commit 726de3c into v5 Oct 30, 2026
79 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the feat/integration-orchestration branch October 30, 2026 23:01
@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

@github-actions

Copy link
Copy Markdown
Contributor

Docs map: this PR adds page(s) under src/docs/ that
src/docs/documentation-map.md does not link:

  • integration-orchestration.md

A page missing from the map is one readers do not find. Please add a
line in the section a reader would actually look in, matching the
format of its neighbours.

If the page is indexed from a subdirectory README, or is deliberately
unlisted for now, ignore this. It is a reminder, not a gate; it never
blocks a merge.

@clubanderson

Copy link
Copy Markdown
Member Author

Cleanup receipt and owner-only bounded-pilot prerequisites

Source merge independently confirmed: #15885 merged2026-10-30T23:01:31Z at 726de3cd704cd6f1c7642d5d323ae052826d1109. No second merge, rollout, activation, configuration mutation or source changes performed.

Cleanup: only /Users/andan02/hc-wt-integration-orchestration switched to detached refreshed origin/v5 at726de3cd. Local feat/integration-orchestration removed; remote was already auto-deleted, confirmed absent; only its stale remote-tracking ref removed. Worktree retained, not force-removed. All22 original .integration-artifacts files (6,047,287bytes) verified SHA256/size-identical across cleanup; this receipt is an additional artifact. No shared checkout touched.

Required owner actions before a bounded pilot (not satisfied merely by source merge or green CI):

  1. Runtime/writer attribution: verify an explicitly selected deployed image/version contains this source, read effective configuration and actual preparation/direct/native writer ownership, and confirm this layer remains OFF initially. Parent owns publication/roll attribution. Source merge and CI images are not runtime proof.

  2. Authoritative producer contracts: configure exact repo, base, engine, workflow path, full workflow blobSHA, producer AppID and required job names. Workflow blob must match both the tested revision and the trusted target base; unlanded producer code cannot bootstrap trusted proof. Validate actual run/attempt, source+target repo, suite/App/check/job identity and exact tested commit/tree/member heads. Author review receipts remain separate; one workflow receipt is not aggregate merge permission.

    • Direct: an external trusted publisher must actually install/publish attempt-named hive-integration-subject-v1-<attempt> with only subject.json: version, repo/base/full baseSHA, actual tested mergeSHA/tree, runID/attempt, configured policy fingerprint and exact PR/full-head members. It must attest actual checkout with base+author merge parents, not echo green author statuses. Publisher installation is UNIMPLEMENTED externally and blocks direct combined-state proof while absent.
    • Native: separately land and verify trusted group-aware workflows/providers (including member-aware DCO/attribution and every required context on actual merge_group SHA), GitHub native queue availability/protection/App transport and explicit repository activation. ✨ Complete native GitHub queue CI and policy integration #15853/native readiness and external provider activation are not supplied by this feature. Validate two-member combined success/failure, head/hold/review/removal and documented prefix behavior in an approved non-production window. Coordinate external branch updaters and existing preparation writer; GitHub retains final merge authority. No unsupported build-window control or implicit current-base-gate changes.
  3. Qualified work and telemetry: explicit small repo/base allowlist; current approved canonical DAG, unambiguous node/revision linkage, anticipated scope/provenance and live ownership. Integration prerequisites need authoritative merged-PR receipts; deployment prerequisites remain UNKNOWN without an external deployment producer. Complete, fresh actual run/job/attempt/candidate and merged-denominator observations are required before enforcement growth. Overflow is a BLOCKER, never harmless or zero:100-item historical listing bound;8relevant runs/scope,24attempt-job reads and4policy reads per rotating3-minute/45-second reconciliation batch. Historical overflow can leave a cohort permanently incomplete under current implementation; choose a genuinely qualified smaller cohort or separately implement authoritative scale-grade telemetry.

  4. Durability and operating approval: stable writable durable journal/lease storage with appropriate instance/writer coordination and restart proof. Current256MiB/100,000-record/1MiB-record caps and corruption refusal explicitly block new observations/admission; automatic archive/compaction is UNIMPLEMENTED, not an ignorable condition. Set owner-approved fixed active/job/runner-minute allocations and retry/lease/age bounds; do not infer capacity from unknown data. Start ADVISORY, inspect actual advice/invalidation/cost/completeness and normal gates, then separately authorize ENFORCE only for the qualified scope. Missing telemetry freezes growth. Maintain rollback to OFF, preserving audit/claims/native entries. No automatic live pilot, concurrency expansion, custom batching or P6/5,000-per-day proof.

Exact configuration, owner API and producer contracts: src/docs/integration-orchestration.md. Default OFF is preserved; no activation or rollout performed during cleanup.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

conflicts-with-open-pr Open PR has a merge conflict with another open PR dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant