Skip to content

✨ feat: fence Integration-aware promotion with exact-tested CAS - #15912

Merged
hivecommons-hive[bot] merged 8 commits into
v5from
feat/integration-candidate-execution
Oct 31, 2026
Merged

hivecommons-hive[bot] merged 8 commits into
v5from
feat/integration-candidate-execution

Conversation

@clubanderson

@clubanderson clubanderson commented Oct 31, 2026 •

Copy link
Copy Markdown
Member

Bounded phase one

Adds the real GitHub atomic promotion transport and deterministic protocol regressions. This is a protocol foundation, not an execution engine completion claim. No current automerge lane calls it; existing OFF/advisory/enforce authoring behavior, tracker, preparation/native presentation, native authority and runtime configuration remain unchanged. No activation or deployment.

  • Promote only the existing exact tested commit/tree with ordered parents [expected destination, exact author head]. Initial protocol deliberately excludes squash/rebase/linear-only destinations.
  • Use documented atomic GraphQL updateRefs beforeOid/afterOid/force:false for the destination AND a same-repository author-ref no-op comparison. Concurrent source-head movement prevents the destination update. Fork heads cannot be fenced across repositories and are explicitly unsupported, not downgraded to read-then-write.
  • Require explicit ExecuteJournaled: active persistent journal, fsync before effect and acknowledgement, frozen effective mode; preserve legacy Execute OFF passthrough. Reject missing live guards.
  • Bind the claim to subject/policy/generation and author head/repository/ref; reconcile ambiguous outcomes conservatively without blind TTL retries.
  • Regress destination/source movement after the live guard, wrong tree/parents, incomplete acknowledgements, OFF/missing journals, mode changes, durable replay, provider/protection refusal and unavailable/changed reconciliation.
  • Existing feature documentation records actual producer/permission/protection blockers, remaining phases, research decisions and explicitly hypothetical latency/capacity budgets.

Validation

No local tests/builds/guards run, per AGENTS.md. Changed Go files formatted only; canonical CI must supply the verdict. Required and full applicable workflows must be terminal green on immutable head 6637e09a27beefb2b45e6ef1a50b382c2d69c922 before a guarded source-only merge.

Still required

A real trusted combined-candidate CI producer, durable execution scheduler and separate versioned experimental opt-in, live authority/eligibility wiring, bounded parallel execution and recovery, opt-in tracker diagnostics and comparative benchmark. Conditional no-op provider/protection compatibility requires actual isolated verification before enabling execution. No parity, producer installation or production throughput claim.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. dco-signoff: yes Indicates the PR's author has signed the DCO. labels Oct 31, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign danathar for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. label Oct 31, 2026
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#15921 ✨ feat: establish job-bound Integration-aware candidate CI same files src/docs/integration-orchestration.md

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@clubanderson

Copy link
Copy Markdown
Member Author

Safety correction: existing mutation Boundary exposes reconcile/lookup even with mode OFF, so method availability did not prove journaling. New ExecuteJournaled contract freezes effective mode, refuses OFF/missing persistence, fsyncs intent before the effect and fsyncs the result/directory before acknowledgement. Ordinary Execute retains legacy OFF passthrough. New regressions cover missing/off journals, mode change during effect, cold journal reopen/replay and logging/Crux alias forwarding. Promotion uses only the explicit journal-required contract. This is a new immutable head; all required/full applicable CI must run again. Prior head results are not final proof. No activation or existing-flow change.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@clubanderson

Copy link
Copy Markdown
Member Author

Safety refinement: destination-only CAS cannot fence a concurrent author push. This head adds a second atomic updateRefs condition: the same-repository author ref must still equal the exact tested author head and is left unchanged. Both conditions succeed or neither destination/source update applies. Fork-head promotion is explicitly unsupported because GitHub cannot perform this transaction across repositories; no read-then-write downgrade or protection/permission change. Added deterministic concurrent-source-head regression and provenance binding. Required-context compatibility, trusted combined CI, production live-guard/controller wiring and actual isolated-provider verification remain blockers; this is not execution-ready. Exact-head full CI restarted; prior-head green is not transferred. No local suites/builds or live mutations run.

@kubestellar-prow kubestellar-prow Bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 31, 2026
@clubanderson
clubanderson marked this pull request as ready for review October 31, 2026 01:34
@kubestellar-prow kubestellar-prow Bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Oct 31, 2026

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read integration_promotion.go against the v5 tree, the new ExecuteJournaled contract in crux/effects and convergence/mutation, and the existing reconcile path in client.go:3357-3430. This looks correct to me as a fenced, unactivated transport.

What I verified:

  • Fail-closed boundary chain. integrationPromotionJournaled unwraps LoggingBoundary and requires the terminal boundary to be Reconciler + Lookup + JournaledBoundary (integration_promotion.go:62-81); *mutation.Boundary satisfies all three (boundary.go:177, :252, new ExecuteJournaled). LoggingBoundary.ExecuteJournaled returns ErrDenied rather than passing through when its Next cannot journal (crux/effects/effects.go:308), and mutation.Boundary.ExecuteJournaled denies on off-mode / nil ledger / nil or path-less journal (boundary.go:56-59), so unlike legacy Execute there is no off-mode passthrough here.
  • CAS pre-conditions are checked before the claim is journaled. Tree, both ordered parents and the tested SHA are compared against Git.GetCommit (integration_promotion.go:160-165), HeadRef == Base is rejected in validate() (two updates to one ref in a single updateRefs would be ambiguous), and all-zero / upper-case OIDs are rejected by integrationPromotionOID.
  • Reconcile path matches the existing idiom. On ErrNeedsReconciliation the destination is read once; == TestedSHA → applied, == ExpectedSHA → not applied and safe to retry because the server-side beforeOid is unchanged, anything else → ErrIntegrationDestinationChanged with no retry (integration_promotion.go:207-222). That mirrors client.go:3357-3430 and is covered by TestIntegrationPromotionReconcilesWithoutBlindTTLRetry.
  • frozen := *b in ExecuteJournaled copies a struct with only pointer/func/value fields (Boundary, Executor — no locks), so the mode freeze is safe. syncPersistent holds j.mu before lockAndRefreshLocked, as that helper's contract requires (journal.go:295).
  • No caller of PromoteIntegrationStep exists in the tree, consistent with the body's "no activation" scope.

One open question, not a verified defect: the author-head fence relies on GitHub accepting a refUpdates entry whose beforeOid == afterOid (integration_promotion.go:186-189) as a pure precondition. I could not confirm from the public updateRefs docs that a no-op update is honoured rather than rejected or ignored; TestIntegrationPromotionAuthorHeadMovesAfterLiveGuard exercises the stub, not the live API. Worth a one-off live check before any lane is wired to this, since a rejection would surface as a hard failure rather than a silent downgrade.

CI: all completed checks green at 6ba38d4 (arm64 build still running).

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@clubanderson

Copy link
Copy Markdown
Member Author

Pre-merge protocol review caught one further authority-scope edge: SDK redirects after repository rename/transfer must not let an old enrolled name authorize writes to a different resolved repository. This head rejects URL/path-ambiguous repository identifiers and requires returned repository full_name to match the enrolled scope. Added rename/alias regressions. The atomic-ref race fixture now evaluates the actual submitted ref updates, so concurrent-head tests cannot accidentally pass from a hard-coded expected-source check. No runtime caller or live write added. Prior-head tests/coverage/lint/security were green, images still running; complete CI must reset for this safety correction. No previous-head proof is transferred.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read integration_promotion.go, the new ExecuteJournaled / syncPersistent paths and the existing Boundary.Execute / Executor.Execute they sit on, at merge-base. The transport looks correct to me as a bounded, uncalled protocol; one design note below.

What I verified:

  • Pre-flight is strict and ordered — validate() rejects cross-repo heads, HeadRef == Base, ExpectedSHA == TestedSHA/AuthorHead, non-lowercase/all-zero OIDs and unsafe ref names (integration_promotion.go:45-62, :88-107), then Git.GetCommit is checked for exact tree and ordered parents [ExpectedSHA, AuthorHead] (:155-159) before any claim is taken.
  • Guard runs inside the effect, before the mutation, and its failure is NotApplied so the journal records it as not applied rather than Unknown (:162-165; executor.go:123-126).
  • Reconciliation is conservative — on ErrNeedsReconciliation, ref == TestedSHA → applied; ref == ExpectedSHA → not applied and safe to retry because beforeOid is unchanged; anything else → ErrIntegrationDestinationChanged, no retry (:204-218). LoggingBoundary does implement Reconcile (crux/effects/effects.go:327), so a logging wrapper doesn't silently skip this path.
  • No lock hazards — frozen := *b copies Boundary, whose only pointer-ish members are *Ledger/*Journal (no embedded mutex); syncPersistent takes j.mu, which Begin/RecordResult hold per-call, not across the effect callback (journal.go:377, :420; executor.go:117), so the pre-effect fsync inside the callback cannot deadlock.
  • ExecuteJournaled's pre-checks (boundary.go:55-58) cover every passthrough branch of Execute (nil, !JournalingEnabled, missing ledger/journal), so the "refuses an unjournaled boundary" claim holds.
  • No non-test caller of PromoteIntegrationStep exists, matching the body.

Design note (low, not a blocker): ExecuteJournaled requires JournalingEnabled(mode), which is true for shadow as well as enforce. In shadow, a failed Ledger.Acquire only logs "shadow fence would have denied" and proceeds with Epoch: 0 (boundary.go:110-125), so the hive-side exclusive-writer fence is advisory there; the real protection is the server-side beforeOid CAS plus the live guard. That is consistent with the comment's "journal-required" wording, but the PR title says "fence … with exact-tested CAS" and the IntegrationPromotionGuard doc says the guard "must revalidate the destination's exclusive writer". When a caller is wired, consider requiring FencingEnabled (enforce) for this effect specifically, or state in the method doc that in shadow the ledger does not withhold. Since nothing calls it yet, this is a follow-up, not a defect in this diff.

The body's scope and its "still required" list match the diff; I agree the no-op refUpdate (beforeOid == afterOid) on the author ref needs a live-API check before any activation — the fake server in the tests cannot prove GitHub accepts it.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the new transport against the tree at 6c45a1c (callers in pkg/convergence/mutation and crux/effects). Looks correct to me; nothing I could verify as a defect.

What I checked:

  • No lock inversion in the fsync hook. Journal.syncPersistent takes j.mu (journal.go:621) from inside the effect closure in Boundary.ExecuteJournaled (boundary.go:63-66). That is safe because Executor.Execute releases the journal between Begin and the effect call (executor.go:106 → :121), so the mutex is free when the closure runs. Post-effect sync (boundary.go:70-74) runs after RecordResult and release, so the acknowledgement it syncs is the one the executor just wrote.
  • Reconcile path matches the boundary's contract. On ErrNeedsReconciliation, PromoteIntegrationStep reads the destination and passes ExternalState{} when it still sits at ExpectedSHA (integration_promotion.go:218-224). mutation.Boundary.Reconcile treats !Applied as "authorize the same logical op to retry" (boundary.go:208-235), and the retry re-sends the identical beforeOid, so a stale duplicate cannot overwrite a moved destination. Any other SHA returns ErrIntegrationDestinationChanged without touching the journal — the conservative choice.
  • Mode freeze is real. frozen.Mode = nil after resolving b.Mode() once (boundary.go:52-56) means frozen.Execute cannot re-evaluate the mode mid-operation; Boundary holds no mutex or noCopy, so the value copy is fine.
  • Logging wrapper is unwrapped consistently. integrationPromotionJournaled descends through LoggingBoundary.Next before checking Reconciler/Lookup/JournaledBoundary, and the outer LoggingBoundary also implements Reconcile (effects.go:327) and ExecuteJournaled (effects.go:307), so the later type assertions on the outer value succeed.
  • KindBranchUpdate exists and Claim.Validate accepts the refs/heads/<base> target; changelog fragment name follows the added-<slug>.md convention; docs/integration-orchestration.md describes the same-repo fork restriction, the no-op author-ref fence and the ExecuteJournaled refusal exactly as the code does.

One open question, not a defect, and the PR body already names it: whether GitHub's updateRefs accepts a beforeOid == afterOid no-op entry alongside the destination update is a provider-behaviour fact I cannot verify from the tree. The code fails closed either way (a refusal is journaled as Unknown and reconciled before any retry), so nothing here depends on the answer until a lane actually calls PromoteIntegrationStep — which, as stated, none does.

— hive: agent=reviewer backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

hivecommons-hive Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

Review findings from reviewer-2 (folded here because this head already carries its hive review):

Read the new transport against the tree at 8c2ec04; this looks correct to me. What I checked:

  • No deadlock in ExecuteJournaled (src/pkg/convergence/mutation/boundary.go:47-74): syncPersistent takes j.mu + the flock, but Executor.Execute (executor.go:106,121) releases both inside Journal.Begin before invoking the effect, so the pre-effect sync at boundary.go:63 runs lock-free. The post-effect sync at boundary.go:69-73 runs after RecordResult returns, same story.
  • Mode freeze is real: frozen.Mode = nil (boundary.go:55) means the inner Execute at boundary.go:81 cannot re-read a changed mode mid-operation.
  • Reconciliation path (src/pkg/github/integration_promotion.go:205-234): ref==TestedSHA → Applied; ref==ExpectedSHA → NotApplied then a retry whose server-side beforeOid is unchanged, so a still-in-flight earlier request can only make the retry CAS-fail into Unknown, never double-apply; anything else → ErrIntegrationDestinationChanged with the op left for ExpireUnresolved. That matches the "no blind TTL retry" claim.
  • "No current automerge lane calls it" — verified: zero non-test callers of PromoteIntegrationStep/ExecuteJournaled outside the three defining files.

Open question, not a finding: updateRefs with beforeOid == afterOid on the author ref as a fence is relied on but, per the PR body, unverified against GitHub; nothing here activates it, so that's for the enabling phase.

Review findings from reviewer (folded here because this head already carries its hive review):

Conclusion: read integration_promotion.go in full plus the ExecuteJournaled / syncPersistent additions; the fail-closed shape looks correct to me. No defects found. One open question I could not settle from documentation, below.

What I verified

  • validate() (integration_promotion.go:45-60) rejects cross-repo heads, malformed refs/OIDs, ExpectedSHA==TestedSHA, ExpectedSHA==AuthorHead, zero generation and oversized/control-char labels before any network call.
  • PromoteIntegrationStep refuses nil/unjournaled boundaries (:148-153, with integrationPromotionJournaled unwrapping LoggingBoundary chains), confirms repo identity by node ID + full name, and checks the tested commit's tree and ordered parents ([ExpectedSHA, AuthorHead]) before building the claim — so a reversed-parent merge cannot be promoted.
  • The guard runs inside apply, immediately before the mutation, and its failure is wrapped effects.NotApplied so the ledger does not record an applied effect (:170-172).
  • clientMutationId is the stable digest of the claim and is checked on the way back (:191-193); an incomplete ack falls through to reconciliation rather than being treated as success.
  • Reconciliation (:203-223): ref==TestedSHA → Applied; ref==ExpectedSHA → not applied, retry with the same beforeOid; anything else → ErrIntegrationDestinationChanged. Correct and conservative.
  • mutation.Boundary.ExecuteJournaled (boundary.go:45-72) snapshots mode, requires ledger+journal+path, fsyncs the journal before the effect and again after any invocation/provenance — so a crash between the GraphQL call and the ack leaves durable intent on disk.

Open question (not a finding): the author-head fence relies on GitHub accepting a refUpdates entry with beforeOid == afterOid (:184-186) as a successful no-op CAS. The GraphQL reference for updateRefs documents atomicity and beforeOid preconditions but does not state the behaviour of an unchanged update. Since no lane calls this yet, nothing is exposed today — but it would be worth one live probe against a scratch repo (or a note in docs/integration-orchestration.md citing where it was observed) before the first consumer lands.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Read the diff against the v5 tree at merge-base 85e9ec7, plus the callers it plugs into. Looks correct to me.

What I verified

  • src/pkg/convergence/mutation/boundary.go:47-73 — ExecuteJournaled fails closed on OFF mode / nil ledger / nil journal / empty journal path, freezes Mode() into the copied executor so a mid-operation mode flip cannot change behaviour, and delegates to the existing Execute, so claim acquire/release and ErrAlreadyApplied → idempotent-success (boundary.go:143-157) are reused rather than duplicated.
  • No deadlock from syncPersistent() inside the effect closure: Journal.Begin (journal.go:372) releases j.mu before Executor.Execute runs the effect (executor.go:121), and syncPersistent only takes j.mu + the flock after that. The post-effect sync at boundary.go:67-71 runs after Execute has already released the claim.
  • src/pkg/github/integration_promotion.go:160-176 — the exact-commit check (sha, tree, exactly two parents in order [ExpectedSHA, AuthorHead]) runs before the boundary, and the live guard runs inside the effect, so a guard refusal is journaled NotApplied and never leaves an Unknown entry.
  • updateRefs input shape (repositoryId, refUpdates[].{name,beforeOid,afterOid,force}, clientMutationId) matches the GraphQL API; the author-ref no-op entry (beforeOid == afterOid) is the fork-safe fence the body describes, and validate() (integration_promotion.go:43) refuses cross-repo heads rather than downgrading.
  • Reconcile path (integration_promotion.go:207-234): only TestedSHA / ExpectedSHA observations resolve the journal; anything else returns ErrIntegrationDestinationChanged and leaves the operation unresolved — consistent with "no blind TTL retry".
  • Nothing outside tests calls PromoteIntegrationStep or ExecuteJournaled — matches the "not wired" claim. Existing Execute OFF-mode passthrough is untouched.

One nit (low): the PR body cites "immutable head 6ba38d4e…" but the PR head is 6637e09a…. Worth refreshing the body so the merge guard it describes points at the commit that actually gets merged.

CI at this head is green (all test shards, golangci-lint, gosec, govulncheck). I have not approved — that signature is a maintainer's.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive
hivecommons-hive Bot merged commit 3d03fe8 into v5 Oct 31, 2026
76 of 77 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the feat/integration-candidate-execution branch October 31, 2026 03:19
@hivecommons-hive

Copy link
Copy Markdown
Contributor
Evidence bundle `#159126637e09` sha256:1d37d49fb956477294d2ae29c075bffe4a8bf2563d2376945c672e68d8d6ec39 — download from the dashboard or `hivectl review evidence #15912`.

@github-actions

Copy link
Copy Markdown
Contributor

Thank you for your contribution! Your PR has been merged.

We'd love to hear how your experience was: share feedback

@clubanderson

Copy link
Copy Markdown
Member Author

Actual source receipt: Hive App merged this bounded promotion foundation at 2026-10-31T03:19:44Z, squash 3d03fe8205c9656b38248f5d445805d907bf8b5e, immutable tested head 6637e09a27beefb2b45e6ef1a50b382c2d69c922. Current exact-head CheckRuns are all terminal: 62 SUCCESS and 15 scope-skips; DCO StatusContext SUCCESS; legacy Tide remains PENDING after merge. No manual merge/rerun or gate weakening was performed. This is source-only transport/journal foundation, not controller completion, protection qualification, image publication/deployment or runtime activation. Existing execution remains OFF; no live settings/rules/permissions changed. Producer #15921 and actual engine/controller work remain separately in flight.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dco-signoff: yes Indicates the PR's author has signed the DCO. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant