Repository navigation
✨ evidence: automate reviewed monthly outcomes publication - #16250
Conversation
Signed-off-by: Andy Anderson <andy@clubanderson.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
Open PR overlap checkBase branch: Conflicts are checked with every PR merged onto the current tip of No conflicting open PRs or same-file overlaps were found. |
|
Implementation checkpoint: exact head 19b2ce8 has green Monthly outcomes selftest (including real offline rendering, sanitization, historical cost preservation and local-Git/Mock-GitHub idempotency), build-and-test, build-go, lint, changelog, DCO and native queue policy. No repository tests/build/lints were executed locally. Remaining checks were overlayfs-exec-guard, patch-coverage and Tide at the last bounded read; no failures. This remains a supplied-input publication mechanism, NOT an operational source collector: an authorized evidence owner must supply an approved complete monthly aggregate and confirm the existing publication credential. Source data/credentials were not accessed or provisioned. Prow says /lgtm then root approver hanthor /approve; author clubanderson cannot provide an independent native approval. No merge or policy bypass attempted. Overlap notice with #16241 is only the mandatory trusted-scripts tree literal; whichever merges second must refresh that pin against actual v5 without changing the other PR’s work. |
19b2ce8
ℹ️ Sentinel notice — informational; author is trusted (owner), merge is not blockedHive flagged this PR (author @clubanderson, head
Hive did not add the |
|
Bounded follow-through completed for unchanged head 19b2ce8: every current v5 branch-protection-required check has successful exact-head evidence, and Monthly outcomes selftest, overlayfs-exec-guard, coverage, test and patch-coverage are all terminal SUCCESS. Superseded duplicate DCO-attribution runs were canceled, with the newer required check successful. No source changes were needed. Remaining real gate is Tide: its exact-head status says “Not mergeable. Needs approved, lgtm labels.” No PR reviews are recorded. Clarification to my earlier wording: hanthor is the notifier-selected/requested approver, NOT the sole eligible approver. Current v5 OWNERS lists clubanderson, hanthor, Danathar, nicholasjackson and kellyaa as approvers/reviewers: https://github.com/hivecommons/hive/blob/v5/OWNERS . Author is clubanderson; I will not self-approve or bypass the missing Prow authorization. An eligible other OWNER (hanthor, Danathar, nicholasjackson or kellyaa) can provide the necessary authorized review/approval under Prow. Branch protection currently has required_pull_request_reviews=null; I am not claiming a universal native second-human approval requirement or that two separate reviewers are required. PR/branch remain preserved, not merged. Publication also retains its separate approved supplied-input and existing-credential prerequisites: no operational scheduled private-evidence feed is claimed. |
|
Final moving-base update: while completing the bounded check, v5 advanced through #16241 (ceb80dd), and GitHub now reports mergeable_state=dirty. A read-only git merge-tree against fetched v5 confirms the sole conflict is .github/scripts/integration-governance.cjs, the mandatory TRUSTED_SCRIPTS_TREE literal; no worktree files were changed. Thus there are TWO remaining gates: reconcile that literal against the combined src/scripts tree (then obtain fresh exact-head CI), and obtain the missing eligible Prow approved/lgtm authorization. Current head 19b2ce8 remains clean/preserved with all required checks terminal-success. I am not chasing moving v5 or presenting those results as a verdict on a future conflict-resolution head. The evidence-owner supplied-input/credential prerequisite remains unchanged. |
Merge v5 8664848 without changing landed scripts. Independently reviewed complete scripts-tree union: all 67 existing entries unchanged; only the two monthly automation files and the byte-identical Twitch script from #16241 are added. Pin exactly 96c95ac. No runtime enrollment, privilege, weekly report source or canonical workflow changes. Signed-off-by: Andy Anderson <andy@clubanderson.com>
|
Completed the identified branch-maintenance repair in our isolated feature worktree. New DCO-signed merge head: 5f2e144; normal push, no force/amend. Merged v5 8664848. Before updating the literal, independently checked the COMPLETE old/new src/scripts inventory: 67 common-base entries are byte/mode-identical in both parents and the combined tree; the only additions are monthly-outcomes.py, test-monthly-outcomes.py and the unchanged twitch-channel-cta.sh from landed #16241 (blob 653c90868bee2e2f94fd124f4cc084a48a6402a3). Both original parent pins matched their actual trees. A separate bounded read-only repair review independently verified that exact union, all three additions, modes/blob IDs and trust policy (integration-orchestration.md lines 46–51). Only then pinned the reviewed combined tree 96c95ace27dc711bc1367c6a36e5cd4c9b7153ef. No runtime/profile enrollment, execution authority or privileges changed. Relative to merged v5 the PR remains its original eight-file monthly scope. No weekly source or canonical CI source/template changed; no regeneration was needed. Fresh exact-head CI is now required; previous-head success is not being reused. Independent repair review is NOT Prow merge approval. Missing eligible approved/lgtm authorization remains separate from the operational prerequisite: owner-approved supplied monthly input and existing publication credentials. No live approved source generation or operational private evidence feed is claimed. |
|
Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor. @clubanderson this PR is held at Why:
Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path. |
|
Repair follow-through is terminal and green on unchanged head 5f2e144. After the normal shared quota reset, bounded checks at three-minute intervals confirmed all 11 branch-protection-required contexts SUCCESS, plus Monthly outcomes selftest, overlayfs-exec-guard and patch-coverage SUCCESS (14/14 required/focused contexts, zero pending or failed). No previous-head result was reused. The shared-pin conflict is reconciled, the overlap label has cleared, and the isolated worktree is clean. Only merge-authority gate remains: current Tide explicitly says “Not mergeable. Needs approved, lgtm labels.” There are no PR reviews. Eligible authority comes from OWNERS, not a requested-reviewer suggestion: https://github.com/hivecommons/hive/blob/866484828d73d241ffd7f03a84b1173ede658749/OWNERS . hanthor, Danathar, nicholasjackson or kellyaa can provide authorized non-author review/approval under Prow; clubanderson authored this PR, so I have not self-approved, synthesized labels, bypassed Tide or merged. No universal two-distinct-reviewer rule is asserted. PR and signed branch are preserved for that genuine authorization. Separate operational prerequisite remains unchanged: an evidence owner must supply approved complete monthly aggregates through MONTHLY_OUTCOMES_INPUT and confirm the existing publication credential. CI exercised offline deterministic rendering and mock publication, not live private source collection; no operational scheduled private-evidence feed is claimed. No further source/canonical/weekly changes were made. |
There was a problem hiding this comment.
Correctness review at head 5f2e144. Verified the src/scripts re-pin (git rev-parse 5f2e144:src/scripts = 96c95ace…, matches .github/scripts/integration-governance.cjs:12), ran the 7 non-Go fixtures in src/scripts/test-monthly-outcomes.py locally (pass; the real-renderer fixture passed in CI selftest). One in-scope gap:
monthly-outcomes-pr.sh can orphan a pushed branch and then never open the PR (.github/scripts/monthly-outcomes-pr.sh:23-27, :36-39, :45-47).
git push (line 43) and gh pr create (line 45) are not atomic. If gh pr create fails after the push (rate limit, transient API error), the run exits 1, but every rerun with the same approved input now takes the early-exit at lines 23-27 — gh pr list returns [] (line 9), the remote branch exists (line 14), the provenance digests match — and exits 0 printing "preserving the existing monthly PR" although no PR exists. Reproduced locally with a mock gh whose create fails once: run 1 exit 1, run 2 exit 0, remote branch present, zero PRs. The same happens via the "No monthly report diff" exit at lines 36-39. Net effect: the month is silently never published until the owner changes the input, contradicting the PR body's fail-visibly / one-branch-one-PR claim. Fix: create the PR on both early exits when none exists:
@@ .github/scripts/monthly-outcomes-pr.sh
branch="automation/monthly-outcomes-$MONTH"
pr="$(gh pr list --repo "$GITHUB_REPOSITORY" --head "$branch" --state all --limit 2 --json number,state)"
+ensure_pr() {
+ [ "$(printf '%s' "$pr" | jq length)" -eq 0 ] || return 0
+ gh pr create --repo "$GITHUB_REPOSITORY" --base v5 --head "$branch" \
+ --title "📖 docs: monthly outcomes $MONTH" \
+ --body "Owner-supplied, sanitized monthly aggregates. Review the cohort, observation time, cost denominator and provenance before merging. No raw evidence, stories or deployment identities are published. Collection completeness and publication approval are owner attestations, not independently verified by this workflow. CI and normal review gates must pass; this workflow does not merge."
+}
...
echo "Approved input unchanged; preserving the existing monthly PR without a repost."
+ ensure_pr
exit 0
...
echo "No monthly report diff; no commit, push or stale PR repost."
+ ensure_pr
exit 0
...
git push origin "$branch"
-if [ "$(printf '%s' "$pr" | jq length)" -eq 0 ]; then
- gh pr create ... (existing block)
-fi
+ensure_prWorth adding a 4th attempt to test_publication_one_pr_rerun_no_diff_and_closed_pr where the mock gh create fails once and the rerun with MOCK_PRS='[]' must call create again.
— hive: agent=adjudicator-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88
Merge v5 7f5097c, preserving its landed sources. Retry orphan monthly PR creation on unchanged/no-staged-diff runs without report or provenance commits. Refuse closed/merged PR reposts and branches with no monthly source diff. Add stateful GitHub/local-Git CI regressions for repeated creation failure, recovery and no-op publications. Requested precommit code-review agent found no significant issues. Verified parent scripts pins and combined inventory: only monthly-outcomes.py and test-monthly-outcomes.py differ from v5 scripts tree 9affee8; pin actual combined tree 1e72143. No local tests/builds/lints/guards; CI is the verdict. Signed-off-by: Andy Anderson <andy@clubanderson.com>
|
Owner-directed completion: the owner explicitly requested “finish these”, “we dont need someone elses review” and “get it going”. This is a PR-specific merge decision, not a reviewer approval or fabricated LGTM. Repaired the real push/create atomicity bug: unchanged reruns retry missing PR creation and surface failures without provenance churn; added stateful mock-GitHub/real-local-Git CI regressions. Requested precommit code-review agent found no significant issues. CI-only validation remains mandatory. Once every protected exact-head check, native-queue-policy and focused Monthly outcomes selftest is green and no unresolved hold/security issue remains, use owner-authorized head-matched admin squash only if Prow approval remains. No global rules or security/DCO safeguards are changed. Operational publication still requires approved MONTHLY_OUTCOMES_INPUT and the existing publication credential; no secrets provisioned or private-source collection run. |
|
Completion receipt for exact repaired head 7dcbb72 (not previous-head CI): all 12 protected entries SUCCESS: both Docker
Rechecked GitHub: same head, mergeable=true, no hold labels, no failed/timed-out/cancelled/action-required check runs. The previous correctness finding at 5f2e144 is repaired and covered above. Only pending legacy Prow status is Tide: “Needs approved, lgtm labels.” Per the recorded explicit owner direction, proceeding with PR-specific admin squash matched to this exact head, with a real DCO signoff. No synthetic review/approval/LGTM or repository-wide policy changes. Operational publication is STILL separately gated on owner-approved MONTHLY_OUTCOMES_INPUT and the existing publication credential. This is code/CI completion, not a claim of live private-source collection or an operational source pipeline. |
|
Thank you for your contribution! Your PR has been merged. We'd love to hear how your experience was: share feedback |
|
Verified MERGED at 2026-11-02T20:34:55Z, squash commit a62db5c, from CI-verified head 7dcbb72. Fetched/pulled v5 after merge and verified the squash is an ancestor. Squash commit retains the real Andy Anderson DCO signoff. Cleanup was strictly limited to the worktree and local repair branch created by this completion: removed |
Scope
Automates publication of EXISTING outcome evidence, not a new metrics framework or a hosted deployment collector. Leaves weekly competitor reports, workflow compiler work, root README graphics and the historical October report unchanged.
Source/access assessment
Existing outcome-benchmark.sh reads deployment-local schema-v1 /data/evidence plus optional saved API files; a hosted runner has no assumed source access. New workflow runs 06:17 UTC on the 15th for the prior completed UTC month, with manual dispatch. It fails visibly unless an evidence owner supplies MONTHLY_OUTCOMES_INPUT (approved sanitized aggregate envelope) and the existing TOPUP_PUSH_TOKEN. No credentials were provisioned or expanded and no live collection was run. End-to-end source collection remains blocked on the evidence owner.
Safety and reproducibility
Validation
Independent bounded precommit review completed; fixed full-maturation validation and unchanged-input open-PR provenance churn. Added focused CI fixtures including real existing Go rendering and local-Git/Mock-GitHub PR idempotency. Per AGENTS.md no repository suites, builds, lints or guards were run locally. CI is the executable verdict. Trusted scripts tree literal re-pinned.
Remaining operational gate
An authorized evidence owner must produce a complete observed snapshot, approve aggregates/privacy/cost source, provision the per-month sanitized input through existing secret management, and confirm existing publication credential availability. This PR does NOT claim an operational scheduled source pipeline.