Skip to content

🐛 fix(ci): report unavailable DCO enumeration without weakening admission - #16442

Merged
kubestellar-prow[bot] merged 1 commit into
hivecommons:v5from
thatguywob:fix/16404-dco-provider-unavailable
Nov 3, 2026
Merged

kubestellar-prow[bot] merged 1 commit into
hivecommons:v5from
thatguywob:fix/16404-dco-provider-unavailable

Conversation

@thatguywob

Copy link
Copy Markdown
Contributor

When App 1861 cannot enumerate a large PR's complete history, native-queue-policy currently reports only DCO: failure. This change reports provider unavailable / verdict UNKNOWN in the job error and Actions summary, while keeping admission failed. It retains the authoritative current-head success requirement and makes no permission or protection changes.

Following the maintainer's fourth direction, this also adds a read-only full-history reproducer comparing the provider's 100-parent selection with the proposed 2-parent selection, plus a recovery runbook. The runbook links both observed #16378 failures, traces the provider fallback, and identifies dcoapp/app#339 as an unmerged candidate, not a verified fix. It explains how the provider operator obtains the hidden exception and reruns the authoritative check after recovery, preserving merge ancestry.

Refs #16404 — repository diagnostics, reproducer and recovery guidance are implemented here; provider-side enumeration repair/deployment and a genuine current-head App-1861 DCO success remain outstanding. This PR does not unblock #16378 by itself.

Validation:

  • Added regressions for UNKNOWN blocking both admission paths, trust/head validation before classification, ordinary failures remaining distinct, and authoritative success restoring admission.
  • Added a 251-commit fixture retaining an unsigned final commit beyond the REST ceiling, with incomplete/duplicate/changed history and GraphQL-error cases. Wired tests into PR CI.
  • Tests not run locally per AGENTS.md; CI is the execution verdict. git diff --check passed.
  • Verified public check-run evidence and pinned provider source. Live GraphQL reproduction was prevented by this contributor environment's POST gate; no successful live query or hosted-provider recovery is claimed.

— hive: backend=codex


🐝 Hive Agent: contributor | SHA: 0ac19dfa3

…sion

Distinguish the authoritative DCO provider's incomplete-history UNKNOWN
diagnostic from a sign-off failure while keeping both admission paths
closed. Show an operator-directed Actions summary and retain the current
App/head/provenance requirements without retries or status substitution.

Add a read-only full-history query reproducer, regression fixtures beyond
the REST ceiling, and a recovery runbook linking the observed failures
and the unmerged provider-side repair candidate.

Hive-Run: hivecommons#16404
Hive-Plan: dco-provider-diagnosis-and-reporting
Hive-Spec: hivecommons#16404#maintainer-fourth-direction
Signed-off-by: thatguywob <thatguywob@users.noreply.github.com>
@kubestellar-prow kubestellar-prow Bot added the dco-signoff: yes Indicates the PR's author has signed the DCO. label Nov 3, 2026
@thatguywob
thatguywob requested a review from Danathar as a code owner November 3, 2026 16:34
@kubestellar-prow kubestellar-prow Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 3, 2026
@github-actions

github-actions Bot commented Nov 3, 2026

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

No conflicting open PRs or same-file overlaps were found.

@hivecommons-hive

Copy link
Copy Markdown
Contributor
0ac19df

⚠️ Sentinel alert — maintainer review required

Hive flagged this PR (author @thatguywob, head 0ac19dfa36e6) because it matches behaviors that can override security controls, escalate privileges or damage the codebase. This is a heuristic, not an accusation — a maintainer should confirm the change is intended before it merges.

  • sensitive_path — changes 2 sensitive path(s) (Touches a sensitive path (OWNERS, workflows, policies, hive config, security docs, …))
    • .github/workflows/native-queue-policy.yml
    • .github/workflows/pr-auto-update-test.yml

Hive added the sentinel-alert label. While it is present, Hive will not approve this PR, apply LGTM/approval labels, or merge it through any auto-merge lane. Remove the label once reviewed; Hive will not re-apply it unless new commits are pushed. Tune paths and behaviors under sentinel in hive.yaml or the dashboard Security tab.

@hivecommons-hive hivecommons-hive Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs-currency review (head 0ac19dfa3) — operator-facing text matches the code: the UNKNOWN classification order (native-queue-policy.cjs:93-98), the fail-closed DCO: failure fallback, the job-summary write in native-queue-policy.yml, and both relative links resolve under check-docs-links.py src/docs / docs. No blocker.

One discoverability gap, advisory:

  • docs/operations/dco-provider-unavailable.md is a new recovery runbook in a brand-new docs/operations/ directory, but it is not linked from src/docs/documentation-map.md, and docs-index-reminder.yml only scans top-level src/docs/*.md, so nothing will flag it. The existing recovery runbooks are indexed under "Recovery, moves, and fleet health" (documentation-map.md:72-73), and root-docs/ pages are already linked from the map via ../../docs/... (e.g. documentation-map.md:132). Suggested addition after line 73:
- [Agent backend-auth failure recovery runbook](backend-auth-recovery-runbook.md) — identify affected agents, re-authenticate the backend, and clear recovered fleet reports.
- [DCO provider unavailable runbook](../../docs/operations/dco-provider-unavailable.md) — what `native-queue-policy` reports when App 1861 cannot enumerate a large PR's commit list (verdict UNKNOWN), the read-only enumeration reproducer, and how the provider operator recovers and re-runs the authoritative check.

— hive: agent=reviewer-2 backend=copilot model=claude-fable-5.1 copilot=1.0.88

@hivecommons-hive

hivecommons-hive Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

Review findings from reviewer (folded here because this head already carries its hive review):

Routing to a maintainer, not blocking on code. This PR carries sentinel-alert because it edits the merge-admission workflow and its policy script. I read the files at 0ac19dfa3 against origin/v5; the facts a maintainer needs:

  • Permissions unchanged. native-queue-policy.yml keeps contents/pull-requests/checks/actions: read and still checks out the base branch under pull_request_target; the diff touches only the test step and the github-script loop.
  • Admission is not weakened. evidenceForHead (native-queue-policy.cjs:97) throws the new ProviderUnavailable after the existing app.id/head_sha trust check (:93-95), and only when run.name === 'DCO', conclusion === 'failure' and the summary text matches the provider's "could not be evaluated" message. On the base branch that same run already threw a plain Error('DCO: failure'); the only behavioural change is that the workflow now exits the 90×15 s poll loop immediately with a summary and core.setFailed instead of throwing — the job still fails, so the queue still blocks.
  • New script is read-only. diagnose-dco-provider.cjs is an operator reproducer (REST + GraphQL commit enumeration via gh api); it is only wired into node --test, not into any job that runs with credentials.
  • Author thatguywob is not in OWNERS (approvers: clubanderson, hanthor, Danathar, nicholasjackson, kellyaa) and pushes from a fork.

I found nothing wrong with the change itself; whether a workflow/admission-policy edit from outside OWNERS should land is the decision.

Review findings from adjudicator (folded here because this head already carries its hive review):

Style/maintainability review (head 0ac19dfa) — no blocker; a few conventions worth tightening before merge.

  1. Runbook location breaks the existing doc layout. docs/operations/ is a new directory created by this PR for a single file, while the repo already keeps operator runbooks in src/docs/operator-reference/ (runtime-and-quota.md, sweeps-and-operations.md, indexed by its README.md). As a result src/docs/native-merge-queue.md:376 has to reach out of the tree via ../../docs/operations/…, and the path is also hard-coded in the ProviderUnavailable message (.github/scripts/native-queue-policy.cjs:36). Suggest moving the file to src/docs/operator-reference/dco-provider-unavailable.md, linking it as operator-reference/dco-provider-unavailable.md from native-merge-queue.md, adding a section/entry in operator-reference/README.md, and updating the string at native-queue-policy.cjs:36.

  2. Admission job now depends on an unrelated tool's tests. .github/workflows/native-queue-policy.yml:36 adds diagnose-dco-provider.test.cjs to the "Test policy behavior" step that gates every PR/merge-group admission. The diagnose script is a standalone operator CLI with no import from the policy module, and pr-auto-update-test.yml:57 already runs its tests on the relevant paths. A regression in the reproducer's mocks would block all admissions for no policy reason. Suggest reverting that one line:

            run: node --test .github/scripts/native-queue-policy.test.cjs
    
  3. Provider-text coupling is inlined and duplicated. dcoEnumerationUnavailable (native-queue-policy.cjs:40-47) embeds two long dcoapp/app literals inside the predicate, and native-queue-policy.test.cjs:13-15 repeats them. Hoisting them to named constants (e.g. DCO_UNAVAILABLE_PREFIX, DCO_UNAVAILABLE_MARKER) next to DCO_APP_ID makes the third-party coupling visible in one place and lets the test import rather than re-type them. Also, ProviderUnavailable is appended as a trailing line in module.exports (:267) rather than alongside PendingEvidence on :265.

  4. Weak negative assertions in the reproducer test. .github/scripts/diagnose-dco-provider.test.cjs:72 uses await assert.rejects(diagnose(options, api)) with no matcher for all seven mutations, so any rejection — including a TypeError from a bug in the loop — satisfies the test. The changeFinal loop below it (:79) does pass a regex; the mutation loop should pair each mutation with its expected message (/GraphQL errors/, /head changed/, /totalCount/, /Incomplete history/, /duplicate/, /did not advance/, /pageInfo/).

Minor: diagnose-dco-provider.cjs:77 page + 1 >= Math.ceil(expected / 100) + 1 reads like an off-by-one; if the extra page of slack is intentional, a comment or const maxPages = … would help.

@hivecommons-hive

hivecommons-hive Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Needs a decision, not a merge:

  • the reviewer swarm returned requires_human @0ac19df
  • the review found: the reviewer marked this change high-risk
  • reviewer question: @clubanderson should decide whether this change to the admission workflow (.github/workflows/native-queue-policy.yml + native-queue-policy.cjs) from a non-OWNERS fork author may land; I verified it does not weaken admission, but workflow/admission policy edits need a maintainer's sign-off.
  • the automerge lane will not merge this unattended

Head: verdict requires_human @0ac19df, 65/68 checks green, 3 pending, DCO yes, soak 0s of 10m0s.

How to release it:

  • After reading the verdict, a trusted merger can submit a GitHub Approve review for the current head and the lane clears the hold on its next pass; /hive approve replies on a PR are acknowledged but not consumed. An ordinary contributor cannot release their own PR.

Clear the cause and the hive re-evaluates on its next pass; the stage moves to stage/090-ready-for-human-merge, stage/095-pre-merge-soak or the merge queue on its own. Approval does not bypass CI, hold labels or other merge gates.

Updated 2026-11-03 21:17 UTC.

@clubanderson clubanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Owner decision: approve this exact head for the scoped repository diagnostics, read-only reproducer and recovery guidance requested in #16404. Reviewed the complete eight-file delta. Provider, exact-head and completed-status checks still precede UNKNOWN classification; UNKNOWN remains a failed admission result, never substituted success or PendingEvidence. The trusted-base workflow keeps the original permission and checkout boundaries. The reproducer only reads complete history, rejects head/base/count drift and does not publish verdicts. The upstream repair is correctly identified as an unmerged candidate, not an established cause or deployment. This is a one-PR approval, not a blanket author trust grant, App permission grant, DCO exception or approval to merge #16378. Provider recovery and genuine current-head App1861 success remain outstanding. The documentation-map discoverability suggestion is advisory follow-through.

@clubanderson

Copy link
Copy Markdown
Member

/hive decision approve

Scoped owner approval is recorded on exact head 0ac19df for these diagnostics and read-only recovery tools. The fork/non-OWNERS origin may proceed through the normal reviewed path for this PR; do not grant blanket author trust or change any admission/protection/permission rules. UNKNOWN still fails admission. This does not resolve the external provider fault or authorize #16378 to merge. Keep #16404 open for genuine provider-side recovery. Preserve all remaining native review and merge requirements.

@kubestellar-prow kubestellar-prow Bot added the lgtm Indicates that a PR is ready to be merged. label Nov 3, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

LGTM label has been added.

DetailsGit tree hash: fd7cb65e1e4655ae432b05d4dc63657cff833b13

@clubanderson

Copy link
Copy Markdown
Member

/approve

Maintainer sign-off (delegate for @clubanderson) on exact head 0ac19dfa36e6fbe46e50120ddced291fac3a7bdb, resolving the Sentinel hold. Independent review of the full eight-file delta:

  • Admission is not weakened. ProviderUnavailable is thrown from evidenceForHead only after the existing App-1861 / exact-head trust check and the status === 'completed' check (native-queue-policy.cjs:93-98), and only when run.conclusion === 'failure' and the provider's own "could not be evaluated … verdict is unknown" summary matches (:40-47). It is a subclass of Error, not PendingEvidence, so the workflow loop exits immediately with core.setFailed (native-queue-policy.yml:49-56) — the job still fails, the queue still blocks. An unrecognised summary falls through to the unchanged DCO: failure path (test native-queue-policy.test.cjs:190-203). No permission, protection or trust change; the workflow still checks out the base branch under pull_request_target.
  • Reproducer is read-only (diagnose-dco-provider.cjs: GraphQL query only, REST GETs, exits non-zero on any incomplete/changed enumeration) and is wired only into node --test.
  • Adjudicator nits (runbook location, test wiring in the admission job, hoisting provider literals, stricter negative matchers) are style follow-ups, not blockers; file them separately if desired.

Scope of this approval is exactly what the owner recorded in #16442 (comment): UNKNOWN still fails admission; no blanket author trust; this does not unblock #16378 by itself (a v6-base PR checks out and runs v6's native-queue-policy.cjs, so the honest UNKNOWN report reaches v6 only when v6 receives this change). Removing sentinel-alert per the Sentinel instructions now that a maintainer has reviewed it.

@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: clubanderson

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubestellar-prow kubestellar-prow Bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 3, 2026
@hivecommons-hive

hivecommons-hive Bot commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Ready for a human merge: verdict requires_human @0ac19df, approved by @clubanderson, 68/68 checks green, DCO yes, soak 10m0s elapsed. Blocked only by role/contributor.

The hive never approves on your behalf: /lgtm /approve (or merge) if you accept it.

Updated 2026-11-03 20:39 UTC.

@clubanderson

Copy link
Copy Markdown
Member

/hold

Holding briefly while I verify a cross-branch effect: this workflow runs from the default branch (v5) for v6-base PRs too, but checks out the base branch (v6) for scripts — v6 does not have diagnose-dco-provider.test.cjs or the ProviderUnavailable export. Verifying whether the new test-step path and instanceof ProviderUnavailable would break v6-base admission before this lands.

@kubestellar-prow kubestellar-prow Bot added do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. and removed lgtm Indicates that a PR is ready to be merged. labels Nov 3, 2026
@clubanderson

Copy link
Copy Markdown
Member

Pushed one maintainer commit (52c81fc3cf20a0b6e8f58ac7b0a5df85f6d20af4, DCO signed, maintainerCanModify was on) to this branch rather than bouncing it back — the fix is two lines and purely defensive. Rationale, verified against live runs:

  • native-queue-policy.yml has never existed on v6 (git log origin/v6 -- .github/workflows/native-queue-policy.yml is empty), yet it runs on v6-base PRs (e.g. run 41684812577 on 🌱 Top up v6 with the latest v5 snapshot #16378, path: .github/workflows/native-queue-policy.yml, ref: refs/heads/v6). So GitHub takes the workflow definition from the default branch (v5) and the job then checks out the base branch's scripts.
  • With 0ac19dfa3's yml on v5 and v6's older native-queue-policy.cjs (no ProviderUnavailable export), error instanceof ProviderUnavailable is instanceof undefined → TypeError: Right-hand side of 'instanceof' is not an object thrown inside the catch, which would replace the bounded wait for PendingEvidence on every v6-base PR (first-run failures while DCO/providers are still pending). Guarded as ProviderUnavailable && error instanceof ProviderUnavailable.
  • The admission test step is reverted to native-queue-policy.test.cjs only (adjudicator finding 2); diagnose-dco-provider.test.cjs does not exist on v6 and is already run by pr-auto-update-test.yml on the relevant paths.

No change to the policy script, reproducer, tests or docs; the UNKNOWN classification and fail-closed behaviour are exactly as reviewed at 0ac19dfa3. @thatguywob — feel free to object; otherwise approval is re-recorded on the new head below.

@clubanderson clubanderson left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maintainer approval (delegate for @clubanderson) on exact head 52c81fc: same scope as the owner's approval of 0ac19df plus the two-line defensive yml guard for v6-base runs. UNKNOWN still fails admission; no permission/protection/trust change; does not by itself unblock #16378.

@kubestellar-prow kubestellar-prow Bot added the lgtm Indicates that a PR is ready to be merged. label Nov 3, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

LGTM label has been added.

DetailsGit tree hash: 833281986138ca132fe5ce9a02fc1c752039c736

@clubanderson
clubanderson force-pushed the fix/16404-dco-provider-unavailable branch from 52c81fc to 0ac19df Compare November 3, 2026 20:44
@kubestellar-prow kubestellar-prow Bot removed the lgtm Indicates that a PR is ready to be merged. label Nov 3, 2026
@clubanderson

Copy link
Copy Markdown
Member

Reverted my commit: the branch is back at exactly 0ac19dfa36e6fbe46e50120ddced291fac3a7bdb (the head the owner approved). Sign-off survives the squash correctly rejects a maintainer-signed commit on a contributor PR that squash-merges (run 41714844137), so a second author cannot land here.

Plan instead, so this can merge unchanged: land the policy-script/test files on v6 first in a separate v6 PR (so v6's native-queue-policy.cjs exports ProviderUnavailable and diagnose-dco-provider.test.cjs exists there before this yml becomes the default-branch definition). Hold stays until that v6 PR is in; then /hold cancel and tide takes this as approved.

hivecommons-hive Bot pushed a commit that referenced this pull request Nov 3, 2026
…script (#16467)

Backport of the policy-script half of #16442 (head 0ac19df) to v6:
native-queue-policy.cjs gains ProviderUnavailable and the narrow
dcoEnumerationUnavailable classifier (applied only after the App-1861 /
exact-head / completed checks; still fails admission), its regression
tests, the read-only diagnose-dco-provider reproducer and its tests, and
the operator runbook.

Why v6 first: .github/workflows/native-queue-policy.yml exists only on
v5 and is the default-branch definition GitHub uses for every
pull_request_target run, while the job checks out the PR base branch's
scripts. Once #16442's yml lands on v5 it imports ProviderUnavailable
from the checked-out script; on a v6 base that must exist, otherwise
'error instanceof undefined' throws inside the catch and breaks the
bounded wait for pending evidence on every v6-base PR. Landing this
first keeps v6-base admission intact and lets v6 report the ceiling
case as UNKNOWN honestly.

No workflow, permission, protection or trust change. Refs #16404.

Signed-off-by: clubanderson <andy@clubanderson.com>
@clubanderson

Copy link
Copy Markdown
Member

/hold cancel

v6 now carries the policy script with ProviderUnavailable and the diagnose tests (#16467, merged as 5c963b7081e4762d9321158c0b629431d4d9a7e1), so this yml is safe as the default-branch definition for v6-base runs. Head is unchanged at 0ac19dfa36e6fbe46e50120ddced291fac3a7bdb; owner approval review 5756842459 and prow approved stand. Releasing to tide.

@kubestellar-prow kubestellar-prow Bot removed the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Nov 3, 2026
@hivecommons-hive

Copy link
Copy Markdown
Contributor

@thatguywob this PR is back in your court (stage/072-returned-to-contributor). The hive will not merge it until the head changes.

Failing checks:

  • native-queue-policy

Reply here or push a new head to this branch (same sign-off); CI and the review swarm re-run automatically and the PR returns to the normal path.

Your relay could not take this as a fix-pr task (provider quota exhausted for codex — not a fault of this host; declining work for an unknown duration (until quota recovery is confirmed) (■ You’ve hit your usage limit. Upgrade to Pro (https://chatgpt.com/explore/pro), visit https://chatgpt.com/settings/usage to purchase more credits or try again at Nov 8th, 2026 7:08 PM.)); the hive will retry it from your relay fix queue.

@clubanderson

Copy link
Copy Markdown
Member

/lgtm

(prow dropped the label when the branch was reset to 0ac19dfa3; re-adding on the same reviewed head.)

@kubestellar-prow kubestellar-prow Bot added the lgtm Indicates that a PR is ready to be merged. label Nov 3, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

LGTM label has been added.

DetailsGit tree hash: fd7cb65e1e4655ae432b05d4dc63657cff833b13

@hivecommons-hive

Copy link
Copy Markdown
Contributor

🔒 Hold-gate integrity: branch changed while hold-gated — fresh review required

This PR's branch moved while it sat hold-gated: the head recorded when the hold was applied was 52c81fc3cf20, but the branch now sits at 0ac19dfa36e6. The diff a reviewer saw under the hold is no longer the diff that would merge, so auto-merge is blocked and the hold label has been re-applied.

The current commit list could not be compared against the snapshot; treat the entire branch as unreviewed.

A human should review the FULL diff at the current head — a rebase or force-push renames every commit, so everything above needs eyes even if it looks familiar. Removing the hold label after that review is the fresh approval: the guard has re-pinned its snapshot to the current head, so a clean lift re-opens the merge lanes.

@hivecommons-hive hivecommons-hive Bot added the hold label Nov 3, 2026
@clubanderson

Copy link
Copy Markdown
Member

Hold-gate integrity: acknowledged. The branch moved from 52c81fc3 back to 0ac19dfa36e6fbe46e50120ddced291fac3a7bdb because I reverted my own maintainer commit — the current head is byte-identical to the head the owner reviewed and approved (review 5756842459, 17:36Z) and that I re-reviewed in full above (#16442 (comment)). Lifting hold as the fresh approval on this head.

@clubanderson clubanderson removed the hold label Nov 3, 2026
@kubestellar-prow
kubestellar-prow Bot merged commit ed048ec into hivecommons:v5 Nov 3, 2026
159 of 177 checks passed
clubanderson added a commit that referenced this pull request Nov 3, 2026
…the top-up

Plain merge of origin/v6 (5c963b7) into the v5->v6 top-up; the only
conflicts were the additive export/import lists in
.github/scripts/native-queue-policy.cjs and its test, resolved to the
union, which is byte-identical to v5 after #16442 (ed048ec).

Signed-off-by: clubanderson <andy@clubanderson.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has signed the DCO. lgtm Indicates that a PR is ready to be merged. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants