diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index c8347f7189..6dbb08b3d3 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -2979,6 +2979,62 @@ verifying "library X can't do Y" requires reading X's own policy/configuration s README/marketing feature list, before recommending against adoption. Saved to `feedback_verify_org_wide_before_declaring_unstarted.md`. +## Org-wide audit: `code-scanning/default-setup` vs. a repository's own advanced-configuration CodeQL workflow — 2026-09-04 + +**Status:** Superseded by a staged central-CodeQL rollout contract. `contextual-orchestrator` was the only +confirmed live instance among the 11 Code Search candidates and repositories inspected directly; it was +already fixed in the same investigation that discovered it +(`contextual-orchestrator` PR #1028's failing "CodeQL analysis" check — `code-scanning/default-setup` was +`state: "configured"` while `.github/workflows/security.yml`'s `codeql_analysis` job also ran a real, +working `github/codeql-action/init` + `analyze` sequence; GitHub rejects that combination outright, failing +the SARIF upload with "CodeQL analyses from advanced configurations cannot be processed when the default +setup is enabled." Fixed with `gh api --method PATCH repos/ContextualWisdomLab/contextual-orchestrator/code-scanning/default-setup -f state=not-configured`, +since `security.yml` was the pre-existing, real coverage mechanism; a related suppression bug found in the +same pass — the whole "Security" workflow, id `300545778`, had been `disabled_manually`, hiding the failure +rather than fixing it — was reversed with `gh api --method PUT .../actions/workflows/300545778/enable`.) + +**Why an org-wide audit was warranted.** The item-41 entry above records that its 2026-09-03 default-setup +rollout deliberately checked real coverage first via the `code-scanning/analyses` API before assigning +default-setup only to the 23 repositories with zero coverage from any source. `contextual-orchestrator` +having both mechanisms simultaneously raised the question of whether it was misclassified during that sweep, +or whether default-setup landed on it (and possibly others) through an unrelated path. + +**Method.** Org-wide `gh api -X GET search/code -f q="codeql-action/analyze org:ContextualWisdomLab path:.github/workflows"` (content search, not a filename grep — the same lesson item-41 already applied, since `contextual-orchestrator`'s own coverage lives in an unexpectedly-named `security.yml` rather than a `codeql.yml`) returned 13 hits across 11 repositories with a local workflow file containing `github/codeql-action/init`/`analyze`: `newsdom-api`, `keyverse`, `ContextualWisdomLab.github.io`, `fast-mlsirm`, `scopeweave`, `bandscope`, `contextual-orchestrator`, `mightyETL`, `litellm-patched-proxy` (2 files), `pg-erd-cloud`, and `.github` itself (2 files — `codeql-scan-dispatch.yml`, the already-known central dispatch handler, and `scheduled-security-scan.yml`; expected, not investigated further as a "local repo" case). `gh api repos/ContextualWisdomLab//code-scanning/default-setup --jq '.state'` was then checked for each of the other 10. + +**Result: `default-setup=configured` alongside a local advanced-config workflow, beyond `contextual-orchestrator`, in exactly 3 repositories — none of which are in item-41's 23-repository rollout list, and none of which are a live conflict.** +- **`ContextualWisdomLab.github.io`** — false positive. Its `.github/workflows/codeql.yml` is named "CodeQL Default Setup Marker," triggers only on `workflow_dispatch` (never on push/PR), and its `analyze` step carries `if: ${{ false }}` (never executes) with an explicit preceding comment: *"Skipping github/codeql-action/analyze because central/default setup owns SARIF upload."* Deliberately engineered to expose `codeql-action` usage to Scorecard's static analysis without ever touching SARIF. No fix needed. +- **`fast-mlsirm`** — false positive. `.github/workflows/codeql.yml` runs two real jobs (`analyze-actions` on every PR, `analyze-python` gated to `workflow_dispatch` only), and **both** `analyze` steps carry `with: upload: never`, with comments stating *"Default setup remains the repository's code-scanning upload owner"* and *"Default setup already owns ordinary Python code-scanning uploads."* Confirmed via a live job log (run `33754939454`, job `100646992008`, `2026-09-04T00:45Z`): `upload: never` present in the action's resolved input dump, `Exported results to SARIF` followed by no upload call, job concluded `success`. Deliberately engineered the opposite way from `contextual-orchestrator`'s fix (default-setup keeps ownership, the local workflow stays silent) rather than the way `contextual-orchestrator` was fixed (local workflow keeps ownership, default-setup disabled) — both are valid resolutions of the same conflict; this repository already had one in place. No fix needed. +- **`scopeweave`** — no live conflict, but two dangling artifacts worth a light cleanup. The workflow with real `init`/`analyze` steps (`.github/workflows/codeql.yml`) is `disabled_manually`, so it never runs and cannot collide with default-setup today. A second, unrelated workflow entry — "CodeQL Required," id `335384625`, `.github/workflows/codeql-required.yml` — is registered `state: "active"` in the Actions API, but the file itself no longer exists on the `develop` default branch (`404` on direct content fetch); GitHub retains the workflow-run registration for a file that has since been deleted, so this entry can never actually trigger. Net effect: default-setup is the sole current CodeQL coverage source for this repository, matching item-41's own "zero coverage from any source" criterion at whatever point `codeql.yml` was disabled — not a misclassification, just a repository whose local workflow went inactive after (or independent of) the rollout. Not fixed in this pass: re-enabling the disabled `codeql.yml` would immediately recreate `contextual-orchestrator`'s exact conflict, so any future re-enable of that workflow must add `upload: never` (matching `fast-mlsirm`'s pattern) or disable default-setup first, whichever this repository's owner intends as the coverage source of record. + +**The remaining 7 repositories** (`newsdom-api`, `keyverse`, `bandscope`, `mightyETL`, `litellm-patched-proxy`, `pg-erd-cloud`, `.github`) all returned `default-setup=not-configured` — no conflict is possible regardless of their local workflow's upload configuration. + +**Conclusion.** `contextual-orchestrator`'s conflict was an isolated incident, not a symptom of a broader misclassification in item-41's rollout (none of the 3 repositories found here with `default-setup=configured` alongside a local workflow were among that rollout's 23 targets) and not evidence of an org policy silently re-enabling default-setup on repositories that already had real coverage. Two of the three already carry a deliberate, working design for this exact conflict (`if: false` / `upload: never`) that predates or is independent of this audit — worth keeping as the reference pattern if this conflict resurfaces elsewhere, in preference to `contextual-orchestrator`'s "disable default-setup" fix when the local workflow does not yet have established real-coverage precedence. + +**Caveat.** This audit trusted GitHub's code-search index for the initial 11-repository candidate list rather than fetching and grepping all 74 repositories' workflow directories individually; code search can lag very recent pushes by a short window. The 10 non-`contextual-orchestrator` candidates it did surface were each verified directly against the live API/content, not from search snippets alone. + +**2026-09-05 staged rollout correction.** The organization now requires the central +`.github/workflows/codeql-pr.yml` through ruleset `18156473`; keeping GitHub's generated +`dynamic/github-code-scanning/codeql` default setup on the same PR spends another CodeQL job set. Removal +must proceed one repository at a time. `scripts/ci/audit_codeql_default_setup_rollout.py` is the read-only +gate: it requires the inherited ruleset and central workflow, binds evidence to the exact PR head, blocks an +active advanced uploader/default-setup collision, and reports either `READY_DISABLE`, `VERIFIED`, `WAIT`, +`ROLLBACK`, or `BLOCK`. A repository advances only after exact-head central CodeQL succeeds. If central +CodeQL fails after default setup is disabled, re-enable default setup before continuing, but only when no +active advanced uploader would make that rollback invalid. `.github`, `noema`, and +`IRT-bibliography-set` are explicit ruleset exceptions and must remain `EXEMPT`, not silently counted as +rollout failures. Run the live collector as +`python3 scripts/ci/audit_codeql_default_setup_rollout.py --repository ContextualWisdomLab/ --pr `; +it uses only authenticated REST `GET` requests and re-reads the PR head after collection to reject a moving +snapshot. + +The xtrmLLMBatchPython pilot is intentionally not yet proof of completion: default setup currently reports +`not-configured`, ruleset `18156473` requires central CodeQL, and PR #292 head +`5f4de312e72da5e1303c701d8e6f65cec7207409` has central run `33904225451`; that run is still `queued`. +The generated default-setup run `33904220801` for the same head was cancelled after the setting change. +No second repository may be changed until the central run reaches an explicit successful terminal state and +the detector reports `VERIFIED` for that exact head. GitHub documents the hard boundary: default setup blocks +CodeQL-generated SARIF uploads from advanced configuration, so rollback must never blindly enable it beside +an active uploader. ## 2026-09-04 org-wide open-PR sweep: severe central Actions capacity congestion confirmed, `noema_review_gate.py`/`strix.yml` confirmed as a multi-PR hot-file collision zone **Status:** Investigated via direct read-only Actions API queries and scratch-clone merge attempts against diff --git a/scripts/ci/audit_codeql_default_setup_rollout.py b/scripts/ci/audit_codeql_default_setup_rollout.py new file mode 100755 index 0000000000..17eaa0146c --- /dev/null +++ b/scripts/ci/audit_codeql_default_setup_rollout.py @@ -0,0 +1,303 @@ +#!/usr/bin/env python3 +"""Classify CodeQL default-setup removal snapshots without mutating GitHub.""" + +from __future__ import annotations + +import argparse +import base64 +import json +import re +import sys +from pathlib import Path +from typing import Any, TextIO +from urllib.parse import quote + +try: + from scripts.ci.organization_commercial_readiness_loop import ( + GitHubClient, + GitHubError, + ) +except ModuleNotFoundError: # Direct ``python scripts/ci/...`` execution. + from organization_commercial_readiness_loop import GitHubClient, GitHubError + +EXEMPT_REPOSITORIES = frozenset({".github", "noema", "IRT-bibliography-set"}) +SUCCESS = frozenset({"success", "neutral", "skipped"}) +PENDING = frozenset({"queued", "in_progress", "pending", "requested", "waiting"}) +RULESET_ID = 18156473 +CENTRAL_CODEQL_PATH = ".github/workflows/codeql-pr.yml" +CENTRAL_REPOSITORY_ID = 1274066402 +MAX_PAGES = 20 +MAX_WORKFLOW_BYTES = 1_048_576 + + +class EvidenceError(RuntimeError): + """Report missing or ambiguous live rollout evidence.""" + + +def _pages(client: Any, path: str, key: str | None = None) -> list[dict[str, Any]]: + """Read every bounded REST page and reject malformed evidence.""" + values: list[dict[str, Any]] = [] + separator = "" if path.endswith("?") else "&" if "?" in path else "?" + for page in range(1, MAX_PAGES + 1): + payload = client.request(f"{path}{separator}per_page=100&page={page}") + batch = payload.get(key) if key and isinstance(payload, dict) else payload + if not isinstance(batch, list) or not all(isinstance(item, dict) for item in batch): + raise EvidenceError(f"GitHub returned malformed pagination data for {path}") + values.extend(batch) + if len(batch) < 100: + return values + raise EvidenceError(f"GitHub pagination exceeded {MAX_PAGES} pages for {path}") + + +def _step_has_disabled_upload(lines: list[str], start: int) -> bool: + """Recognize only explicit, local neutralization of one CodeQL action step.""" + uses_indent = len(lines[start]) - len(lines[start].lstrip()) + block_start = start + for index in range(start - 1, -1, -1): + stripped = lines[index].lstrip() + indent = len(lines[index]) - len(stripped) + if stripped.startswith("-") and indent <= uses_indent: + block_start = index + break + step_indent = len(lines[block_start]) - len(lines[block_start].lstrip()) + block = [lines[block_start]] + for line in lines[block_start + 1 :]: + stripped = line.lstrip() + line_indent = len(line) - len(stripped) + if stripped.startswith("-") and line_indent <= step_indent: + break + block.append(line) + text = "\n".join(block) + return bool( + re.search(r"(?m)^\s*if:\s*(?:false|\$\{\{\s*false\s*\}\})\s*$", text) + or re.search(r"(?m)^\s*upload:\s*['\"]?never['\"]?\s*$", text) + ) + + +def _has_active_advanced_upload(source: str) -> bool: + """Conservatively detect an executable local CodeQL/SARIF upload step.""" + lines = source.splitlines() + for index, line in enumerate(lines): + if re.search( + r"uses:\s*github/codeql-action/(?:analyze|upload-sarif)@", line + ) and not _step_has_disabled_upload(lines, index): + return True + return False + + +def _active_advanced_uploader(client: Any, repository: str, head_sha: str) -> bool: + """Inspect active repository-owned workflow sources at the exact PR head.""" + workflows = _pages(client, f"/repos/{repository}/actions/workflows?", "workflows") + inspected_paths: set[str] = set() + for workflow in workflows: + path = str(workflow.get("path") or "") + if workflow.get("state") != "active" or not path.startswith(".github/workflows/"): + continue + if path in inspected_paths: + raise EvidenceError(f"active workflow identity is ambiguous: {path}") + inspected_paths.add(path) + encoded = quote(path, safe="/") + try: + source = client.request( + f"/repos/{repository}/contents/{encoded}?ref={head_sha}" + ) + except GitHubError as exc: + if "HTTP 404" in str(exc): + continue + raise EvidenceError(f"active workflow source lookup failed: {path}") from exc + if not isinstance(source, dict) or source.get("encoding") != "base64": + raise EvidenceError(f"active workflow source is unavailable: {path}") + size = source.get("size") + if not isinstance(size, int) or size < 0 or size > MAX_WORKFLOW_BYTES: + raise EvidenceError(f"active workflow source has invalid size: {path}") + try: + encoded_content = "".join(str(source.get("content") or "").split()) + decoded = base64.b64decode(encoded_content, validate=True).decode() + except (ValueError, UnicodeDecodeError) as exc: + raise EvidenceError(f"active workflow source is invalid: {path}") from exc + if len(decoded.encode()) != size: + raise EvidenceError(f"active workflow source size mismatch: {path}") + if _has_active_advanced_upload(decoded): + return True + return False + + +def collect_live_snapshot(client: Any, repository: str, pr_number: int) -> dict[str, Any]: + """Collect one exact-head rollout snapshot using read-only GitHub requests.""" + if not re.fullmatch(r"ContextualWisdomLab/[A-Za-z0-9_.-]+", repository): + raise EvidenceError("repository must belong to ContextualWisdomLab") + if pr_number < 1: + raise EvidenceError("pull request number must be positive") + + pull = client.request(f"/repos/{repository}/pulls/{pr_number}") + head_sha = str(((pull or {}).get("head") or {}).get("sha") or "") + if (pull or {}).get("state") != "open" or not re.fullmatch(r"[0-9a-f]{40}", head_sha): + raise EvidenceError("pull request is not open or has no valid exact head") + + inherited = _pages(client, f"/repos/{repository}/rulesets?includes_parents=true") + matches = [item for item in inherited if item.get("id") == RULESET_ID] + if len(matches) > 1: + raise EvidenceError("central ruleset evidence is ambiguous") + ruleset_applies = len(matches) == 1 + central_required = False + if ruleset_applies: + detail = client.request( + f"/repos/{repository}/rulesets/{RULESET_ID}?includes_parents=true" + ) + owners = [ + workflow + for rule in (detail or {}).get("rules", []) + if isinstance(rule, dict) and rule.get("type") == "workflows" + for workflow in (rule.get("parameters") or {}).get("workflows", []) + if isinstance(workflow, dict) + and workflow.get("path") == CENTRAL_CODEQL_PATH + and workflow.get("ref") == "refs/heads/main" + and workflow.get("repository_id") == CENTRAL_REPOSITORY_ID + ] + if len(owners) > 1: + raise EvidenceError("central CodeQL ruleset owner is ambiguous") + central_required = len(owners) == 1 + + name = repository.partition("/")[2] + if name in EXEMPT_REPOSITORIES: + latest_pull = client.request(f"/repos/{repository}/pulls/{pr_number}") + if str(((latest_pull or {}).get("head") or {}).get("sha") or "") != head_sha: + raise EvidenceError("pull request head changed during live evidence collection") + return {"name": name, "ruleset_applies": ruleset_applies} + + default_setup = client.request(f"/repos/{repository}/code-scanning/default-setup") + default_state = str((default_setup or {}).get("state") or "") + if default_state not in {"configured", "not-configured"}: + raise EvidenceError("default-setup state is unavailable") + + runs = _pages( + client, + f"/repos/{repository}/actions/runs?head_sha={head_sha}", + "workflow_runs", + ) + central_runs = [ + run + for run in runs + if run.get("path") == CENTRAL_CODEQL_PATH + and run.get("event") == "pull_request" + and run.get("head_sha") == head_sha + ] + if len(central_runs) != 1: + raise EvidenceError( + "exact-head central CodeQL run is missing or ambiguous" + ) + run = central_runs[0] + status = str(run.get("conclusion") or run.get("status") or "") + if not status: + raise EvidenceError("exact-head central CodeQL run has no status") + + result = { + "name": name, + "ruleset_applies": ruleset_applies, + "central_codeql_required": central_required, + "expected_head": head_sha, + "central_codeql_head": str(run.get("head_sha") or ""), + "central_codeql_status": status, + "default_setup_state": default_state, + "active_advanced_upload": _active_advanced_uploader( + client, repository, head_sha + ), + } + latest_pull = client.request(f"/repos/{repository}/pulls/{pr_number}") + if str(((latest_pull or {}).get("head") or {}).get("sha") or "") != head_sha: + raise EvidenceError("pull request head changed during live evidence collection") + return result + + +def classify(repository: dict[str, Any]) -> tuple[str, str]: + """Return a fail-closed rollout state and its operator-facing reason.""" + name = str(repository.get("name") or "") + ruleset_applies = repository.get("ruleset_applies") is True + if name in EXEMPT_REPOSITORIES: + if ruleset_applies: + return "BLOCK", "documented exception is unexpectedly covered by the central ruleset" + return "EXEMPT", "documented ruleset exception" + + if not ruleset_applies or repository.get("central_codeql_required") is not True: + return "BLOCK", "central CodeQL is not enforced by ruleset 18156473" + + expected_head = repository.get("expected_head") + observed_head = repository.get("central_codeql_head") + if not isinstance(expected_head, str) or len(expected_head) != 40 or observed_head != expected_head: + return "BLOCK", "central CodeQL evidence is absent or belongs to another head" + + central_status = repository.get("central_codeql_status") + default_state = repository.get("default_setup_state") + active_advanced_upload = repository.get("active_advanced_upload") is True + + if default_state == "configured": + if active_advanced_upload: + return "BLOCK", "default setup conflicts with an active advanced CodeQL uploader" + if central_status in SUCCESS: + return "READY_DISABLE", "exact-head central CodeQL passed; disable one repository only" + return "WAIT", "keep default setup until exact-head central CodeQL passes" + + if default_state != "not-configured": + return "BLOCK", "default-setup state is unavailable or unsupported" + if central_status in SUCCESS: + return "VERIFIED", "default setup is off and exact-head central CodeQL passed" + if central_status in PENDING: + return "WAIT", "default setup is off; wait for the exact-head central CodeQL verdict" + if active_advanced_upload: + return "BLOCK", "central CodeQL failed and default setup cannot coexist with the active uploader" + return "ROLLBACK", "central CodeQL failed; re-enable default setup before continuing" + + +def audit(repositories: list[dict[str, Any]]) -> list[tuple[str, str, str]]: + """Classify every repository snapshot in input order.""" + return [ + (str(repository.get("name") or ""), *classify(repository)) + for repository in repositories + ] + + +def load_payload(path: Path | None, stdin: TextIO) -> list[dict[str, Any]]: + """Load a repository snapshot array from a file or standard input.""" + if path: + with path.open(encoding="utf-8") as handle: + payload = json.load(handle) + else: + payload = json.load(stdin) + if not isinstance(payload, list) or not all(isinstance(item, dict) for item in payload): + raise ValueError("repository snapshot root must be an array of objects") + return payload + + +def parse_args(argv: list[str] | None = None) -> argparse.Namespace: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("snapshots_json", nargs="?", type=Path) + parser.add_argument("--repository") + parser.add_argument("--pr", type=int) + return parser.parse_args(argv) + + +def main(argv: list[str] | None = None) -> int: + args = parse_args(argv) + try: + live_mode = args.repository is not None or args.pr is not None + if live_mode: + if args.snapshots_json or not args.repository or args.pr is None: + raise ValueError("live mode requires --repository and --pr only") + repositories = [ + collect_live_snapshot( + GitHubClient.from_environment(), args.repository, args.pr + ) + ] + else: + repositories = load_payload(args.snapshots_json, sys.stdin) + results = audit(repositories) + except (OSError, ValueError, json.JSONDecodeError, EvidenceError, GitHubError) as exc: + print(f"ERROR: unable to load CodeQL rollout snapshots: {exc}", file=sys.stderr) + return 2 + for name, state, reason in results: + print(f"CODEQL_ROLLOUT repository={name} state={state} reason={reason}") + return 0 if all(state in {"EXEMPT", "VERIFIED"} for _, state, _ in results) else 1 + + +if __name__ == "__main__": # pragma: no cover + raise SystemExit(main()) diff --git a/tests/test_codeql_default_setup_rollout.py b/tests/test_codeql_default_setup_rollout.py new file mode 100644 index 0000000000..665eed5aa3 --- /dev/null +++ b/tests/test_codeql_default_setup_rollout.py @@ -0,0 +1,266 @@ +import base64 +import json +from io import StringIO + +from scripts.ci import audit_codeql_default_setup_rollout as rollout + +HEAD = "a" * 40 + + +class FakeClient: + def __init__(self, responses): + self.responses = responses + self.paths = [] + + def request(self, path): + self.paths.append(path) + response = self.responses.get(path) + if isinstance(response, Exception): + raise response + if response is None: + raise AssertionError(f"unexpected request: {path}") + return response + + +def encoded_workflow(source): + raw = source.encode() + return { + "encoding": "base64", + "size": len(raw), + "content": base64.b64encode(raw).decode(), + } + + +def live_responses(*, run_status="queued", uploader_source="name: CI\n"): + repository = "ContextualWisdomLab/xtrmLLMBatchPython" + return { + f"/repos/{repository}/pulls/292": {"state": "open", "head": {"sha": HEAD}}, + f"/repos/{repository}/rulesets?includes_parents=true&per_page=100&page=1": [ + {"id": rollout.RULESET_ID} + ], + f"/repos/{repository}/rulesets/{rollout.RULESET_ID}?includes_parents=true": { + "rules": [ + { + "type": "workflows", + "parameters": { + "workflows": [ + { + "path": rollout.CENTRAL_CODEQL_PATH, + "ref": "refs/heads/main", + "repository_id": rollout.CENTRAL_REPOSITORY_ID, + } + ] + }, + } + ] + }, + f"/repos/{repository}/code-scanning/default-setup": { + "state": "not-configured" + }, + f"/repos/{repository}/actions/runs?head_sha={HEAD}&per_page=100&page=1": { + "workflow_runs": [ + { + "path": rollout.CENTRAL_CODEQL_PATH, + "event": "pull_request", + "head_sha": HEAD, + "status": run_status, + "conclusion": None, + } + ] + }, + f"/repos/{repository}/actions/workflows?per_page=100&page=1": { + "workflows": [ + { + "path": ".github/workflows/ci.yml", + "state": "active", + } + ] + }, + f"/repos/{repository}/contents/.github/workflows/ci.yml?ref={HEAD}": encoded_workflow( + uploader_source + ), + } + + +def snapshot(**changes): + value = { + "name": "xtrmLLMBatchPython", + "ruleset_applies": True, + "central_codeql_required": True, + "expected_head": HEAD, + "central_codeql_head": HEAD, + "central_codeql_status": "success", + "default_setup_state": "not-configured", + "active_advanced_upload": False, + } + value.update(changes) + return value + + +def test_pilot_is_verified_only_after_exact_head_central_success(): + assert rollout.classify(snapshot()) == ( + "VERIFIED", + "default setup is off and exact-head central CodeQL passed", + ) + assert rollout.classify(snapshot(central_codeql_status="queued"))[0] == "WAIT" + assert rollout.classify(snapshot(central_codeql_head="b" * 40))[0] == "BLOCK" + + +def test_default_setup_stays_on_until_central_success(): + assert ( + rollout.classify( + snapshot(default_setup_state="configured", central_codeql_status="queued") + )[0] + == "WAIT" + ) + assert rollout.classify(snapshot(default_setup_state="configured"))[0] == "READY_DISABLE" + + +def test_default_setup_and_advanced_uploader_conflict_fails_closed(): + result = rollout.classify( + snapshot(default_setup_state="configured", active_advanced_upload=True) + ) + assert result[0] == "BLOCK" + assert "conflicts" in result[1] + + +def test_rollback_is_blocked_when_it_would_conflict_with_advanced_upload(): + result = rollout.classify( + snapshot(central_codeql_status="failure", active_advanced_upload=True) + ) + assert result[0] == "BLOCK" + assert "cannot coexist" in result[1] + + +def test_ruleset_and_documented_exceptions_are_enforced(): + assert rollout.classify(snapshot(ruleset_applies=False))[0] == "BLOCK" + for name in rollout.EXEMPT_REPOSITORIES: + assert rollout.classify(snapshot(name=name, ruleset_applies=False))[0] == "EXEMPT" + assert rollout.classify(snapshot(name=name, ruleset_applies=True))[0] == "BLOCK" + + +def test_cli_returns_nonzero_for_wait_and_zero_for_verified(capsys, monkeypatch): + monkeypatch.setattr(rollout.sys, "stdin", StringIO(json.dumps([snapshot()]))) + assert rollout.main([]) == 0 + monkeypatch.setattr( + rollout.sys, + "stdin", + StringIO(json.dumps([snapshot(central_codeql_status="queued")])), + ) + assert rollout.main([]) == 1 + assert "state=WAIT" in capsys.readouterr().out + + +def test_live_snapshot_collects_exact_head_ruleset_run_and_uploader(): + source = ( + "steps:\n - uses: github/codeql-action/analyze@" + "b" * 40 + "\n" + ) + snapshot = rollout.collect_live_snapshot( + FakeClient(live_responses(uploader_source=source)), + "ContextualWisdomLab/xtrmLLMBatchPython", + 292, + ) + assert snapshot == { + "name": "xtrmLLMBatchPython", + "ruleset_applies": True, + "central_codeql_required": True, + "expected_head": HEAD, + "central_codeql_head": HEAD, + "central_codeql_status": "queued", + "default_setup_state": "not-configured", + "active_advanced_upload": True, + } + + +def test_live_snapshot_paginates_workflows_and_accepts_upload_never(): + responses = live_responses( + uploader_source=( + "steps:\n - uses: github/codeql-action/analyze@" + "b" * 40 + + "\n with:\n upload: never\n" + ) + ) + first_path = ( + "/repos/ContextualWisdomLab/xtrmLLMBatchPython/actions/workflows?" + "per_page=100&page=1" + ) + workflows = [ + {"path": f"dynamic/filler/{index}", "state": "active"} + for index in range(99) + ] + responses[first_path]["workflows"] + responses[first_path] = {"workflows": workflows} + responses[first_path[:-1] + "2"] = {"workflows": []} + client = FakeClient(responses) + result = rollout.collect_live_snapshot( + client, "ContextualWisdomLab/xtrmLLMBatchPython", 292 + ) + assert result["active_advanced_upload"] is False + assert first_path[:-1] + "2" in client.paths + + +def test_live_snapshot_rejects_ambiguous_exact_head_central_runs(): + responses = live_responses() + path = f"/repos/ContextualWisdomLab/xtrmLLMBatchPython/actions/runs?head_sha={HEAD}&per_page=100&page=1" + responses[path]["workflow_runs"] *= 2 + try: + rollout.collect_live_snapshot( + FakeClient(responses), "ContextualWisdomLab/xtrmLLMBatchPython", 292 + ) + except rollout.EvidenceError as exc: + assert "missing or ambiguous" in str(exc) + else: + raise AssertionError("ambiguous runs must fail closed") + + +def test_live_snapshot_rejects_missing_ruleset_and_workflow_source_evidence(): + responses = live_responses() + ruleset_path = "/repos/ContextualWisdomLab/xtrmLLMBatchPython/rulesets?includes_parents=true&per_page=100&page=1" + responses[ruleset_path] = [] + result = rollout.collect_live_snapshot( + FakeClient(responses), "ContextualWisdomLab/xtrmLLMBatchPython", 292 + ) + assert rollout.classify(result)[0] == "BLOCK" + + responses = live_responses() + source_path = f"/repos/ContextualWisdomLab/xtrmLLMBatchPython/contents/.github/workflows/ci.yml?ref={HEAD}" + responses[source_path] = {"encoding": "none", "size": 1, "content": "x"} + try: + rollout.collect_live_snapshot( + FakeClient(responses), "ContextualWisdomLab/xtrmLLMBatchPython", 292 + ) + except rollout.EvidenceError as exc: + assert "unavailable" in str(exc) + else: + raise AssertionError("missing source evidence must fail closed") + + +def test_live_snapshot_ignores_registered_workflow_deleted_at_exact_head(): + responses = live_responses() + source_path = f"/repos/ContextualWisdomLab/xtrmLLMBatchPython/contents/.github/workflows/ci.yml?ref={HEAD}" + responses[source_path] = rollout.GitHubError("gh: Not Found (HTTP 404)") + result = rollout.collect_live_snapshot( + FakeClient(responses), "ContextualWisdomLab/xtrmLLMBatchPython", 292 + ) + assert result["active_advanced_upload"] is False + + +def test_live_snapshot_rejects_head_movement_during_collection(): + class MovingHeadClient(FakeClient): + reads = 0 + + def request(self, path): + if path.endswith("/pulls/292"): + self.reads += 1 + if self.reads == 2: + return {"state": "open", "head": {"sha": "b" * 40}} + return super().request(path) + + try: + rollout.collect_live_snapshot( + MovingHeadClient(live_responses()), + "ContextualWisdomLab/xtrmLLMBatchPython", + 292, + ) + except rollout.EvidenceError as exc: + assert "head changed" in str(exc) + else: + raise AssertionError("moving exact-head evidence must fail closed")