-
Notifications
You must be signed in to change notification settings - Fork 0
ci(ops): add hourly read-only gap freshness audit #115
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
seonghobae
wants to merge
20
commits into
develop
Choose a base branch
from
feat/hourly-pr-gap-loop
base: develop
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Draft
Changes from all commits
Commits
Show all changes
20 commits
Select commit
Hold shift + click to select a range
7464562
ci(ops): add hourly trusted PR and gap loop
seonghobae b68fc24
test(ops): require pinned reusable workflow and full queue pagination
seonghobae d689b99
fix(ops): call central scheduler as pinned reusable job
seonghobae f0e80a9
fix(ops): paginate complete live queue truth
seonghobae 97095e1
test(ops): keep pre-baseline hourly audit nonfatal
seonghobae e22aed5
fix(ops): keep pre-baseline audit nonfatal
seonghobae 845f150
test(ops): forbid forwarding independent-review secrets
seonghobae 494be78
fix(ops): keep reviewer credentials outside scheduler caller
seonghobae 829bcca
docs(ops): record primary reusable-workflow security references
seonghobae c0d1465
test(ops): forbid a second scheduled PR writer
seonghobae 928948b
fix(ops): preserve central scheduler as sole PR writer
seonghobae 4ffdcdb
docs(ops): align references with single-writer audit boundary
seonghobae 4bd7288
test(ops): fail first on gap freshness time and fetch semantics
seonghobae 77e7265
fix(ops): make gap freshness calendar-aware and fail closed
seonghobae 6f9d540
docs(ops): document fail-closed freshness semantics
seonghobae 3c9a31f
chore(ops): remove stale timezone import
seonghobae b59820f
test(ops): reject future-dated gap inventory before live reads
seonghobae 72819cb
fix(ops): fail closed on future-dated gap inventory
seonghobae ece19f3
docs(ops): record future-inventory fail-closed contract
seonghobae de2b0aa
fix(ops): fail closed on stale gap queue evidence
seonghobae File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,45 @@ | ||
| name: Hourly Gap Baseline Freshness Audit | ||
|
|
||
| # Read-only Orgmetra heartbeat for buyer-facing gap truth. | ||
| # | ||
| # The central ContextualWisdomLab/.github scheduler already owns review dispatch, | ||
| # branch updates and protected PR integration on its established cadence. This | ||
| # workflow deliberately does not invoke or duplicate that writer. It only audits | ||
| # docs/product-technical-gap-baseline.md against fresh GitHub state so a quiet | ||
| # repository cannot leave buyer-facing baseline copy silently stale. | ||
|
|
||
| on: | ||
| schedule: | ||
| - cron: "37 * * * *" | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: read | ||
| issues: read | ||
|
|
||
| concurrency: | ||
| group: hourly-gap-baseline-freshness | ||
| cancel-in-progress: false | ||
|
|
||
| jobs: | ||
| gap-baseline-freshness: | ||
| name: Gap baseline freshness audit | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 10 | ||
| steps: | ||
| - name: Checkout default branch truth | ||
| uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1 | ||
| with: | ||
| ref: ${{ github.event.repository.default_branch }} | ||
| persist-credentials: false | ||
| - name: Set up Python | ||
| uses: actions/setup-python@5fda3b95a4ea91299a34e894583c3862153e4b97 # v7.0.0 | ||
| with: | ||
| python-version: "3.13" | ||
| - name: Audit gap baseline against live repository state | ||
| env: | ||
| GH_TOKEN: ${{ github.token }} | ||
| run: | | ||
| python3 scripts/ops/gap_baseline_freshness.py \ | ||
| --baseline docs/product-technical-gap-baseline.md >> "$GITHUB_STEP_SUMMARY" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| # Hourly gap-baseline audit primary references | ||
|
|
||
| Verified against GitHub's current official documentation on **2026-08-25**. These sources define the execution and least-privilege assumptions used by the active Orgmetra read-only freshness audit. They do not replace the central `.github` repository's published scheduler contract or Orgmetra's effective ruleset. | ||
|
|
||
| ## APA 7 references | ||
|
|
||
| GitHub. (n.d.). *Workflow syntax for GitHub Actions*. GitHub Docs. Retrieved August 25, 2026, from https://docs.github.com/en/actions/reference/workflows-and-actions/workflow-syntax | ||
|
|
||
| GitHub. (n.d.). *Reusing workflow configurations*. GitHub Docs. Retrieved August 25, 2026, from https://docs.github.com/en/actions/reference/workflows-and-actions/reusing-workflow-configurations | ||
|
|
||
| ## Decision relevance | ||
|
|
||
| - GitHub permits workflow- and job-level `permissions` to constrain `GITHUB_TOKEN`. The Orgmetra heartbeat needs only repository contents, pull-request metadata and issue metadata reads, so its token remains read-only. | ||
| - GitHub defines reusable workflow calls as writer-capable job boundaries when the caller grants write permissions. Orgmetra deliberately does **not** call the central review/merge scheduler from this hourly workflow because the central `.github` automation already owns that mutation lane on its established cadence. | ||
| - The scheduled audit runs from Orgmetra's default branch after integration and only reports whether the buyer-facing gap baseline is current. It does not dispatch reviews, update branches, approve, enable auto-merge, merge, forward secrets, or obtain an OIDC mutation token. | ||
| - The baseline's `Inventory date` is explicitly an **Asia/Seoul calendar date**, not a midnight-UTC timestamp. A `develop` commit later on the same Korea calendar day therefore does not by itself make a date-only snapshot stale; a commit on a later Korea calendar day does. | ||
| - The audit compares the recorded open pull-request and non-PR issue counts with the complete live queues. A queue change is reported as a refresh candidate even when `develop` has not advanced, because active-PR and issue truth can change without a protected-branch commit. | ||
| - A baseline inventory date later than the current Asia/Seoul calendar date is internally impossible evidence and fails closed before any live GitHub read. A future-dated snapshot must never be reported as current merely because no `develop` commit is later than that future date. | ||
| - Live GitHub state is authoritative evidence for this audit. If the PR/issue/commit reads cannot be established or their payload shape is invalid, the audit fails closed with a non-zero result rather than reporting a successful/current audit from missing evidence. | ||
| - The central scheduler remains the single writer for review dispatch and protected PR integration. This active PR only adds a read-only truth-audit surface. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,256 @@ | ||
| #!/usr/bin/env python3 | ||
| """Audit the product-technical gap baseline against live repository truth. | ||
|
|
||
| This script is the operational regression required by the baseline document's | ||
| own execution-loop contract: every loop must refetch live GitHub state and | ||
| reject stale buyer copy before acting. It never hard-codes volatile payloads; | ||
| every comparison is computed at runtime from the recorded inventory date, the | ||
| live default branch, and the complete live open pull-request/issue queues. | ||
|
|
||
| Exit codes: | ||
| 0 audit completed, or the baseline has not integrated yet; findings are reported | ||
| 2 contract violation or live-state evidence could not be established | ||
|
|
||
| The output is plain Markdown so callers can append it directly to | ||
| ``$GITHUB_STEP_SUMMARY``. | ||
| """ | ||
|
|
||
| from __future__ import annotations | ||
|
|
||
| import argparse | ||
| import json | ||
| import os | ||
| import re | ||
| import subprocess | ||
| import sys | ||
| from datetime import datetime | ||
| from pathlib import Path | ||
| from zoneinfo import ZoneInfo | ||
|
|
||
| INVENTORY_DATE_PATTERN = re.compile( | ||
| r"^Inventory date:\s*(\d{4}-\d{2}-\d{2})", re.MULTILINE | ||
| ) | ||
| SNAPSHOT_QUEUE_PATTERN = re.compile( | ||
| r"^At this snapshot,\s+(?P<open_pull_requests>\d+)\s+pull requests\s+" | ||
| r"and\s+(?P<open_issues>\d+|zero|one|two|three|four|five|six|seven|" | ||
| r"eight|nine|ten)\s+non-PR issues?(?:\s+\([^)]*\))?\s+are open\b", | ||
| re.MULTILINE, | ||
| ) | ||
| REPOSITORY_PATTERN = re.compile(r"^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$") | ||
| BASELINE_TIMEZONE = ZoneInfo("Asia/Seoul") | ||
| COUNT_WORDS = { | ||
| "zero": 0, | ||
| "one": 1, | ||
| "two": 2, | ||
| "three": 3, | ||
| "four": 4, | ||
| "five": 5, | ||
| "six": 6, | ||
| "seven": 7, | ||
| "eight": 8, | ||
| "nine": 9, | ||
| "ten": 10, | ||
| } | ||
|
|
||
|
|
||
| def _run_gh_json( | ||
| arguments: list[str], *, paginate: bool = False | ||
| ) -> list[dict[str, object]]: | ||
| """Return a list payload from ``gh api``, flattening all pages when asked.""" | ||
| command = ["gh", *arguments] | ||
| if paginate: | ||
| command.extend(["--paginate", "--slurp"]) | ||
| completed = subprocess.run( | ||
| command, | ||
| check=False, | ||
| capture_output=True, | ||
| text=True, | ||
| ) | ||
| if completed.returncode != 0: | ||
| raise RuntimeError(f"gh {' '.join(arguments)} failed: {completed.stderr.strip()}") | ||
|
|
||
| payload = json.loads(completed.stdout) | ||
| if paginate: | ||
| if not isinstance(payload, list): | ||
| raise RuntimeError("unexpected paginated GitHub payload shape") | ||
| flattened: list[dict[str, object]] = [] | ||
| for page in payload: | ||
| if not isinstance(page, list) or not all(isinstance(item, dict) for item in page): | ||
| raise RuntimeError("unexpected paginated GitHub page shape") | ||
| flattened.extend(page) | ||
| return flattened | ||
|
|
||
| if not isinstance(payload, list) or not all(isinstance(item, dict) for item in payload): | ||
| raise RuntimeError("unexpected GitHub list payload shape") | ||
| return payload | ||
|
|
||
|
|
||
| def _live_repository() -> str: | ||
| """Return the repository this workflow actually executes in.""" | ||
| repository = os.environ.get("GITHUB_REPOSITORY", "ContextualWisdomLab/Orgmetra") | ||
| if REPOSITORY_PATTERN.fullmatch(repository) is None: | ||
| raise RuntimeError("invalid GITHUB_REPOSITORY shape") | ||
| return repository | ||
|
|
||
|
|
||
| def _snapshot_queue_counts(baseline_text: str) -> tuple[int, int]: | ||
| """Read the recorded open PR and non-PR issue counts from the baseline.""" | ||
| match = SNAPSHOT_QUEUE_PATTERN.search(baseline_text) | ||
| if match is None: | ||
| raise RuntimeError("missing snapshot open queue counts") | ||
| issue_count = match.group("open_issues") | ||
| parsed_issue_count = ( | ||
| int(issue_count) if issue_count.isdigit() else COUNT_WORDS[issue_count] | ||
| ) | ||
| return int(match.group("open_pull_requests")), parsed_issue_count | ||
|
|
||
|
|
||
| def _live_state() -> dict[str, object]: | ||
| """Fetch complete live open PR/issue queues and newest develop integration.""" | ||
| repository = _live_repository() | ||
| open_pull_requests = _run_gh_json( | ||
| ["api", f"repos/{repository}/pulls?state=open&per_page=100"], | ||
| paginate=True, | ||
| ) | ||
| open_issues = _run_gh_json( | ||
| ["api", f"repos/{repository}/issues?state=open&per_page=100"], | ||
| paginate=True, | ||
| ) | ||
| # Issues and PRs share the issues endpoint; keep only genuine issues. | ||
| genuine_issues = [item for item in open_issues if "pull_request" not in item] | ||
| commits = _run_gh_json( | ||
| ["api", f"repos/{repository}/commits?sha=develop&per_page=1"] | ||
| ) | ||
| if not commits: | ||
| raise RuntimeError("develop commit payload is empty") | ||
| newest_commit_date = None | ||
| if commits: | ||
| commit = commits[0].get("commit") | ||
| if not isinstance(commit, dict): | ||
| raise RuntimeError("unexpected commit payload shape") | ||
| committer = commit.get("committer") | ||
| if not isinstance(committer, dict) or not isinstance(committer.get("date"), str): | ||
| raise RuntimeError("unexpected commit committer payload shape") | ||
| newest_commit_date = committer["date"] | ||
| return { | ||
| "open_pull_requests": len(open_pull_requests), | ||
| "open_issues": len(genuine_issues), | ||
| "newest_develop_commit_date": newest_commit_date, | ||
| } | ||
|
seonghobae marked this conversation as resolved.
|
||
|
|
||
|
|
||
| def main() -> int: | ||
| """Print a Markdown freshness report for the gap baseline snapshot.""" | ||
| parser = argparse.ArgumentParser(description=__doc__) | ||
| parser.add_argument("--baseline", default="docs/product-technical-gap-baseline.md") | ||
| arguments = parser.parse_args() | ||
|
|
||
| baseline_path = Path(arguments.baseline) | ||
| try: | ||
| baseline_text = baseline_path.read_text(encoding="utf-8") | ||
| except FileNotFoundError: | ||
| print( | ||
| "gap-baseline freshness: baseline not present on this integrated " | ||
| "branch yet; audit remains non-mutating and will activate when the " | ||
| "baseline owner integrates" | ||
| ) | ||
| return 0 | ||
| except OSError as error: | ||
| print(f"gap-baseline freshness: FAIL unreadable baseline: {error}") | ||
| return 2 | ||
|
|
||
| match = INVENTORY_DATE_PATTERN.search(baseline_text) | ||
| if match is None: | ||
| print("gap-baseline freshness: FAIL missing 'Inventory date:' header") | ||
| return 2 | ||
|
seonghobae marked this conversation as resolved.
|
||
|
|
||
| inventory_date = datetime.strptime(match.group(1), "%Y-%m-%d").date() | ||
| now_date = datetime.now(tz=BASELINE_TIMEZONE).date() | ||
| if inventory_date > now_date: | ||
| print( | ||
| "gap-baseline freshness: FAIL future inventory date " | ||
| f"{inventory_date.isoformat()} exceeds current Asia/Seoul date " | ||
| f"{now_date.isoformat()}" | ||
| ) | ||
| return 2 | ||
| age_days = (now_date - inventory_date).days | ||
| try: | ||
| snapshot_open_pull_requests, snapshot_open_issues = _snapshot_queue_counts( | ||
| baseline_text | ||
| ) | ||
| except (RuntimeError, ValueError) as error: | ||
| print(f"gap-baseline freshness: FAIL malformed baseline: {error}") | ||
| return 2 | ||
|
|
||
| lines = [ | ||
| "## Gap baseline freshness", | ||
| "", | ||
| f"- Baseline file: `{baseline_path.as_posix()}`", | ||
| f"- Inventory date: {inventory_date.isoformat()} " | ||
| f"(age {float(age_days):.1f} days)", | ||
| f"- Snapshot open pull requests: {snapshot_open_pull_requests}", | ||
| f"- Snapshot open issues: {snapshot_open_issues}", | ||
| ] | ||
|
|
||
| try: | ||
| state = _live_state() | ||
| except (RuntimeError, ValueError, KeyError) as error: | ||
| lines.append(f"- FAIL live-state fetch: {error}") | ||
| print("\n".join(lines)) | ||
| return 2 | ||
|
|
||
| lines.extend( | ||
| [ | ||
| f"- Live open pull requests: {state['open_pull_requests']}", | ||
| f"- Live open issues: {state['open_issues']}", | ||
| "- Newest develop integration: " | ||
| f"{state['newest_develop_commit_date']}", | ||
| ] | ||
| ) | ||
|
|
||
| newest_integration = state.get("newest_develop_commit_date") | ||
| integrations_after_snapshot = False | ||
| try: | ||
| if not isinstance(newest_integration, str): | ||
| raise ValueError("newest develop integration timestamp is missing") | ||
| integration_time = datetime.fromisoformat( | ||
| newest_integration.replace("Z", "+00:00") | ||
| ) | ||
| integrations_after_snapshot = ( | ||
| integration_time.astimezone(BASELINE_TIMEZONE).date() > inventory_date | ||
| ) | ||
| except (TypeError, ValueError) as error: | ||
| lines.append(f"- FAIL live-state fetch: invalid develop timestamp: {error}") | ||
| print("\n".join(lines)) | ||
| return 2 | ||
|
|
||
| queue_changed = ( | ||
| state["open_pull_requests"] != snapshot_open_pull_requests | ||
| or state["open_issues"] != snapshot_open_issues | ||
| ) | ||
|
|
||
| if integrations_after_snapshot or queue_changed: | ||
| reasons = [] | ||
| if integrations_after_snapshot: | ||
| reasons.append("develop integrated commits after the recorded inventory date") | ||
| if queue_changed: | ||
| reasons.append("the live open PR/issue queue differs from the snapshot") | ||
| lines.append( | ||
| "- Result: **refresh candidate** — " | ||
| + " and ".join(reasons) | ||
| + ". Per the execution loop, refresh the baseline only when " | ||
| "buyer/product-visible truth changed; otherwise record this audit " | ||
| "as observed." | ||
| ) | ||
| else: | ||
| lines.append( | ||
| "- Result: current — no develop integration is newer than the " | ||
| "recorded inventory snapshot." | ||
| ) | ||
|
|
||
| print("\n".join(lines)) | ||
| return 0 | ||
|
|
||
|
|
||
| if __name__ == "__main__": | ||
| sys.exit(main()) | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.