Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion .github/workflows/repository-metadata-reconcile.yml
Original file line number Diff line number Diff line change
Expand Up @@ -73,8 +73,10 @@ jobs:
--include=scripts/ci/reconcile_repository_metadata.py \
-m pytest -q \
tests/test_repository_metadata_reconciliation.py \
tests/test_repository_metadata_convergence.py \
tests/test_repository_metadata_identity.py \
tests/test_repository_metadata_live_verification.py \
tests/test_repository_metadata_workflow.py \
tests/test_repository_metadata_workflow_pages.py
python -m coverage report \
--fail-under=100 \
Expand All @@ -85,6 +87,7 @@ jobs:
--branch \
--include=scripts/ci/reconcile_repository_labels.py \
-m pytest -q \
tests/test_repository_label_taxonomy.py \
tests/test_repository_label_reconciliation.py \
tests/test_repository_label_convergence.py \
tests/test_repository_label_identity.py \
Expand All @@ -97,7 +100,6 @@ jobs:
--fail-under 100 \
scripts/ci/reconcile_repository_metadata.py \
scripts/ci/reconcile_repository_labels.py
python -m pytest -q
git diff --check

apply:
Expand Down
76 changes: 42 additions & 34 deletions docs/doctoring/actions-plan-concurrency-ceiling-20260903.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,16 @@
# Doctoring record: the org's GitHub Actions concurrency ceiling is a plan-level quota, not a workflow defect (2026-09-03)
# Actions 용량 한도와 workflow 부하: 관측 및 추론의 경계

2026-09-07 정정: 아래 2026-09-03 관측은 역사적 기록으로 보존한다.
용량 한도와 workflow 부하를 배타적 원인으로 단정했던 해석만 좁혔다.
현재 운영 상태나 이번 수리의 효과를 증명하는 문서가 아니다.

- **Date:** 2026-09-03
- **Subject:** two peer sessions independently observed the org's GitHub Actions run queue growing rather
than shrinking this week and, in that tick, proposed auditing/consolidating/centralizing workflow files
across the org as the fix. Before either session sank time into that plan, this root cause needed a
durable record: the actual bottleneck this session identified is a **plan-level concurrent-job quota**,
not workflow duplication, and consolidating workflow files cannot lift it.
across the org as the fix. 당시 사용자 보고는 동시 job 한도에 가까운 사용량을
뒷받침한다. 그러나 한도 포화와 중복 실행·장시간 점유·admission 결함은 함께
존재할 수 있다. 이 기록만으로 지배적 원인을 확정하거나 workflow 부하 원인을
배제할 수 없다. 파일 통합은 계약상 한도를 높이지 않지만 실행량은 줄일 수 있다.
- **Decision record:** none in `docs/adr/` — this is a diagnostic/root-cause finding for the org owner's
awareness and eventual plan-tier decision, not an architecture decision this repository can make.
- **PR:** see the PR that carries this commit.
Expand Down Expand Up @@ -34,29 +39,24 @@ gh api "repos/ContextualWisdomLab/<repo>/actions/runs?status=in_progress&per_pag
gh api "repos/ContextualWisdomLab/<repo>/actions/runs?status=queued&per_page=1" --jq '.total_count'
```

| Repository | `in_progress` | `queued` |
| Repository | `in_progress` workflow runs | `queued` workflow runs |
|---|---|---|
| `.github` | 5 | 1,877 |
| `contextual-orchestrator` | 0 | 727 |
| `naruon` | 5 | 416 |
| **Total (3-repo sample)** | **10** | **3,020** |

This is a deliberately small sample, not a full 63-repo census — an attempted full sweep across every
non-archived, non-fork repository (the same corpus as the 2026-09-02 workflow-duplication audit) hung
indefinitely on this run and was aborted; a post-hoc `gh api rate_limit` check immediately after showed
5,000/5,000 REST calls remaining, so the hang was not caused by hitting the org's shared REST rate limit
(consistent with this session's standing practice of preferring REST over GraphQL to avoid that limit) —
its actual cause is undetermined and not investigated further here, since the 3-repo sample already
establishes the pattern this record needs.

The pattern itself is the useful signal: single-digit `in_progress` counts (5, 0, 5) against
quadruple-digit `queued` counts (1,877; 727; 416) in the same moment, across independently-owned
repositories, each triggering its own workflows on its own schedule. That shape — many jobs queued,
very few ever concurrently running — is exactly what a hard, roughly-constant, **org-wide** (not
per-repository) concurrent-job ceiling produces, and is hard to explain by per-repository causes alone
(each repository's own workflow volume, trigger frequency, and CI design differ substantially). It is
consistent with, though does not by itself prove, the specific 58-60/60 figure from the primary evidence
above.
이 표는 당시 63개 저장소 전수가 아닌 세 저장소 표본이다. 당시 전체 조회는
멈춘 상태가 지속돼 중단했고, 이후 `gh api rate_limit`은 5,000/5,000을 반환했다.
실패 당시 응답과 헤더가 보존되지 않아 원인은 미확정이다. 사후 잔여량만으로
같은 인증 주체·resource의 실패 당시 primary quota, reset 또는 secondary 제한을
배제할 수 없다.

이 API가 세는 대상은 job이나 점유 runner가 아니라 workflow run이다. Run 하나에
여러 job이 있을 수 있고, in-progress run 안에서도 job이 대기할 수 있다.
세 저장소의 비원자적 표본만으로 조직 전체 동시 job 수, 실제 한도 포화 또는
저장소별 원인과 조직 공통 원인의 우선순위를 식별할 수 없다. 사용자 보고의
58–60/60과 양립하는 관측이지만 그 수치를 독립적으로 증명하지는 않는다.

## Relationship to other queue-related findings already in this repository

Expand Down Expand Up @@ -86,29 +86,37 @@ on this tick as *the* fix for the growing queue — is real hygiene and can redu
runs triggered* (fewer redundant CI paths competing for the same slots), which helps the queue drain
somewhat faster once jobs are submitted. It does **not** change how many jobs GitHub will run concurrently
for this organization at once: that number is set by the plan tier, not by how many `.yml` files exist or
how many of them are centralized versus per-repository. A large cross-repo consolidation-and-deletion
effort undertaken on the theory that it would resolve the backlog would be solving the wrong layer of the
problem, at real cost (each deletion needs branch-protection `required_status_checks` re-verified per
repo, and any repo-specific `with:` tuning preserved or intentionally dropped).
how many of them are centralized versus per-repository.

저장소 간 workflow 통합·삭제는 파일 재배치와 실제 중복 trigger·job·점유시간 감소를 구분해야 한다.
후자는 한도를 바꾸지 않고도 backlog를 줄일 수 있다. 삭제할 때는 저장소별
branch-protection required checks와 입력 조정값을 재검증하고 보존해야 한다.

## Recommendation

This is a plan/billing decision, not a code change either agent session can make: raising the concurrent-job
ceiling (a higher GitHub plan tier, purchasing additional included concurrency, or provisioning
self-hosted/larger runners with their own separate capacity pool) is the org owner's call to make with the
actual billing page in front of them, not something to infer further from repository-side evidence.
Workflow consolidation remains worth pursuing for its own, independent hygiene reasons (see
`docs/doctoring/ci-workflow-duplication-audit-20260902.md` for what is and is not already duplicated
org-wide) — but should not be scoped or prioritized as *the* fix for the current backlog growth.
용량 확대와 workflow 부하 감소는 별도 선택지다. 요금제·유료 용량·runner 추가는
실제 설정과 비용을 확인한 조직 소유자의 결정이 필요하다. 그 판단과 별개로 중복
실행, 불필요한 대기, stale-head 실행과 admission 결함은 기존 권한 안에서 수리한다.
정확한 전후 revision, trigger, job 실행시간, 취소 원인과 queue 표본 범위를 남겨
효과를 검증한다. 로컬 테스트 통과나 파일 수 감소만으로 운영 적체 해소를 주장하지 않는다.

## Audit trail

- User-reported screenshot of the organization's Actions usage view, shared earlier in this session
(primary source for the 58-60/60 figure; not independently re-verifiable from this record alone).
- Live `gh api` sample gathered 2026-09-03 for this record (table above); `gh api rate_limit` confirmed
5,000/5,000 REST calls remaining immediately after the aborted full-org sweep, ruling out rate-limiting
as the sweep's failure cause.
5,000/5,000 REST calls remaining immediately after the aborted full-org sweep. 사후 해당
primary quota 소진이 관측되지 않았다는 뜻이며 실패 당시 rate-limit을 배제하지 않는다.
- `docs/product-technical-gap-baseline.md` — the 2026-08-31 chained-poller-removal entry and the
`ubuntu-latest` starved-image entry, both cross-referenced above.
- `docs/doctoring/ci-workflow-duplication-audit-20260902.md` — the org-wide workflow-duplication sweep this
record's "Implication" section points back to.

## 정정 근거

- GitHub. (n.d.). *Using jobs in a workflow*. Retrieved September 7, 2026, from
https://docs.github.com/en/actions/how-tos/write-workflows/choose-what-workflows-do/use-jobs
— 하나의 workflow에 여러 job이 존재할 수 있으므로 두 개수를 구분한다.
- GitHub. (n.d.). *Rate limits for the REST API*. Retrieved September 7, 2026, from
https://docs.github.com/en/rest/using-the-rest-api/rate-limits-for-the-rest-api
— 인증 방식과 resource별 primary 제한, 별도 secondary 제한 및 조회 한계를 구분한다.
116 changes: 116 additions & 0 deletions tests/test_repository_metadata_workflow.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,104 @@
"""Static contracts for the privileged repository metadata workflow."""

import os
from pathlib import Path
import stat
import subprocess

import pytest

from tests.test_opencode_workflow_shell_syntax import _extract_run_block


ROOT = Path(__file__).resolve().parents[1]
WORKFLOW = ROOT / ".github" / "workflows" / "repository-metadata-reconcile.yml"
OWNER_TESTS = {
"tests/test_repository_metadata_reconciliation.py",
"tests/test_repository_metadata_convergence.py",
"tests/test_repository_metadata_identity.py",
"tests/test_repository_metadata_live_verification.py",
"tests/test_repository_metadata_workflow.py",
"tests/test_repository_metadata_workflow_pages.py",
"tests/test_repository_label_taxonomy.py",
"tests/test_repository_label_reconciliation.py",
"tests/test_repository_label_convergence.py",
"tests/test_repository_label_identity.py",
"tests/test_repository_label_live_verification.py",
}


def _metadata_test_script() -> str:
"""Extract the metadata test step's executable shell body."""

return _extract_run_block(
WORKFLOW.read_text(encoding="utf-8"),
"Run metadata contract tests at repository quality gates",
)


def _write_command_recorder(path: Path) -> None:
"""Create a fake executable that records argv without running project code."""

path.write_text(
'#!/bin/sh\nprintf "%s\\n" "$*" >> "$COMMAND_LOG"\n', encoding="utf-8"
)
path.chmod(path.stat().st_mode | stat.S_IXUSR)


def _record_metadata_commands(tmp_path: Path) -> list[str]:
"""Execute the workflow shell with inert commands and return recorded argv."""

fake_bin = tmp_path / "bin"
fake_bin.mkdir()
for command_name in ("python", "git"):
_write_command_recorder(fake_bin / command_name)
command_log = tmp_path / "commands.log"
environment = os.environ.copy()
environment.update(
PATH=f"{fake_bin}{os.pathsep}{environment['PATH']}",
COMMAND_LOG=str(command_log),
)
result = subprocess.run(
["bash", "-c", _metadata_test_script()],
cwd=ROOT,
env=environment,
check=False,
capture_output=True,
text=True,
)
assert result.returncode == 0, result.stderr
return command_log.read_text(encoding="utf-8").splitlines()


def _assert_metadata_commands(commands: list[str]) -> None:
"""Require the exact metadata-owner test set and no bare full-suite command."""

pytest_commands = [command for command in commands if "-m pytest -q" in command]
assert pytest_commands
invoked_tests: set[str] = set()
for command in pytest_commands:
targets = {
argument
for argument in command.split()
if argument.startswith("tests/") and argument.endswith(".py")
}
assert targets
assert targets <= OWNER_TESTS
invoked_tests.update(targets)
assert invoked_tests == OWNER_TESTS
assert sum(command.startswith("-m coverage run --branch") for command in commands) == 2
assert sum(
command.startswith("-m coverage report --fail-under=100")
for command in commands
) == 2
assert commands.count("-m coverage erase") == 1
interrogate = next(
command for command in commands if command.startswith("-m interrogate ")
)
assert "--fail-under 100" in interrogate
assert "scripts/ci/reconcile_repository_metadata.py" in interrogate
assert "scripts/ci/reconcile_repository_labels.py" in interrogate
assert commands.count("diff --check") == 1


def test_metadata_apply_uses_dedicated_least_privilege_credential() -> None:
Expand All @@ -16,3 +110,25 @@ def test_metadata_apply_uses_dedicated_least_privilege_credential() -> None:
assert "secrets.PR_REVIEW_MERGE_TOKEN" not in apply_source
assert "Require dedicated repository settings credential" in apply_source
assert 'test -n "${GH_TOKEN}"' in apply_source


def test_metadata_quality_step_runs_every_owned_test_without_the_full_suite(
tmp_path: Path,
) -> None:
"""The hourly job must run every metadata-owner test and no unrelated suite."""

commands = _record_metadata_commands(tmp_path)
_assert_metadata_commands(commands)


def test_metadata_quality_step_rejects_flagged_bare_full_suite(tmp_path: Path) -> None:
"""A pytest flag must not disguise an untargeted repository-wide invocation."""

commands = _record_metadata_commands(tmp_path)
pytest_index = next(
index for index, command in enumerate(commands) if "-m pytest -q" in command
)
commands[pytest_index] = "-m pytest -q -W error"

with pytest.raises(AssertionError):
_assert_metadata_commands(commands)
Loading