Skip to content

fix(ci): serialize AKS deploys without cancelling in-progress applies - #1596

Open
seonghobae wants to merge 2 commits into
developfrom
fix/deploy-aks-concurrency
Open

seonghobae wants to merge 2 commits into
developfrom
fix/deploy-aks-concurrency

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

목표와 현재 권위

deploy.yml은 release tag publication path에서 docker-publish.yml이 호출하는 AKS apply owner입니다. 기존에는 workflow-level concurrency가 없어 연속 publication이 동일한 naruon-dev namespace에 겹쳐 apply될 수 있었습니다.

  • protected base: develop@042b0c70531b229af3acbd0421a2f23098d848b3
  • current exact head: 7e29bfcacc02d950fb738a7d43a257582971e995
  • lifecycle: Ready for independent review / mergeable / not merge-authorized
  • effective scope: .github/workflows/deploy.yml, backend/tests/test_deploy_workflow_concurrency.py

bandit.yml, docker-publish.yml, mail-smoke.yml의 별도 owner PR은 수정하지 않았습니다.

finding → causal fix

GitHub Actions의 default queue: single은 running 1개 + pending 1개 상태에서 추가 run이 들어오면 기존 pending run을 교체합니다. 배포 publication은 오래된 requested deploy를 조용히 버리는 동작이 적절하지 않으므로 repository-wide deploy group에 cancel-in-progress: falsequeue: max를 사용합니다. queue: max는 현재 concurrency group당 pending run 최대 100개를 보존하며 초과 요청은 취소될 수 있습니다. 실행 시작 순서는 플랫폼 scheduling 영향이 있으므로 사용자 수준 FIFO를 보장한다고 주장하지 않습니다.

Original source 56d30a07d48f688519543815fd601e533dc8c3cd의 workflow comment는 이 실제 contract와 달리 “Keep every requested deploy queued”라고 적혀 있었습니다. #1595에서 같은 계열의 CodeRabbit finding을 검증한 뒤 이 PR도 fresh source로 대조해 동일한 과장 표현임을 확인했습니다. Ordinary child 7e29bfcacc02d950fb738a7d43a257582971e995에서 동작은 그대로 두고 comment만 bounded semantics로 수정했습니다: pending 최대 100개, bound 초과 시 취소 가능, FIFO 보장 없음. predecessor GREEN은 새 head로 승계하지 않습니다.

Primary authority:

Focused regression backend/tests/test_deploy_workflow_concurrency.py는 repository-wide deploy-aks-${{ github.repository }}, cancel-in-progress: false, queue: max, cancel-in-progress: true 비재도입을 고정합니다.

exact-head validation boundary

Current unchanged 7e29bfc... repository-owned runs are terminal except the shared required CodeQL verdict:

  • Application CI 34173300788success
  • Bandit Security Scan 34173300820success
  • Build and Publish Docker Images 34173300951success
  • Security Scan 34173300812success
  • SAST Semgrep 34173300835success
  • CodeQL PR 34173300797failure at the shared fail-closed terminal-verdict bridge; 이 PR의 deploy source-analysis finding으로 입증된 실패가 아님.

Current inline review thread는 0개이고 formal review submission도 아직 0개입니다. Exact-head @coderabbitai review를 다시 요청했으며 response/approval을 merge evidence로 미리 간주하지 않습니다.

CodeQL failure는 canonical ContextualWisdomLab/.github#1929가 소유하는 current-head terminal receipt/publication 문제와 같은 패턴입니다. Naruon에서 synthetic status, dummy commit, authorization 확대, central workflow 복제나 gate 완화로 우회하지 않습니다.

acceptance

  1. 동일 repository의 AKS apply는 한 번에 하나만 실행 — source/test contract PASS.
  2. 실행 중 deploy를 superseding publication이 취소하지 않음 — cancel-in-progress: false, PASS at source.
  3. pending publication은 bounded queue에 보존 — queue: max, PASS at source; platform maximum 100, FIFO/unlimited claim 없음.
  4. queue semantics comment가 실제 GitHub contract와 일치 — PASS.
  5. current-head Application/Bandit/Docker/Security/Semgrep — PASS; required CodeQL terminal verdict FAIL CLOSED.
  6. unresolved source-backed review finding 0 + post-last-push independent approval — source finding 0 observed / approval PENDING.

Merge는 이 unchanged exact head에서 당시의 모든 required context가 terminal-success이고 qualifying post-last-push independent approval이 있을 때만 허용합니다. force-push, destructive rebase, self-approval, synthetic status, dummy/no-op requeue, central authorization widening, gate weakening은 금지합니다.

Queue concurrent Deploy to AKS calls per repository so the shared
naruon-dev namespace is not applied twice at once. Keep
cancel-in-progress false because this is a deploy publication path.
@coderabbitai

coderabbitai Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

Next included review available in 21 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Team

Run ID: 494da7c4-efd9-4942-b581-ebc059670e72

📥 Commits

Reviewing files that changed from the base of the PR and between 042b0c7 and 7e29bfc.

📒 Files selected for processing (2)
  • .github/workflows/deploy.yml
  • backend/tests/test_deploy_workflow_concurrency.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae
seonghobae marked this pull request as ready for review September 8, 2026 00:12
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, add credits to your account and enable them for code reviews in your settings.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Exact head 56d30a07d48f688519543815fd601e533dc8c3cd를 protected develop@042b0c70531b229af3acbd0421a2f23098d848b3 기준으로 검토해 주세요. Application CI·Bandit·Security는 current head에서 GREEN이고 Docker·Semgrep·CodeQL은 fresh/non-terminal 상태입니다. 대기 상태를 approval로 간주하지 말고 source finding만 검증해 주세요.

@coderabbitai

coderabbitai Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

base="042b0c70531b229af3acbd0421a2f23098d848b3"
head="56d30a07d48f688519543815fd601e533dc8c3cd"

echo '== checked-out commit =='
git rev-parse HEAD
echo '== exact range changed files =='
git diff --name-status "$base" "$head"
echo '== exact range diff =='
git diff --check "$base" "$head"
git diff --find-renames --find-copies --unified=80 "$base" "$head" -- \
  .github/workflows/deploy.yml \
  backend/tests/test_deploy_workflow_concurrency.py

echo '== workflow outline/content =='
ast-grep outline .github/workflows/deploy.yml --items all || true
cat -n .github/workflows/deploy.yml

echo '== regression test outline/content =='
ast-grep outline backend/tests/test_deploy_workflow_concurrency.py --items all || true
cat -n backend/tests/test_deploy_workflow_concurrency.py

echo '== deploy workflow references and concurrency settings =='
rg -n -C 4 --glob '*.yml' --glob '*.yaml' \
  '(^|\s)(concurrency:|group:|cancel-in-progress:|queue:)|deploy-aks-|deploy\.yml' \
  .github

Length of output: 13682


🤖 web_search executed:

GitHub Actions workflow concurrency queue: max supported syntax maximum pending runs 100 cancel-in-progress false documentation

Length of output: 11658


@seonghobae: 소스 검토를 완료했습니다. Exact head는 요청한 56d30a07d48f688519543815fd601e533dc8c3cd와 일치합니다.

변경 범위는 .github/workflows/deploy.ymlbackend/tests/test_deploy_workflow_concurrency.py뿐입니다. 소스 기반 finding은 없습니다.

deploy.yml은 workflow-level concurrency를 사용합니다. Group은 deploy-aks-${{ github.repository }}입니다. 이 설정은 동일 repository의 AKS deploy를 하나로 직렬화합니다.

cancel-in-progress: false는 실행 중인 deploy를 취소하지 않습니다. queue: max는 최대 100개의 pending run을 보존합니다. 이 조합은 GitHub Actions concurrency 문서의 유효한 설정입니다.

회귀 테스트는 group, cancel-in-progress: false, queue: max를 확인합니다. 또한 workflow header에 cancel-in-progress: true가 없음을 확인합니다.

Docker, Semgrep, CodeQL의 non-terminal 상태는 이 검토의 approval 근거로 사용하지 않았습니다.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Exact head 7e29bfcacc02d950fb738a7d43a257582971e995를 protected develop@042b0c70531b229af3acbd0421a2f23098d848b3 기준으로 검토해 주세요. 56d30a07...의 queue semantics comment 과장을 bounded 100-pending/no-FIFO contract로 고쳤습니다. Predecessor checks/reviews는 새 head로 승계하지 않습니다.

@coderabbitai

coderabbitai Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

base="042b0c70531b229af3acbd0421a2f23098d848b3"
head="7e29bfcacc02d950fb738a7d43a257582971e995"

echo "== checked-out commit =="
git rev-parse HEAD
echo "== exact range changed files =="
git diff --name-status "$base" "$head"
echo "== whitespace check =="
git diff --check "$base" "$head"
echo "== exact scoped diff =="
git diff --find-renames --find-copies --unified=100 "$base" "$head" -- \
  .github/workflows/deploy.yml \
  backend/tests/test_deploy_workflow_concurrency.py

echo "== workflow content =="
cat -n .github/workflows/deploy.yml
echo "== regression test content =="
cat -n backend/tests/test_deploy_workflow_concurrency.py

echo "== relevant workflow settings across workflow files =="
rg -n -C 4 --glob '*.yml' --glob '*.yaml' \
  '(^|\s)(concurrency:|group:|cancel-in-progress:|queue:)|deploy-aks-|deploy\.yml' \
  .github

echo "== current PR check rollup =="
gh pr checks 1596 --repo ContextualWisdomLab/naruon --json name,state,link,workflow 2>&1 || true

Length of output: 18954


@seonghobae 소스 검토를 완료했습니다.

Checked-out head는 7e29bfcacc02d950fb738a7d43a257582971e995와 일치합니다. 비교 기준은 develop@042b0c70531b229af3acbd0421a2f23098d848b3입니다.

변경 범위는 .github/workflows/deploy.ymlbackend/tests/test_deploy_workflow_concurrency.py입니다. 소스 기반 finding은 없습니다.

.github/workflows/deploy.yml은 workflow-level concurrency를 사용합니다.

  • group: deploy-aks-${{ github.repository }}는 repository별 AKS apply를 직렬화합니다.
  • cancel-in-progress: false는 실행 중인 deploy를 취소하지 않습니다.
  • queue: max는 bounded pending queue를 사용합니다.
  • Line 13부터 Line 15의 comment는 pending 최대 100개, 초과 요청 취소 가능, FIFO 미보장을 명시합니다.

backend/tests/test_deploy_workflow_concurrency.py는 group, cancel-in-progress: false, queue: max를 검증합니다. 이 테스트는 workflow header에 cancel-in-progress: true가 없음을 검증합니다.

git diff --check는 문제를 보고하지 않았습니다.

현재 head의 CI에는 IN_PROGRESSQUEUED 상태가 남아 있습니다. 이 검토는 predecessor 결과나 non-terminal 결과를 approval 근거로 사용하지 않았습니다. Fresh terminal checks와 post-last-push independent approval은 아직 별도 조건입니다.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

Current exact head is 7e29bfcacc02d950fb738a7d43a257582971e995. Please review only the protected-base delta for .github/workflows/deploy.yml and backend/tests/test_deploy_workflow_concurrency.py; verify the bounded queue: max semantics, non-cancellation of in-progress deploys, and the regression test. No predecessor verdict transfer.

@coderabbitai

coderabbitai Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor
🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

base="042b0c70531b229af3acbd0421a2f23098d848b3"
head="7e29bfcacc02d950fb738a7d43a257582971e995"

echo "== checked-out revision =="
git rev-parse HEAD
echo "== base and head objects =="
git cat-file -t "$base"
git cat-file -t "$head"

echo "== protected-base changed paths =="
git diff --name-status "$base" "$head"
echo "== whitespace errors =="
git diff --check "$base" "$head"

echo "== scoped protected-base delta =="
git diff --find-renames --find-copies --unified=100 "$base" "$head" -- \
  .github/workflows/deploy.yml \
  backend/tests/test_deploy_workflow_concurrency.py

echo "== deploy workflow =="
cat -n .github/workflows/deploy.yml
echo "== regression test =="
cat -n backend/tests/test_deploy_workflow_concurrency.py

echo "== concurrency-related workflow configuration =="
rg -n -C 5 --glob '*.yml' --glob '*.yaml' \
  '(^|\s)(concurrency:|group:|cancel-in-progress:|queue:)|deploy-aks-|deploy\.yml' \
  .github

echo "== exact-head PR check rollup (informational only) =="
gh pr checks 1596 --repo ContextualWisdomLab/naruon --json name,state,link,workflow 2>&1 || true

Length of output: 22824


🤖 web_search executed:

GitHub Actions workflow concurrency queue max syntax maximum 100 pending runs cancel-in-progress false official GitHub documentation

Length of output: 13554


@seonghobae Source review completed for exact head 7e29bfcacc02d950fb738a7d43a257582971e995 against protected base develop@042b0c70531b229af3acbd0421a2f23098d848b3.

I found no source-backed issues in the requested scope.

  • The protected-base delta changes only .github/workflows/deploy.yml and backend/tests/test_deploy_workflow_concurrency.py.
  • .github/workflows/deploy.yml uses workflow-level concurrency.
  • group: deploy-aks-${{ github.repository }} creates one concurrency group per repository. This serializes AKS applies that invoke this reusable workflow in the repository.
  • cancel-in-progress: false prevents a later publication from canceling an in-progress deploy.
  • queue: max permits up to 100 pending runs in the concurrency group. Further runs can be canceled when the queue is full.
  • The comment at Line 13 through Line 15 describes the bounded queue and does not claim unlimited retention or FIFO ordering.
  • queue: max is not combined with cancel-in-progress: true. This matches the GitHub Actions concurrency contract.
  • backend/tests/test_deploy_workflow_concurrency.py verifies the group, cancel-in-progress: false, and queue: max. It also rejects cancel-in-progress: true in the workflow header.
  • git diff --check reported no whitespace errors.

I did not transfer predecessor review results or check results. I did not use the current check rollup as an approval basis.

⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode could not approve from deterministic current-head evidence because GitHub Checks have failed.

Findings

1. HIGH Current-head GitHub Checks - Fix failed required checks before approval

  • Problem: Failed same-head checks remain for 7e29bfcacc02d950fb738a7d43a257582971e995.
  • Root cause: The model-unavailable evidence fallback is allowed only when peer GitHub Checks are complete and clean.
  • Fix: Read and fix the failed check logs below, then rerun the current-head checks.
  • Regression test: Keep the model-unavailable fallback gated on an empty failed-check rollup.

Failed checks:

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: deploy.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: deploy.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Test: test_deploy_workflow_concurrency.py"]
  S2 --> I2["regression suite"]
  I2 --> R2["Review risk: Test: test_deploy_workflow_concurrency.py"]
  R2 --> V2["targeted test run"]
Loading

@opencode-agent

opencode-agent Bot commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

@seonghobae seonghobae added bug Something isn't working priority: medium Normal-priority or P2 work labels Sep 8, 2026 — with ChatGPT Codex Connector
@seonghobae

Copy link
Copy Markdown
Contributor Author

Current exact-head audit (2026-09-08): head 7e29bfcacc02d950fb738a7d43a257582971e995, base develop@042b0c70531b229af3acbd0421a2f23098d848b3. The blocking CodeQL/Noema/OpenCode results are attached to the same historical head, but the referenced workflow runs return GitHub API HTTP 404, so their logs are not retrievable evidence. CodeRabbit source review reports no finding in the changed workflow/test scope. I am not transferring the stale verdict, changing source without a reproducible finding, or blindly rerunning the unchanged head. A fresh exact-head validation/review is required after any authorized repair or base/head change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working priority: medium Normal-priority or P2 work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant