Skip to content

fix(coverage): trust validated Python head locks - #1398

Open
seonghobae wants to merge 20 commits into
mainfrom
fix/coverage-python-head-lock-trust
Open

fix(coverage): trust validated Python head locks#1398
seonghobae wants to merge 20 commits into
mainfrom
fix/coverage-python-head-lock-trust

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • materialize changed Python requirement locks from the authenticated exact PR HEAD;
  • replace the stale base lock instead of installing both revisions;
  • revalidate bounded includes beneath unchanged base locks against exact HEAD blobs;
  • re-export changed or deleted uv.lock projects from exact HEAD, including registry and VCS inputs;
  • preserve every distinct package/import owner sharing one immutable VCS repository revision while rejecting conflicting commits;
  • pass PR_HEAD_SHA through the central OpenCode coverage workflow; and
  • document the boundary and regression contract.

Root cause

coverage-evidence built its trusted Python image only from PR_BASE_SHA. A dependency PR such as ContextualWisdomLab/accounting-information-platform#38 could add the Python 3.14 wheel hash while the builder still consumed the older base lock, so pip failed before target tests with a stale hash set. Unchanged parent requirements locks could likewise retain stale included locks. Changed or deleted uv.lock projects could retain stale base registry and VCS exports, and collapsing shared VCS ownership could hide an unaffected source or conflict.

Security boundary

Only regular candidate lock blobs read from the validated HEAD can replace a base candidate. They must be flat, exact == requirements with complete SHA-256 hashes. Bounded includes beneath unchanged parents are compared by exact base/head blob identity and read from HEAD only when they remain regular, flat, and fully pinned. Changed uv.lock projects use the existing isolated frozen/offline exporter; URLs, unpinned content, source distributions, and lifecycle execution remain outside this path. The image install retains pip install --require-hashes --only-binary=:all:, and the later PR sandbox remains networkless and credential-free.

Verification

  • RED: same-repository/same-commit package owners collapsed to one record, and Pingora admitted unbounded aggregate content requests/bytes;
  • GREEN focused regression/contract suite: 183 passed, plus both exhausted and crossing-file aggregate byte-budget cases;
  • latest exact-head full suite: 1905 passed, 1 skipped, 17 subtests passed;
  • statement/branch coverage: 100% (10072/10072 statements, 3982/3982 branches);
  • production docstring coverage: 100%;
  • compileall and git diff --check passed;
  • exact AIP Record scheduler rollout PRs #38 base/head lock selection selects the HEAD digest 8c0056529e68dbe9184cd4019a1f3d8f3a4ead2f6fc7a5afcf27d3314edd1277.

Live non-passing evidence (2026-08-30 KST)

  • Exact head: ca084e7c12066f54b86754c1548baec5ea95a5b3 on protected main@6c8ee24046d743b3981c566c6e29f99f09137f6a.
  • The two original CodeRabbit defects are repaired in source and covered by fail-closed regressions: VCS owner records no longer overwrite one another, and Pingora enforces both a 256-request and 16 MiB aggregate content budget.
  • Fresh review regressions prove that a changed generic requirements lock with an empty logical dependency closure removes the stale base selection, while undecodable bytes cannot impersonate an empty closure; option-only, non-empty unpinned, and malformed replacements fail closed.
  • Current-head SAST, Security Scan, OSV, CodeQL, Python Security, SBOM/attestation, and Trusted uv Materializer Quality completed successfully. SAST, Dependency Review, Trivy, and OSV logs attest exact head ca084e7c; Trivy and Semgrep report 0 blocking findings, and OSV reports 0 results.
  • Required Strix and Noema each failed closed after contextual-orchestrator sidecar gateway preflight returned HTTP 502; Strix produced no authoritative vulnerability report. The Changed Path Quality gate independently fails at the fix(opencode): remove redundant bootstrap event guard #1414 required-workflow bootstrap contract. Generated code-scanning neutral/skipped evidence remains non-passing.\n- Fresh OpenCode Dispatch run 33286427421 completed fail-closed: exact-head coverage jobs passed, but sidecar provisioning/status publication failed and no formal review was written. Reviews API still has 0 exact-current-head verdicts; the required OpenCode gate correctly fails closed, and predecessor CHANGES_REQUESTED reviews do not satisfy this head.
  • The local Strix quick gate still reaches the independent required-workflow bootstrap failure tracked by #1414; it was not reclassified or bypassed.

Refs #1250

Summary by CodeRabbit

  • 개선 사항

    • Python 소스나 관련 의존성 변경이 있을 때만 Python 요구사항 검증을 수행합니다.
    • 변경된 프로젝트는 검증된 최신 PR 기준의 잠금 정보를 사용하며, 변경되지 않은 프로젝트는 기존 기준을 유지합니다.
    • 모든 추적 .txt 변경을 Python 의존성 검사 대상에 포함합니다.
    • 리뷰 및 보안 검증에서 커밋 무결성, 허용 라우트, 로컬 readiness 상태를 더욱 엄격하게 확인합니다.
    • 대규모 변경 파일 목록을 페이지 단위로 안정적으로 처리합니다.
  • 문서

    • 최신 잠금 정보 검증 및 안전 경계를 관련 문서와 변경 기록에 반영했습니다.

@coderabbitai

coderabbitai Bot commented Aug 29, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

Next included review available in 33 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: 9b763aad-7718-49b1-9a30-468c27a369f2

📥 Commits

Reviewing files that changed from the base of the PR and between 6f8c51d and 8ff7cc0.

📒 Files selected for processing (13)
  • .github/workflows/opencode-review-dispatch.yml
  • ARCHITECTURE.md
  • CHANGELOG.md
  • docs/doctoring/opencode-exact-vcs-dependency-evidence.md
  • docs/doctoring/opencode-python-head-lock-trust.md
  • docs/doctoring/trusted-uv-lock-materialization.md
  • scripts/ci/materialize_base_python_requirements.py
  • scripts/ci/pingora_edge_policy.py
  • tests/test_materialize_base_python_requirements.py
  • tests/test_opencode_agent_contract.py
  • tests/test_pingora_edge_policy.py
  • tests/test_pr_review_autofix_nvidia_nim_contract.py
  • tests/test_uv_export_isolation_contract.py
📝 Walkthrough

Walkthrough

검증된 PR HEAD의 변경 상태에 따라 Python 잠금과 uv.lock 프로젝트를 선택적으로 구체화합니다. bounded include를 HEAD에서 재검증합니다. 변경 파일 페이지 조회의 3,000개 제한과 Strix 계약 값도 갱신합니다.

Changes

현재 HEAD 잠금 구체화

Layer / File(s) Summary
HEAD 잠금 선택과 uv 입력 재내보내기
scripts/ci/materialize_base_python_requirements.py
변경된 HEAD 잠금을 flat SHA-256 pin으로 검증합니다. 변경된 uv.lock 프로젝트를 HEAD에서 재내보내고 삭제된 입력을 제거합니다.
워크플로 연결과 Python 회귀 검증
.github/workflows/opencode-review-dispatch.yml, tests/test_materialize_base_python_requirements.py, tests/test_opencode_agent_contract.py, tests/test_pr_review_autofix_nvidia_nim_contract.py
Python 관련 변경이 있을 때만 materialization을 실행합니다. 검증된 PR_HEAD_SHA를 전달하고 잠금, include, uv 입력, 정책 아카이브 계약을 검증합니다.
변경 파일 페이지네이션 보정
scripts/ci/pingora_edge_policy.py, tests/test_pingora_edge_policy.py
변경 파일을 동적으로 페이지 조회합니다. 3,001개 응답에서 오류가 발생하는지 검증합니다.
설계와 릴리스 기록 문서화
ARCHITECTURE.md, CHANGELOG.md, docs/doctoring/*
HEAD 잠금 선택, uv 프로젝트 재내보내기, VCS 경계, 검증 조건과 워크플로 변경을 기록합니다.

Estimated code review effort: 4 (Complex) | ~45 minutes

Merge Risk: 🟠 High · up to 8c086

The PR updates dependency evidence to use validated PR-head locks, but same-repository VCS dependencies can still be collapsed incorrectly and omit required imports, while large pull requests can trigger excessive policy-content downloads that delay or exhaust required CI capacity. These concrete correctness and availability risks should be addressed before merge.

Sequence Diagram(s)

sequenceDiagram
  participant CoverageWorkflow
  participant Materializer
  participant GitRevision
  participant UVExporter
  CoverageWorkflow->>Materializer: base SHA와 head SHA 전달
  Materializer->>GitRevision: 잠금과 sibling metadata 비교
  GitRevision-->>Materializer: 변경 경로와 검증된 blob 반환
  Materializer->>UVExporter: 변경된 uv.lock 프로젝트 HEAD export 요청
  UVExporter-->>Materializer: registry 및 VCS 입력 반환
  Materializer-->>CoverageWorkflow: trusted Python lock 생성
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed Docstring coverage is 88.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 50 functions across 6 files. (6 skipped: 6 …
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 검증된 PR HEAD의 Python 잠금을 신뢰하도록 coverage 처리를 수정하는 주요 변경 사항을 정확하고 간결하게 설명합니다.
Full details: Docstring Coverage

Explanation

Docstring coverage is 88.00% which is sufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 50 functions across 6 files. (6 skipped: 6 unsupported.)

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/coverage-python-head-lock-trust

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

devin-ai-integration[bot]

This comment was marked as resolved.

coderabbitai[bot]

This comment was marked as resolved.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Addressed in 2ef42b67 (current exact head). The VCS trust-boundary document now explicitly includes changed or newly added exact-head uv.lock projects in both the decision and safety statements, matching the materializer and regression coverage. git diff --check passes.

devin-ai-integration[bot]

This comment was marked as resolved.

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

OpenCode could not approve from deterministic current-head evidence because GitHub Checks have failed.

Findings

1. HIGH Current-head GitHub Checks - Fix failed required checks before approval

  • Problem: Failed same-head checks remain for 2ef42b67637c43097401ddbd3c3ad5878b9dc4e0.
  • Root cause: The model-unavailable evidence fallback is allowed only when peer GitHub Checks are complete and clean.
  • Fix: Read and fix the failed check logs below, then rerun the current-head checks.
  • Regression test: Keep the model-unavailable fallback gated on an empty failed-check rollup.

Failed checks:

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Workflow: opencode-review-dispatch.yml"]
  S1 --> I1["GitHub Actions review job"]
  I1 --> R1["Review risk: Workflow: opencode-review-dispatch.yml"]
  R1 --> V1["actionlint plus required checks"]
  Evidence --> S2["Repository file: ARCHITECTURE.md"]
  S2 --> I2["repository behavior"]
  I2 --> R2["Review risk: Repository file: ARCHITECTURE.md"]
  R2 --> V2["required checks"]
  Evidence --> S3["Repository file: CHANGELOG.md"]
  S3 --> I3["repository behavior"]
  I3 --> R3["Review risk: Repository file: CHANGELOG.md"]
  R3 --> V3["required checks"]
  Evidence --> S4["Docs: opencode-exact-vcs-dependency-evidence.md (3 files)"]
  S4 --> I4["operator or user guidance"]
  I4 --> R4["Review risk: Docs: opencode-exact-vcs-dependency-evidence.md (3 files)"]
  R4 --> V4["docs review"]
  Evidence --> S5["CI script: materialize_base_python_requirements.py"]
  S5 --> I5["review and security gate shell path"]
  I5 --> R5["Review risk: CI script: materialize_base_python_requirements.py"]
  R5 --> V5["bash -n plus Strix self-test"]
  Evidence --> S6["Test: test_materialize_base_python_requirements.py (3 files)"]
  S6 --> I6["regression suite"]
  I6 --> R6["Review risk: Test: test_materialize_base_python_requirements.py (3 files)"]
  R6 --> V6["targeted test run"]
Loading

@opencode-agent

opencode-agent Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor

OpenCode Review Overview

…ead-lock-trust

# Conflicts:
#	CHANGELOG.md
#	tests/test_pr_review_autofix_nvidia_nim_contract.py
@seonghobae

Copy link
Copy Markdown
Contributor Author

Updated the existing central repair PR on exact head ac5c3aae904fe10e0f2e6226220c3522d008a764 by merging origin/main with a regular merge commit. The two conflicts were resolved by retaining both the validated-head Python-lock repair and the current main branch's Strix/loopback changes; the workflow contract blob SHA was recalculated against the merged workflow.

Local verification on the merged tree: 1779 passed, 1 skipped, 17 subtests passed; focused materializer/OpenCode/autofix contracts also passed (144 passed), with no diff-check errors. The branch remains subject to the required exact-head hosted checks, independent approval, and protected-branch rules.

devin-ai-integration[bot]

This comment was marked as resolved.

…ead-lock-trust

# Conflicts:
#	.github/workflows/opencode-review-dispatch.yml
#	CHANGELOG.md
#	tests/test_pr_review_autofix_nvidia_nim_contract.py
devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Pushed exact-head repair 727e7245 (727e7245...):

  • changed/deleted uv.lock projects are inventoried before base export, so a stale base export cannot block an exact-HEAD repair; unchanged projects remain base-bound and deleted projects remove registry/VCS inputs;
  • added RED->GREEN regressions for changed and deleted projects whose base export fails;
  • fixed the pre-existing pingora_edge_policy.py:274 dead pagination fallback exposed by the hosted full-quality gate, preserving fail-closed behavior without a coverage pragma.

Verification on this exact local commit: 1896 passed, 1 skipped, 17 subtests passed; full statement/branch coverage 100%; Interrogate 100%; Ruff, compileall, and git diff --check passed. The prior exact-head hosted full-quality failure was pingora_edge_policy.py:274 at 99%; the local equivalent now reports 100%.

No merge or approval bypass was attempted; please review the new exact head and rerun the protected checks.

devin-ai-integration[bot]

This comment was marked as resolved.

@seonghobae

Copy link
Copy Markdown
Contributor Author

Pushed exact-head follow-up 8c0869a5:

  • changed/deleted uv.lock projects are still inventoried before base export;
  • every tracked .txt change now keeps exact-head Python materialization enabled, covering bounded includes outside requirements/ (for example locks/child.txt), with content validation remaining fail-closed;
  • materialize rejects malformed base and HEAD SHAs before any Git tree read;
  • the workflow contract assertion is anchored to the Python materializer invocation and its current workflow blob SHA is paired.

RED->GREEN regressions cover the outside-directory include and pre-Git HEAD validation. Final local verification on the resulting tree: 1897 passed, 1 skipped, 17 subtests passed, statement/branch coverage 100%, Interrogate 100%, Ruff, compileall, and git diff --check passed.

No approval or merge bypass was attempted.

Copy link
Copy Markdown
Contributor Author

Fresh BandScope downstream canary for this exact ownership boundary:

  • consumer: ContextualWisdomLab/bandscope#1033
  • protected consumer base: develop@749511c3ad4000090048718f685c6bee6b3d2c25
  • exact consumer head: 046db562497a8104fa525f56a6437eb13fbf4760
  • OpenCode dispatch: run 33245610267, coverage job 99082941694
  • trusted central workflow head for that dispatch: 3a7941aa92de00b8b39fd11cbe7bf3da2fbbeddc
  • coverage-source-tree succeeded and materialized synthetic merge ca6d32d77db1c7c4ef9b8125fb16ea37f4af55d0 with parents 749511c3... + 046db562...; the uploaded source artifact contains the changed current-head services/analysis-engine/uv.lock and pyproject.toml.
  • coverage-evidence then failed specifically in Measure test and docstring evidence; syntax/replay/source-materialization gates immediately before it were green, and OpenCode consequently published COVERAGE_BLOCKED rather than a formal verdict.

The consumer is a Python dependency-security PR: its head changes the analysis-engine lock/manifest to the patched baseline (setuptools 84.0.0, torch 2.13.0 through the Demucs-owned graph, yt-dlp 2026.8.19). The central workflow used by this failed run invokes Python dependency materialization from PR_BASE_SHA without the exact-head replacement path that this PR adds. Reverting the BandScope lock or restoring the old dependency set is not a correct consumer-local repair because it would undo the security-baseline change.

This therefore provides a second concrete consumer for #1398's stale-base-lock class, independent of the AIP #38 reproducer. Acceptance after protected integration: rerun unchanged BandScope #1033 head 046db562...; coverage-evidence must consume the authenticated exact-head changed uv.lock/sibling metadata under #1398's existing bounded hash/VCS rules, proceed beyond trusted-image dependency materialization, and emit current-head test/coverage evidence (or a new source-backed BandScope failure). Do not treat the historical COVERAGE_BLOCKED run as success.

…hon-head-lock-trust

# Conflicts:
#	CHANGELOG.md
seonghobae pushed a commit that referenced this pull request Aug 30, 2026
and removing a now-forbidden Python verdict mirror

main landed substantial independent work on the scheduler/review
surface while this large PR was open (183 commits behind). Reconciled:

- scripts/ci/pr_review_merge_scheduler.py: kept both independently
  added review-dispatch-priority helpers (this branch's
  has_any_opencode_verdict/review_dispatch_priority/
  prioritize_review_dispatch_queue) and main's
  can_retry_check_gated_opencode_review; combined the two competing
  queue-sort strategies (this branch's never-reviewed-first vs. main's
  stacked-PRs-first) into one sort key
  (baseRefName-matches-default, review_dispatch_priority) so both
  properties hold together.
- .github/workflows/pr-review-merge-scheduler.yml: adopted main's
  target_default_branch fix over this branch's live_default_branch:
  main's design is the correct one — using the PR's own base ref
  (e.g. "develop" for a stacked naruon PR) as the scheduler's
  base_branch output would make a stacked PR compare equal to its own
  base and silently bypass the stacked-review dispatch budget path.
  Removed the now-dead repository_json/live_default_branch fetch.
- .github/workflows/opencode-review-dispatch.yml /
  tests/test_pr_review_autofix_nvidia_nim_contract.py: kept this
  branch's VCS-license-validator install and pnpm packageManager
  helper alongside main's Python-coverage change-scoping and lock
  revision-tracking helper (independent, non-overlapping additions);
  refreshed the reviewed-workflow blob-pin hash.
- opencode-review.yml: kept main's added APPROVED/CHANGES_REQUESTED
  review-state filter.
- Removed scripts/ci/opencode_dispatch_status.py's
  current_head_opencode_verdict/decide_required_verdict_check and the
  --mode required-verdict CLI path this branch had added: main added
  tests/test_opencode_required_verdict_regression.py, which executes
  the real embedded jq gate in opencode-review.yml directly and
  explicitly forbids a Python "mirror" of that logic
  (test_required_verdict_has_one_executable_owner) so the two
  implementations cannot drift apart. This branch's mirror was never
  wired into any workflow (only the default dispatch-status mode is
  invoked), so it was dead code; deleted the now-obsolete
  tests/test_opencode_required_verdict_gate.py that unit-tested it.
- Updated stale test fixtures/assertions across
  tests/test_opencode_workflow_shell_syntax.py,
  tests/test_pr_review_autofix_nvidia_nim_contract.py, and
  tests/test_required_workflow_queue_contract.py to match the
  post-merge workflow content (main independently migrated repository
  visibility checks from the `.private` boolean to the `.visibility`
  field, and the model-pool step to the contextual-orchestrator
  gateway loader, while this branch's tests still asserted the old
  shape).
- CHANGELOG.md / docs/org-required-workflow-rollout.md: union merge /
  newer date.
- Added a missing docstring on
  scripts/ci/validate_vcs_dependency_license.py's
  RejectRedirectHandler.redirect_request (pre-existing gap on this
  branch's own new file, caught by the 100% interrogate gate).

Full suite: 1916 passed, 1 skipped, 25 subtests; coverage 100% except
the pre-existing scripts/ci/pingora_edge_policy.py:274 gap already
tracked and owned by PR #1398 (also visible on unmodified main);
interrogate 100%.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX
seonghobae pushed a commit that referenced this pull request Aug 30, 2026
main advanced with the owner's own parallel investigation into the
same sidecar-preflight incident this branch was fixing, with far more
precise evidence than this branch's own analysis had (actual hosted-run
preflight/discovery artifacts, not just log-pattern reading). The real
root cause turned out to be contextual_orchestrator_review_policy.py's
family_cap selecting the same alphabetically-first candidates every
run -- 2 of which are permanently-retired NVIDIA model ids returning
HTTP 404 forever, not a transient failure -- plus a too-tight gateway
smoke-test timeout and a max_tokens/probe-budget desync. All three are
already fixed on main (family_cap 4->8, gateway timeout 30s->120s,
max_tokens 16->4096).

Conflict resolution:
- CHANGELOG.md / docs/product-technical-gap-baseline.md: kept both
  sides' entries; revised this branch's own "correction" entry to
  defer to main's much more thorough "sidecar-preflight outage:
  consolidated evidence" entry as the authoritative root cause and
  fix, rather than presenting this branch's own generic
  "two call sites, no retry" theory as confirmed. That mechanism is
  still real (kept, reframed as independent/complementary), just not
  what caused this specific incident.
- scripts/ci/contextual_orchestrator_review_sidecar.sh: kept main's new
  preflight-report console dump and this branch's
  SIDECAR_STDERR_TAIL_LINES threading -- complementary, not
  overlapping.

Deliberate follow-up simplification in the same commit: dropped this
branch's own _log_preflight_rejections (launcher.py), its sanitizer
allowlist entry, and its tests -- main's own
`log "sidecar preflight route evidence: ..."` fix already surfaces the
same already-bounded-safe preflight_report JSON to the console,
achieving the same operator-visibility goal more directly. Keeping
both would have been duplicate code solving the same problem twice.

Full suite: 1897 passed, 1 skipped, 21 subtests; coverage 100% on all
touched files (pre-existing pingora_edge_policy.py:274 gap, owned by
#1398, unaffected); interrogate 100%; bash -n clean.
…ead-lock-trust

# Conflicts:
#	CHANGELOG.md
#	scripts/ci/pingora_edge_policy.py
#	tests/test_pingora_edge_policy.py
#	tests/test_pr_review_autofix_nvidia_nim_contract.py

Copy link
Copy Markdown
Contributor Author

Resolved the stale merge conflict against current main and pushed a merge commit (a3f3c96b).

Conflicts, in ascending complexity:

  1. CHANGELOG.md — both sides independently appended distinct ## [Unreleased] bullets with no textual overlap (this PR's lock-trust entries vs. main's sidecar/Strix/ZDR entries). Kept both, this PR's entries first.

  2. tests/test_pr_review_autofix_nvidia_nim_contract.pyREVIEW_DISPATCH_BLOB_SHA pins the git blob SHA of .github/workflows/opencode-review-dispatch.yml; both sides had bumped it independently. Recomputed post-merge with git hash-object58fc032de5437b8206e0ee5ffa9953e3a538a09c (no other file references the stale values).

  3. scripts/ci/pingora_edge_policy.py + tests/test_pingora_edge_policy.py — the substantive one. Two independent, non-competing fixes touched the same functions:

    • This PR rewrote _load_changed_files's pagination from for page in range(1, 32) (whose trailing fallback raise was dead code, needing a # pragma: no cover + a static-arithmetic test to prove it unreachable) to an explicit while True with a directly reachable page == 31 terminal check, and added a MAX_CONTENT_REQUESTS/MAX_TOTAL_CONTENT_BYTES aggregate download budget with a max_bytes parameter on the content-loading function.
    • main (independently, via fix(sidecar): give the gateway preflight probe a real reasoning budget #1436-class work) added a .pdf documentation-citation exemption: ContentSizeExceededError, _is_binary_documentation_pdf, _pdf_evidence_confirms_binary, and split _load_file_content into _load_raw_file_bytes (the real fetch/validate logic) + a thin UTF-8-decoding wrapper.

    These aren't competing designs on the same problem — they're orthogonal features that both happened to touch _load_changed_files's tail and _load_file_content. I merged them by: keeping this PR's while True rewrite (strictly better than main's dead-code-plus-pragma approach — it makes the 31-page bound real, reachable, directly-tested code instead of an unreachable branch requiring a coverage exemption); layering this PR's max_bytes parameter onto main's new _load_raw_file_bytes (rather than the old single-function shape), with the aggregate-budget check placed after main's existing ContentSizeExceededError/malformed-response checks so the PDF exemption's specific-exception semantics are undisturbed; and forwarding max_bytes through the _load_file_content wrapper. main's now-obsolete test_changed_file_pagination_bound_is_provably_unreachable (which asserted the old for-loop's dead-code invariant via source regex) was removed since that code path no longer exists, and replaced with test_changed_file_pagination_stops_at_page_31_without_a_page_32_request, which directly asserts the new page == 31 check fires and the opener is never called for page 32 — keeping the same intent (pin that this boundary is live, tested code, not something needing a pragma) against the new implementation. The test file's other conflict was purely a git line-interleaving artifact between this PR's two new budget tests and main's four new PDF-exemption tests; reconstructed both sets in full against each side's pre-merge branch content (verified no test body was truncated or altered), renumbering the PDF tests' pull_request/head_sha fixture values (15–18) only to avoid confusion with the budget tests' reused 10/11.

    Flagging one thing I deliberately did not extend, for a human to weigh in on: the aggregate MAX_CONTENT_REQUESTS/MAX_TOTAL_CONTENT_BYTES budget in evaluate_pull_request only wraps the text-scanning _load_file_content call path; main's PDF-verification path (_pdf_evidence_confirms_binary_load_raw_file_bytes) still fetches unconditionally, outside that budget. Wiring it in looked straightforward but would add new, currently-untested branches against the repo's 100%-coverage gate, and is really a small scope extension beyond "merge two existing features" — flagging rather than doing it as part of a conflict resolution.

Test evidence (post-merge, in an isolated worktree):

  • PYTHONPATH=. python3 -m pytest tests/test_pingora_edge_policy.py tests/test_pr_review_autofix_nvidia_nim_contract.py -q → 88 passed
  • PYTHONPATH=. python3 -m pytest tests -q (full suite) → 1929 passed, 1 skipped, 21 subtests passed
  • coverage run -m pytest tests -q && coverage report --show-missing → 100% overall (10108/10108 statements, 4000/4000 branches); scripts/ci/pingora_edge_policy.py itself 100% (256/256, 96/96 branches)
  • interrogate (repo-configured) → PASSED, 100%
  • git diff --check origin/main...HEAD → clean
  • No .sh files are touched by this PR's own diff

Generated by Claude Code

devin-ai-integration[bot]

This comment was marked as resolved.

…dence budget

Devin Review confirmed: evaluate_pull_request applied MAX_CONTENT_REQUESTS /
MAX_TOTAL_CONTENT_BYTES only to the ordinary _load_file_content scanning
path. _pdf_evidence_confirms_binary made its own separate Contents API
request and could read up to MAX_FILE_BYTES (1 MiB) without consulting or
updating either aggregate counter, so a pull request with many patchless
documentation PDFs could exhaust the required check's GitHub API quota and
wall-clock budget despite the budget mechanism existing specifically to
prevent that.

_pdf_evidence_confirms_binary now takes the same max_bytes remaining-budget
parameter _load_file_content already uses, and returns the number of raw
bytes actually decoded (zero for the oversized-PDF exemption, since no
content is ever fetched in that case). A new _reserve_content_budget helper
centralizes the request/byte budget check so both the PDF-verification path
and the ordinary scan path share identical accounting and both fail closed
consistently once either budget is spent. The oversized-PDF exemption itself
is preserved -- only its resource cost now counts against the shared budget.

Added regression tests: many small patchless PDFs collectively exceeding
MAX_TOTAL_CONTENT_BYTES, a set of PDFs crossing MAX_CONTENT_REQUESTS, and
the oversized-PDF exemption itself consuming one request from the budget.
100% statement/branch coverage and 100% docstring coverage retained.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KPmJErfkcHer4UVEgrQxUX

Copy link
Copy Markdown
Contributor Author

Devin Review finding — verified and fixed

Re-checked the review-comment list against the exact current head before this push. Every earlier finding on this PR was already resolved by prior commits; the one remaining open thread was the PDF budget-bypass finding, confirmed real:

PDF verification bypasses evidence budgets — confirmed real, fixed

evaluate_pull_request applied MAX_CONTENT_REQUESTS / MAX_TOTAL_CONTENT_BYTES only to the ordinary _load_file_content scanning path. _pdf_evidence_confirms_binary made its own separate Contents API request ahead of that path and could read up to MAX_FILE_BYTES (1 MiB) without consulting or updating either aggregate counter — so a pull request containing many patchless documentation PDFs could exhaust the required check's GitHub API quota and wall-clock budget despite the budget mechanism existing specifically to prevent that.

Fix: _pdf_evidence_confirms_binary now takes the same max_bytes remaining-budget parameter _load_file_content already uses, and returns the number of raw bytes actually decoded (zero for the oversized-PDF exemption, since no content is fetched in that case). A new _reserve_content_budget helper centralizes the request/byte budget check so the PDF-verification path and the ordinary scan path share identical accounting, both failing closed once either budget is spent — consistent with the existing "Pingora limits reject partial scans" behavior (an exhausted budget still raises PolicyError, never a silent partial pass). The oversized-PDF exemption itself is preserved unchanged; only its resource cost now counts against the shared budget.

Regression tests added:

  • test_evaluate_pull_request_enforces_byte_budget_across_pdf_verification_reads — many small patchless PDFs whose combined decoded bytes exceed MAX_TOTAL_CONTENT_BYTES; asserts the check fails closed and a second PDF-verification request is never made once the budget is exhausted.
  • test_evaluate_pull_request_enforces_request_budget_across_pdf_verification_reads — three patchless PDFs against a MAX_CONTENT_REQUESTS of 2; asserts the check fails closed after exactly 2 Contents API reads.
  • test_evaluate_pull_request_charges_oversized_pdf_exemption_against_request_budget — confirms the oversized-PDF exemption path itself still consumes one request from the shared budget even though it reads zero bytes.

Validation

PYTHONPATH=. python3 -m pytest tests -q
# 1932 passed, 1 skipped, 21 subtests passed

coverage run -m pytest tests -q && coverage report --include=scripts/ci/pingora_edge_policy.py --show-missing
# scripts/ci/pingora_edge_policy.py: 263 stmts, 96 branches, 100% cover, 0 missing

interrogate -v scripts/ci/pingora_edge_policy.py
# RESULT: PASSED (minimum: 100.0%, actual: 100.0%)

_Generated by Claude Code


Generated by Claude Code

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 new potential issues.

Devin Review

Comment on lines +954 to +961
if changed_uv_paths:
locks, vcs_manifest = _base_python_inputs(
resolved_repo,
base_sha,
excluded_uv_paths=changed_uv_paths,
)
else:
locks, vcs_manifest = _base_python_inputs(resolved_repo, base_sha)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Changed uv projects skip stale exports

changed_uv_paths excludes changed and deleted projects before base export. A broken stale project cannot block its exact-head replacement.

Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Comment thread scripts/ci/pingora_edge_policy.py Outdated

Copy link
Copy Markdown
Contributor Author

opencode-review failure on 68a51d7e4 (job 99272270483) is the standard fail-closed wait-state, not a defect: No APPROVED or CHANGES_REQUESTED from opencode-agent on the current head — the async OpenCode dispatch simply hasn't posted a verdict for this exact head yet (it was pushed a few minutes prior, right after the PDF-budget fix above). This check is designed to fail until the dispatch catches up; no action needed here beyond waiting for the next dispatch cycle to post its verdict.


Generated by Claude Code

# Conflicts:
#	CHANGELOG.md
#	scripts/ci/pingora_edge_policy.py
#	tests/test_materialize_base_python_requirements.py
#	tests/test_pr_review_autofix_nvidia_nim_contract.py

Copy link
Copy Markdown
Contributor Author

Stale base resolved — one real "combine both" logic conflict in pingora_edge_policy.py, rest tractable

Branch was dirty against current main (base recorded here was 1d8e872). git fetch origin main && git merge --no-edit conflicted in CHANGELOG.md, scripts/ci/pingora_edge_policy.py, tests/test_materialize_base_python_requirements.py, and tests/test_pr_review_autofix_nvidia_nim_contract.py.

scripts/ci/pingora_edge_policy.py — real logic conflict, verified as a "both survive" case, not guessed. main independently renamed _is_binary_documentation_pdf/_pdf_evidence_confirms_binary_is_binary_documentation_asset/_binary_documentation_evidence_confirms and expanded binary-asset detection from PDF-only to PDF+PNG (with a full ~150-line structural PNG validator: chunk CRCs, IHDR/PLTE/IDAT/IEND ordering, Adam7-interlace unfiltering) + a generic BINARY_DOCUMENT_MAGIC dict for other formats. Meanwhile this PR independently added aggregate content-byte-budget tracking (_reserve_content_budget, MAX_CONTENT_REQUESTS = 256, a 16 MiB aggregate cap, max_bytes threaded through _load_raw_file_bytes/_load_file_content) to what was then the PDF-only verifier — confirmed as a real, still-live gap on main today: this PR's own RED evidence ("Pingora admitted unbounded aggregate content requests/bytes") and main's current evaluate_pull_request genuinely has no aggregate cap at all, only the per-file MAX_FILE_BYTES (1 MiB) ceiling — so a PR with many changed files could still force unbounded aggregate Contents-API reads on main as it stands. Neither side is stale relative to the other; both are real, independently-shipped, non-overlapping fixes. Combined them: kept main's multi-format detection/validation logic, added this PR's max_bytes: int parameter and tuple[bool, int] return (so the caller can charge the read against the shared budget) to the now-multi-format verifier, and reinstated the budget-tracking calls in evaluate_pull_request's loop using main's function names. Confirmed by diffing this PR's own isolated diff against its recorded base: it never touched the naming or PDF/PNG-dispatch logic at all — that's 100% main's independent work, just as the budget tracking is 100% this PR's.

Other conflicts, tractable:

  • CHANGELOG.md: purely additive, same pattern as sibling PRs this session.
  • tests/test_pr_review_autofix_nvidia_nim_contract.py: the now-familiar REVIEW_DISPATCH_BLOB_SHA pin — this PR also touches opencode-review-dispatch.yml, which auto-merged cleanly, so used a fresh git hash-object.
  • tests/test_materialize_base_python_requirements.py: this PR's own diff added 3 new tests; 2 (test_materialization_revalidates_includes_at_current_head, test_included_head_lock_requires_head_tree_paths) test the still-live materialize_base_python_requirements.py module directly and were kept. The 3rd, test_bounded_repair_driver_runs_against_a_staged_fixture, exercises a one-shot self-deleting repair driver (scripts/ci/repair_pr827_coderabbit_comments.py) that has already run and deleted itself from main (confirmed: the file no longer exists there) — kept it out rather than landing a test that would fail on a FileNotFoundError for a script that was always meant to be transient.

Net diff versus current main after the merge: 13 files, 1527 insertions / 135 deletions — matches this PR's own stated scope (1526/134, 13 files) almost exactly.

Verified before pushing (head confirmed unchanged at 68a51d7e immediately prior; targeted tests run before committing this time, per the process gap I found and disclosed on #1612 last round):

  • python3 -m py_compile scripts/ci/pingora_edge_policy.py — compiles
  • .github/workflows/opencode-review-dispatch.yml re-parses as valid YAML
  • python3 -m pytest tests/test_pingora_edge_policy.py tests/test_materialize_base_python_requirements.py tests/test_pr_review_autofix_nvidia_nim_contract.py -q172 passed
  • Full suite: coverage run -m pytest tests -q2796 passed, 1 skipped, 21 subtests (excluding the 2 pre-existing Python-3.11-vs-3.12+ sandbox-incompatible files, unrelated to this diff)
  • coverage report --include="scripts/ci/*"100% (12488 statements / 5066 branches)
  • interrogate --fail-under=100 scripts/ci100%
  • git diff --check — clean

Pushed non-force as 4aeac60d (two-parent merge commit, 68a51d7e..4aeac60d). Given the real synthesis in pingora_edge_policy.py, this is a good candidate for a careful look at that specific diff before treating it as routine — fresh exact-head checks and review are required regardless per this repo's governance model.


Generated by Claude Code

Two conflicts:

- CHANGELOG.md — kept both sides, this branch's entry above main's, matching
  the file's newest-first convention.
- REVIEW_DISPATCH_BLOB_SHA — the pin is `git hash-object` over
  `.github/workflows/opencode-review-dispatch.yml`; both sides' values are
  correct only for their own tree (branch 9aae7f9…, main ade10b3…) and both
  wrong for the merge. Recomputed against the merged file:
  c341291.

Verified after resolution: 2925 passed, 1 skipped, coverage 100%,
interrogate 100%; zero conflict markers; `ruff check --select F821` clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
#1932 changed `.github/workflows/opencode-review-dispatch.yml`, which
REVIEW_DISPATCH_BLOB_SHA pins by whole-file `git hash-object`. The workflow
auto-merged cleanly; only the pin line conflicted, and neither side's value is
right for the merge:

  this branch  c341291…  (its own tree)
  main         26e8555…  (after #1932)
  merged       cbd0c0c…  (recomputed)

Verified: 2926 passed, 1 skipped, coverage 100%, interrogate 100%; zero
conflict markers; `ruff check --select F821` clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@seonghobae seonghobae added bug Something isn't working priority: medium Normal-priority or P2 work type: bug Defect or incorrect behavior labels Sep 7, 2026 — with ChatGPT Codex Connector
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working priority: medium Normal-priority or P2 work type: bug Defect or incorrect behavior

Projects

Status: In Progress

Development

Successfully merging this pull request may close these issues.

2 participants