fix(pingora): read Contents-API-oversized files through the Git Blobs API

[seonghobae]

Summary

required-workflow-bootstrap fails closed in about 5 s with Pingora edge policy could not establish complete evidence: GitHub content evidence for docs/sbom/inventory.json exceeds the size contract on #1678 (run 33989047645, and every push of that automation branch). The mechanism is deterministic and has nothing to do with Nginx:

• The SBOM inventory automation regenerates docs/sbom/inventory.json at 1,148,611 bytes (236 bytes on main).
• GitHub omits the diff patch for a file that large, so _needs_content_scan returns True on patch absence (.json is neither a documentation suffix nor a binary document format).
• The Contents API answers encoding: "none" with an accurate size for anything over its 1 MiB inline ceiling, and _load_raw_file_bytes turned that into ContentSizeExceededError, which only the documentation-PDF path knows how to absorb. Everything else fails the whole check closed.
• Scanned offline with main's own scan_content, the file carries zero Nginx runtime forms (it does not even contain the string nginx).

So the SBOM inventory could never pass the required policy, and the same would hold for any PR that adds a text file over 1 MiB.

Change

scripts/ci/pingora_edge_policy.py

• _resolve_oversized_content follows a Contents response that could not inline the file to its blob sha and fetches the same blob through the Git Blobs API (/repos/{repo}/git/blobs/{sha}, already inside _validate_github_api_url's origin and /repos/ path rule, same contents: read permission the Contents call needs).
• _load_oversized_blob_bytes binds every blob field back to the Contents metadata it was reached from: the response sha must be the one requested, its size must equal the declared size, and the decoded bytes must have exactly that length; any other shape is a malformed-evidence PolicyError that fails closed, exactly as for an inline response.
• New MAX_BLOB_BYTES = 11 MiB, chosen so a 60-column-wrapped base64 blob response plus its JSON envelope fits _github_open_json's existing 16 MiB MAX_RESPONSE_BYTES bound with margin; test_blob_ceiling_fits_the_bounded_response pins the arithmetic.
• ContentSizeExceededError keeps its narrow meaning for the remainder: a declared size over the blob ceiling, or a response with no well-formed blob sha to follow. The oversized-documentation-PDF suffix convention therefore now applies only there, and a .pdf within the blob ceiling is verified by its %PDF- magic bytes instead of trusted on its suffix (strictly stronger than before).

docs/policies/PINGORA_EDGE_POLICY.md and CHANGELOG.md describe the new bound.

Verification

• tests/test_pingora_edge_policy.py: 18 new tests — the chore: refresh org SBOM inventory #1678 shape end to end (oversized patchless docs/sbom/inventory.json, clean → (), with FROM nginx appended → nginx_container_image at line 2, proving it is scanned rather than exempted); an oversized real PDF passes on magic bytes and a textual file named .pdf is rejected; a PDF over MAX_BLOB_BYTES still uses the suffix convention with no blob request; the older base64-with-empty-content oversized shape takes the same route; five malformed/missing-sha shapes keep the size-contract signal without any blob request; seven malformed blob responses fail closed as plain PolicyError, never as ContentSizeExceededError; the ceiling arithmetic contract. Every pre-existing test is unchanged and passes.
• Full gate on db361daa8 with GITHUB_ACTIONS=true: 2921 passed, 1 skipped, coverage report --fail-under=100 → 100%, interrogate → 100%.
• Offline reproduction: scan_content("docs/sbom/inventory.json", <#1678 head content>) → 0 violations, 1,148,611 bytes.

Chicken-and-egg note: required-workflow-bootstrap runs the base branch's pingora_edge_policy.py at the immutable required-workflow SHA, so this PR's own contexts cannot exercise the change; #1678 (or any oversized-text PR) will, once this is on main.

Developer experience

Two small helpers with docstrings that state the binding rules and why the ceiling is 11 MiB; no new dependency, no change to the opener contract, no change to how any inline file is handled.

User experience

None visible to PR authors of ordinary files. Authors of a PR that adds or regenerates a text file over 1 MiB get a real content verdict instead of an unconditional "exceeds the size contract" failure; the SBOM inventory automation becomes mergeable on its merits.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BV96rXhqoR3tYZ9AeAVur4

---

Generated by Claude Code

[coderabbitai]

Warning

Review limit reached

Next included review available in 32 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Team

Run ID: b15ff7ef-5abb-4a9b-83e7-5d2f6ee7e84d

📥 Commits

Reviewing files that changed from the base of the PR and between 4302463 and 790ef33.

📒 Files selected for processing (4)

• CHANGELOG.md
• docs/policies/PINGORA_EDGE_POLICY.md
• scripts/ci/pingora_edge_policy.py
• tests/test_pingora_edge_policy.py

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[seonghobae]

CI status note on head db361daa8 — standing down on the two red checks.

CodeQL compatibility analysis (python) and (actions) (run 33996197171) ended with VERDICT_STATE=pending: the compat job's designed fail-fast after dispatching a CodeQL PR scan whose terminal verdict has not been published yet (catalog signature 9). Nothing in this diff is involved; the dispatch workflow re-runs these exact jobs once the verdict lands. Not re-running them manually — a re-run dispatches again and enlarges the queue that delays the verdict — and the dispatched scan also has to clear the OPENCODE_REPOSITORY_DISPATCH_ACTOR gate that #1929 tracks, which is an owner action.

Everything else on this head is green or queued: required-workflow-bootstrap, coverage source/evidence and the Pingora policy itself passed; strix, noema-review, and opencode-review are queued behind the organization runner ceiling.

---

Generated by Claude Code

[seonghobae]

CI note on head db361daa8: noema-review (run 33996197307, job 101393472189) failed in "Prepare Noema model verdict" with HTTP Error 502: Bad Gateway; caller attempts=1, duration=2539.2s, phase=response_error, served_model=deepseek-ai/deepseek-v4-flash-0731. Gateway-side; nothing in this diff (the Pingora policy's blob route) is on that path. strix passed on this head; the two CodeQL shards are the designed pending state noted above.

This run is the first of mine created after #1944, so it shipped noema-sidecar-evidence (artifact 9980320306), and the per-candidate timeline is no longer inference:

• Preflight: ready 5 / 12 — nvidia_nim deepseek-v4-pro, nvidia_nim_sub deepseek-v4-flash and v4-pro, openrouter cohere/north-mini-code and dots-3-note; rejected: primary-key v4-flash 529, the four gemma-3 404s, two OpenRouter gemma-4 429s.
• Review phase, 00:43–01:26Z: nvidia_nim_sub deepseek-v4-flash was attempted 29 times and answered none — 24 TimeoutError at the 90 s per-recv limit, 2 HTTP errors; provider_exhausted attempts=3 nine times. Its circuit opened twice (failures=3.0 threshold=3 reset_seconds=30.0, 01:00:08 and 01:19:40), was back in service within a minute each time (circuit_reset 01:04:58, 01:25:10), and the final passthrough attempt at 01:25:10 (attempt=1/1) went to the same route and timed out at 01:26:40 → 502. openrouter cohere answered ProviderResponseError (permanent) three times and was also re-admitted after its circuit reset. nvidia_nim deepseek-v4-pro was attempted 4 times with no failure line.
• Cost: about 36 of the 43 minutes were spent waiting on one route that never answered once.

So the ceiling on this class is not the pool alone: the breaker's 30 s reset is shorter than a single 90 s attempt, so a stalled route is re-admitted every cycle, and the candidate order keeps putting it first. That is contextual-orchestrator#1045's "request-scoped candidate exclusion" RED, now with the evidence it asked for (posted there). Standing down on this check; no re-run while the served set contains a route that stalls on every contact. The next base-merge push carries the current sidecar pin and re-runs it.

---

Generated by Claude Code

[seonghobae]

CI note on head db361daa8, third red check: opencode-review (run 33996197314, job 101401467187) failed in seconds at "Fail closed without a current-head OpenCode verdict" — catalog signature 1, not a review verdict; no opencode-agent review exists on this head. The dispatch step succeeded and created handler run 34004439058 (queued 01:39Z), but every app-token handler completed since 2026-09-05T17:00Z has been rejected in validate-pr-metadata because opencode-agent[bot] is not in OPENCODE_REPOSITORY_DISPATCH_ACTOR (latest rejection 01:30Z on #1916's handler). #1932's multi-identity parser is on main; the variable is an owner action tracked in #1929, where I have also noted that the one scheduler-dispatched handler that passed the gate tonight ended in a deterministic REQUEST_CHANGES on an exhausted model pool.

Nothing in this diff is involved, and no push or re-run from this side changes the outcome — a re-run re-dispatches into the same rejected gate, and the dispatch workflow re-runs this exact job by itself once a verdict is published. Standing down on this check. Current head state: strix passed, noema-review failed on the stalled-route loop noted above, CodeQL shards designed-pending.

---

Generated by Claude Code

[opencode-agent]

Pull request overview

OpenCode reviewed the current-head mergeability evidence and changed-file flow before approval, then found merge conflicts on the affected path.

Findings

1. HIGH Merge Conflict Guidance - Resolve the PR branch against the latest base branch

• Problem: GitHub reports mergeStateStatus DIRTY for this pull request.
• Root cause: Branch lane-jan/pingora-oversized-blob-evidence cannot be merged cleanly into main; the changed-file flow below shows which review/runtime path is blocked by the conflict.
• Fix: Merge or rebase the latest main into lane-jan/pingora-oversized-blob-evidence, resolve conflict markers in the PR branch, rerun the focused checks, and push the same branch.
• Repair commands:

gh pr checkout 1946 --repo ContextualWisdomLab/.github
git fetch origin main
git merge --no-ff origin/main  # or: git rebase origin/main
git status --short
# resolve files, then git add <resolved-files>
# merge path: git commit
# rebase path: git rebase --continue
git push origin HEAD:lane-jan/pingora-oversized-blob-evidence
# rebase path only: git push --force-with-lease origin HEAD:lane-jan/pingora-oversized-blob-evidence

• Regression test: Keep OpenCode approval gated on mergeability so model-output failures cannot approve a conflicted PR.

Merge Conflict Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Repository file: CHANGELOG.md"]
  S1 --> I1["repository behavior"]
  I1 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V1["required checks"]
  Evidence --> S2["Docs: PINGORA_EDGE_POLICY.md"]
  S2 --> I2["operator or user guidance"]
  I2 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V2["docs review"]
  Evidence --> S3["CI script: pingora_edge_policy.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_pingora_edge_policy.py"]
  S4 --> I4["regression suite"]
  I4 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V4["targeted test run"]

Loading

• Result: REQUEST_CHANGES
• Reason: mergeStateStatus is DIRTY; mergeable is CONFLICTING.
• Head SHA: db361daa819ef31cf99e5cc5431405218abe6f2a
• Workflow run: 33998627665
• Workflow attempt: 1

Changed-File Evidence Map

flowchart LR
  PR["PR changed files"] --> Evidence["OpenCode bounded evidence"]
  Evidence --> S1["Repository file: CHANGELOG.md"]
  S1 --> I1["repository behavior"]
  I1 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V1["required checks"]
  Evidence --> S2["Docs: PINGORA_EDGE_POLICY.md"]
  S2 --> I2["operator or user guidance"]
  I2 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V2["docs review"]
  Evidence --> S3["CI script: pingora_edge_policy.py"]
  S3 --> I3["review and security gate shell path"]
  I3 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V3["bash -n plus Strix self-test"]
  Evidence --> S4["Test: test_pingora_edge_policy.py"]
  S4 --> I4["regression suite"]
  I4 --> Conflict["Merge conflict blocks this path"]
  Conflict --> V4["targeted test run"]

Loading

[opencode-agent]

OpenCode Review Overview

• Head SHA: db361daa819ef31cf99e5cc5431405218abe6f2a
• Workflow run: 33998627665
• Workflow attempt: 1
• Gate result: REQUEST_CHANGES
• Coverage gate: success
• Model pool: exhausted
• Verdict: REQUEST_CHANGES
• Formal review: fix(pingora): read Contents-API-oversized files through the Git Blobs API #1946 (review)

[seonghobae]

CI note and push: head db361daa8 → cf76b40c7 (a base merge only; no change to the PR's own commits; 4 h 47 m since the previous push — earlier than the usual batching because it resolves a real conflict).

Merges main@efb892692, the owner's bypass-merged sidecar pin bump to contextual-orchestrator@414f2297 (contextual-orchestrator#1081, the fix for the _invoke retry-stacking loop that this head's own noema-sidecar-evidence artifact showed: one route attempted 29 times), plus #1943 / #1944 / #1945. That bump added a CHANGELOG.md entry at the same top position as this PR's, which is the DIRTY / CONFLICTING state OpenCode's REQUEST_CHANGES on db361daa8 reported; resolved by keeping both entries with main's newer one first, no other file touched.

Local gate on cf76b40c7 with GITHUB_ACTIONS=true: 2921 passed / 1 skipped, coverage 100%, interrogate 100%, git diff --check clean; the head is mergeable again. strix passed on the previous head; noema-review now runs on the fixed pin; opencode-review stays fail-closed until #1929's variable is set (the OpenCode verdict above was produced by a scheduler-path dispatch and will need a fresh one for this head); the CodeQL shards will read designed-pending again.

---

Generated by Claude Code

[seonghobae]

CI note on head cf76b40c7: noema-review (run 34008655765, job 101427555591) failed at "Provision contextual-orchestrator review sidecar" after 14 min 4 s (05:06:51–05:20:55Z) on the new pin 414f2297; "Prepare Noema model verdict" skipped. This one is a different shape from the zero-ready failures on my other heads, and it is the first post-co#1081 serving-phase sample:

• Route preflight passed with one ready route. Artifact noema-sidecar-evidence (9983259344): ready 1 / rejected 11 — nvidia_nim_sub deepseek-v4-pro answered the 16-token probe in 88 s; its nvidia_nim twin timed out at 90 s; flash 429 ×2, gemma-3 404 ×4, OpenRouter 429 ×4. "healthz and provider-route preflight confirmed after 182s".
• The gateway preflight then could not get one completion through that route. Three attempts (REVIEW_PREFLIGHT_GATEWAY_MAX_ATTEMPTS=3), each of which the orchestrator served with exactly two 90 s tries on the only route (05:11:54, 05:13:24 | 05:14:55, 05:16:25 | 05:17:55, 05:19:25 — attempt=1/1 lines, circuit failures=1.0 → 3.0, circuit_opened at 05:16:25 and re-admitted after its 30 s reset), each ending request_failed status=502 code=provider_connection_error; "gateway preflight returned HTTP 502 after 3 attempts". So 3 × 2 × 90 s = 9 minutes, which is the post-fix(opencode): system llvm for cargo-llvm-cov (v2 concurrency) #1081 arithmetic (1 + tool_retry_attempts tries per request, no transport-retry stacking underneath); on the old pin the same failure would have taken 3 × 6 × 90 s.

Not this PR's: a route that answers a 16-token probe in 88 s and never answers a real request is #1948's capacity condition, not anything in pingora_edge_policy.py. No re-run (the latest artifact shows exactly 1 ready route, the rule's boundary, and the route it names is the one that just failed six times); nothing to push. strix on this head provisioned successfully at 05:19Z (421 s) and is running; opencode-review is still queued. Their outcomes get one note each only if they differ from what is recorded here.

---

Generated by Claude Code

[seonghobae]

CI note on head cf76b40c7: strix (run 34008655751, job 101428234016) failed after 2 h 14 m (05:12:24–07:26:19Z; "Run Strix (quick)" 126 min) with STRIX_PROVIDER_UNAVAILABLE on the new pin 414f2297 — the third post-bump Strix scan on my heads to end in the same class as #1930 and #1916. Artifact strix-reports (9984885189):

• Route preflight 05:17Z: ready 4 / rejected 8 — both keys' deepseek-v4-flash and deepseek-v4-pro ready.
• Serving 05:23–07:26Z: Strix completed 32 requests (22 with usage; 1,236,576 input / 7,627 output tokens) while the gateway logged 65 TimeoutErrors and 65 × status=500 code=internal_error; nvidia_nim deepseek-v4-flash, the first-ranked route, was attempted 99 times. 0 of the 65 timeouts is followed by a circuit_failure record; the 13 that exist follow fast HTTP failures. 65 × 90 s is 97 of the 126 minutes.

Not this PR's (the Pingora policy change is not on this path at all); nothing to push; the sanctioned re-run is held for the reason given on #1930 and #1916 — the failure is deterministic while the first-ranked route stalls, and co#1082 is the fix. The three scans' consolidated numbers are on #1948.

---

Generated by Claude Code

[seonghobae]

CI note and push: head cf76b40c7 → 790ef33ea (a base merge only; no change to the PR's own commits; 7 h 14 m since the previous push).

Merges main@43024633e — #1947 (deferred transient routes), #1949 (lazy fill + per-account 429 skip), #1950 and #1953 — on top of the efb892692 pin bump this head already carried. Worth a fresh run rather than a wait, because the merged launcher changes both ends of this head's failure mode: a fully rate-limited hour now stops after 6 probes and fails provisioning in 126 s instead of 300–844 s (#1913 run 34021019274, artifact 9987026530), and a live hour reaches routes the old fill never did — contextual-orchestrator#1044's 08:26Z run preflighted ready 6 / probed 16 / skipped 4 / deferred 2, including both keys' llama-3.2-11b, and carried a review to a served completion.

The five red checks on the previous head were all attributed and none touched the Pingora policy this PR changes: strix (run 34008655751, noted 07:26Z) spent 126 minutes on the tool-bearing passthrough timeout class — 65 timeouts, 65 × 500 internal_error, 99 attempts on one first-ranked route, 0 circuit records (co#1082's scope); noema-review (run 34008655765) passed route preflight with one ready route and then lost the gateway preflight to 3 × 2 × 90 s of timeouts on that same route; opencode-review signature 1's fail-closed wait behind #1929; both CodeQL shards the designed dispatch-and-exit.

Gate on the pushed tree: 2945 passed, 1 skipped, 21 subtests; coverage 100 % (13,206 statements / 5,338 branches, 0 missed); interrogate 100 %; git diff --check clean. The oversized-blob tests and the 1,148,611-byte inventory check the PR is about are unaffected by the merge.

---

Generated by Claude Code

[seonghobae]

CI note on head 790ef33ea: three reds, all known classes, none touching the Pingora policy this PR changes.

• strix (run 34027492927, job 101477672594) failed at "Provision contextual-orchestrator Strix sidecar" after 5 min 44 s; "Run Strix (quick)" skipped. Preflight ready 0 / probed 6 / skipped 18 — the same rate-limited window that has held since about 09:50Z, with fix(sidecar): fill the preflight served set lazily to a readiness target #1949's per-account skip ending the walk after two 429s per account.
• Both CodeQL compatibility analysis shards (run 34027493446): the designed dispatch-and-exit blocked at opencode-review-dispatch rejects the current dispatcher identity: allowlist says github-actions[bot], sender is opencode-agent[bot] #1929's actor gate — already noted here.

Nothing to push, no re-run (0 ready routes). The before/after on this PR: the identical zero-ready outcome cost 421 s of provisioning plus 126 minutes of scanning at 05:12Z on the pre-#1949 launcher, and 344 s with no scan attempted now — the base merge did what it was for, and the remaining wait is the pool's, not the pipeline's.

---

Generated by Claude Code

[seonghobae]

Addition to my note an hour ago: this head has two more reds, and one of them is a different class from the zero-ready strix failure I described.

noema-review (run 34027492861, job 101477735981, 12:05–12:43Z) provisioned successfully in 572 s — so the pool was live for this job, unlike the strix job on the same head at 12:00Z — and then failed in "Prepare Noema model verdict" after 1691 s with HTTP Error 503: Service Unavailable; caller attempts=1, phase=response_error, served_model=deepseek-ai/deepseek-v4-flash-0731. A serving-phase exhaustion, not a preflight one; caller attempts=1 with the gateway owning repair and failover, as designed.

opencode-review (run 34027492878) is signature 1's 7-second fail-closed wait behind #1929's actor gate, already noted here.

That makes three distinct outcomes on this one head within 45 minutes — ready 0 at 12:00Z, a served-but-503 verdict attempt at 12:05Z, and the actor-gate wait — which is the sharpest evidence yet that the pool's capacity alternates on a minute scale rather than in hour-long blocks. Still nothing this PR's; nothing to push; no re-run.

---

Generated by Claude Code

[seonghobae]

Status note: the CHANGES_REQUESTED on this PR is already satisfied, and nothing is owed on it.

That review (opencode-agent[bot], 2026-09-06T03:12:56Z) was submitted against head db361daa81 and raised exactly one finding:

HIGH Merge Conflict Guidance — GitHub reports mergeStateStatus DIRTY … Merge or rebase the latest main into lane-jan/pingora-oversized-blob-evidence, resolve conflict markers, rerun the focused checks, and push the same branch.

That was done. The head is now 790ef33ea6 and GitHub reports mergeable: true with no conflict. The review is stale evidence against a condition that no longer exists — I am flagging it because a standing CHANGES_REQUESTED otherwise reads as unaddressed work, both to a human reader and to anything that gates on review state.

Why I am not pushing a base merge to refresh the checks. This PR is 14 commits behind main, and a refresh would give its noema-review and strix checks fresh runs that carry today's fixes — they currently hold pre-#1957 failures. I am deliberately not doing it: opencode-review on this head fails for the organization-wide authorization reason tracked at #1929 and #1927, and that does not clear on a re-run. So a refresh cannot make this PR mergeable; it would only spend runner slots from the pool whose saturation (#1531) is part of why everything here is slow. When the central variable is corrected, a refresh becomes worth doing and I will do it then.

For the same reason I am leaving #1930 untouched: it holds a live Noema APPROVED on its current head e13bd337e, and a base merge would discard that for no reachable gain.

Head unchanged at 790ef33ea6. Keeping the PR watched.

---

Generated by Claude Code