Skip to content

fix(errors): snapshot structured evidence inputs - #105

Merged
seonghobae merged 14 commits into
mainfrom
fix/exception-detail-snapshot-integrity
Aug 12, 2026
Merged

seonghobae merged 14 commits into
mainfrom
fix/exception-detail-snapshot-integrity

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

RCA

PgLlmBatchError retained a caller-owned non-empty details mapping by reference, and GatewayError retained caller-owned response_data by reference in both the public attribute and structured details. A caller could therefore mutate structured exception evidence after construction, making later diagnostics depend on unrelated caller mutation.

Test-first remediation

  • RED source 0e42e2a54f0098ae9b8450268bfd013e5cffcc68 added focused alias-drift regressions. Superseded/cancelled workflow results remain development provenance, not passing evidence.
  • Production fix 4fe66f246199f3a2aeaab0e9bc739f24bcb0f647 snapshots only the outer caller-owned mappings at construction. Existing messages, codes, status fields, public mapping types, and direct mutation of the exception-owned mappings remain compatible.
  • Documentation RED f6b36971a573d4d2f1f45c8f3f31b8a88f016e4c required the assurance/standards record; 1032ae48fa8c398ac814b8dceb3c418d1587012e added feature doctoring with APA 7 references while explicitly rejecting immutability/compliance overclaims.
  • Current source 4f9358c55cf63506783251826e8edf181cb5e45b adds the absent-response_data branch regression so the new production branch is exercised under the repository's 100% statement/branch requirement.

The accepted contract is a constructor-time shallow snapshot, not an immutable, deep-copied, cryptographic, append-only, or durable audit record. Nested mutable values remain shared; deep copying arbitrary caller object graphs was rejected because it widens execution cost and compatibility semantics.

Canonical documentation PR #93 now classifies this exact assurance boundary in API Contract, TRD, Threat Model, Data Governance, Traceability, and Documentation Fitness without promoting #105 to protected-main behavior.

Current exact evidence

  • Source head: 4f9358c55cf63506783251826e8edf181cb5e45b.
  • Protected main: bf2cc2e140dc3ff4a56c3203f80f41bb9fed5d10.
  • GitHub reports Draft and mergeable.
  • CI 31382903146: completed / success, including Python 3.10/3.12/3.14, exact coverage/docstrings/lint/package, lock/package, Compose, and container jobs.
  • SAST Semgrep 31382902855: completed / success.
  • Security Scan 31382902869: completed / success.
  • Formal reviews: none. Unresolved inline review threads: zero.

These runs still precede protected-main exact-source governance #88, so they are staged integration evidence rather than final exact-source acceptance. Queued/pending/predecessor/cancelled/status-only/synthetic evidence is not final acceptance.

Scope and integration boundary

Changed files are limited to pg_llm_batch/exceptions.py, two focused test modules, and feature doctoring. No database schema, provider transport/parsing, credential, CLI, workflow, release, or persistence authority changes.

Keep Draft while repository exact-source governance #88 plus its read-only central .github#790 coverage/review prerequisite remain outside protected main. Do not churn this clean source solely to retrigger an unchanged external reviewer path. After those dependencies materially change, reacquire applicable exact-source CI/security/package/provenance/review/policy evidence on the unchanged final source before promotion or merge.

Summary by CodeRabbit

  • 버그 수정

    • 오류 생성 후 원본 상세 정보나 응답 데이터가 변경되어도 오류에 기록된 정보가 생성 당시 상태로 유지됩니다.
    • 응답 데이터가 없는 게이트웨이 오류도 상태 코드와 세부 정보가 일관되게 처리됩니다.
  • 문서

    • 구조화된 오류 정보의 스냅샷 동작, 제한 사항, 보안 관련 고려 사항 및 호환성 범위를 문서화했습니다.
  • 테스트

    • 오류 정보 보존과 관련 문서 내용을 검증하는 회귀 테스트를 추가했습니다.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@seonghobae, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 48 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e67ea385-2d87-446b-bfd5-c1b7e0112349

📥 Commits

Reviewing files that changed from the base of the PR and between 5ba4d27 and 972f930.

📒 Files selected for processing (3)
  • docs/doctoring/structured-error-evidence-integrity.md
  • tests/test_exception_evidence_documentation.py
  • tests/test_exceptions.py
📝 Walkthrough

Walkthrough

예외 클래스가 생성 시점에 입력 매핑을 얕게 복사합니다. 외부 매핑 변경 후에도 예외 데이터가 유지되는지 검증합니다. 스냅샷 계약과 한계를 문서화하고 문서 참조를 테스트합니다.

Changes

구조화된 예외 증거 무결성

Layer / File(s) Summary
스냅샷 계약 문서화
docs/doctoring/structured-error-evidence-integrity.md, tests/test_exception_evidence_documentation.py
예외 매핑의 생성 시점 스냅샷 계약, 얕은 복사의 한계, 관련 표준 및 릴리스 참조를 문서화합니다. 필수 문서 내용과 경로를 테스트합니다.
예외 매핑 스냅샷 적용
pg_llm_batch/exceptions.py, tests/test_exceptions.py
PgLlmBatchError.details와 GatewayError.response_data가 입력 매핑의 복사본을 저장하도록 변경합니다. 외부 매핑 변경과 응답 데이터가 없는 경우의 기본 상태를 검증합니다.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 구조화된 오류 증거 입력을 스냅샷하는 핵심 변경을 정확하고 간결하게 설명합니다.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/exception-detail-snapshot-integrity

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@seonghobae
seonghobae marked this pull request as ready for review August 11, 2026 22:15

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
docs/doctoring/structured-error-evidence-integrity.md (1)

31-33: 🔒 Security & Privacy | 🔵 Trivial | ⚡ Quick win

정확한 1차 출처와 확인 날짜를 문서와 테스트에서 함께 고정하세요.

문서의 시점 의존적인 표준 상태 주장과 테스트의 독립적인 substring 검사가 같은 추적성 공백을 만듭니다.

  • docs/doctoring/structured-error-evidence-integrity.md#L31-L33: 각 현재성 주장에 공식 릴리스 페이지와 확인 날짜를 추가하세요.
  • docs/doctoring/structured-error-evidence-integrity.md#L39-L45: SP 800-53 5.2.0과 SSDF 1.2 초안의 정확한 공식 출처를 참고문헌에 추가하세요.
  • tests/test_exception_evidence_documentation.py#L15-L27: 전체 버전 문구, 공식 출처 URL, 확인 날짜를 검증하도록 assertions를 강화하세요.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/doctoring/structured-error-evidence-integrity.md` around lines 31 - 33,
문서의 현재성 표준 주장마다 공식 릴리스 페이지 URL과 확인 날짜를 명시하고, 참고문헌에 NIST SP 800-53 Release 5.2.0
및 SSDF Version 1.2 초안의 정확한 공식 출처를 추가하세요.
docs/doctoring/structured-error-evidence-integrity.md 31-33에서는 각 주장에 출처와 날짜를
연결하고, 39-45에서는 해당 참고문헌을 보강하세요. tests/test_exception_evidence_documentation.py
15-27의 검증을 전체 버전 문구, 공식 출처 URL, 확인 날짜까지 확인하도록 강화하세요.

Source: MCP tools

tests/test_exceptions.py (1)

22-42: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

얕은 복사 경계를 회귀 테스트로 고정하세요.

현재 테스트는 호출자 매핑의 최상위 변경만 검증합니다. 문서는 중첩 mutable 값이 공유되고 예외 소유 매핑의 직접 변경이 호환된다고 명시합니다.

중첩 값 하나의 변경과 error.details, gateway.response_data의 직접 변경을 검증하세요. 그렇지 않으면 이후에 deep copy 또는 참조 연결이 잘못 변경되어도 현재 테스트가 통과할 수 있습니다.

추가 검증 예시
     assert gateway.response_data is gateway.details["response_data"]
+
+    nested_response = {"attempts": []}
+    nested_gateway = GatewayError(
+        "nested", 503, {"nested": nested_response, "retry": True}
+    )
+    nested_response["attempts"].append("late")
+    assert nested_gateway.response_data is not None
+    assert nested_gateway.response_data["nested"]["attempts"] == ["late"]
+    nested_gateway.response_data["retry"] = False
+    assert nested_gateway.details["response_data"]["retry"] is False
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@tests/test_exceptions.py` around lines 22 - 42, 보호 복사가 얕은 복사임을 검증하도록
test_error_detail_mappings_snapshot_constructor_inputs를 확장하세요. PgLlmBatchError의
중첩 mutable 값 변경이 error.details에 반영되는지 확인하고, error.details를 직접 변경한 결과도 검증하세요.
GatewayError에 대해서도 중첩 값 공유와 gateway.response_data 직접 변경이
gateway.details["response_data"]에 반영되는지 확인해 deep copy나 참조 단절을 감지하도록 하세요.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/doctoring/structured-error-evidence-integrity.md`:
- Around line 31-33: Update the references in the document’s standards-status
discussion to add official NIST release pages for SP 800-53 Release 5.2.0 and
SSDF publications or SP 800-218 Revision 1, including the date each status was
verified. Keep the “current” and “still draft” wording only when supported by
those citations; otherwise remove the time-dependent claims.

---

Nitpick comments:
In `@docs/doctoring/structured-error-evidence-integrity.md`:
- Around line 31-33: 문서의 현재성 표준 주장마다 공식 릴리스 페이지 URL과 확인 날짜를 명시하고, 참고문헌에 NIST SP
800-53 Release 5.2.0 및 SSDF Version 1.2 초안의 정확한 공식 출처를 추가하세요.
docs/doctoring/structured-error-evidence-integrity.md 31-33에서는 각 주장에 출처와 날짜를
연결하고, 39-45에서는 해당 참고문헌을 보강하세요. tests/test_exception_evidence_documentation.py
15-27의 검증을 전체 버전 문구, 공식 출처 URL, 확인 날짜까지 확인하도록 강화하세요.

In `@tests/test_exceptions.py`:
- Around line 22-42: 보호 복사가 얕은 복사임을 검증하도록
test_error_detail_mappings_snapshot_constructor_inputs를 확장하세요. PgLlmBatchError의
중첩 mutable 값 변경이 error.details에 반영되는지 확인하고, error.details를 직접 변경한 결과도 검증하세요.
GatewayError에 대해서도 중첩 값 공유와 gateway.response_data 직접 변경이
gateway.details["response_data"]에 반영되는지 확인해 deep copy나 참조 단절을 감지하도록 하세요.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 640f865c-bc12-45f9-8dc4-5671519a6a37

📥 Commits

Reviewing files that changed from the base of the PR and between 602ec07 and 5ba4d27.

📒 Files selected for processing (4)
  • docs/doctoring/structured-error-evidence-integrity.md
  • pg_llm_batch/exceptions.py
  • tests/test_exception_evidence_documentation.py
  • tests/test_exceptions.py

Comment thread docs/doctoring/structured-error-evidence-integrity.md Outdated
@seonghobae
seonghobae merged commit 2833e5f into main Aug 12, 2026
33 checks passed
@seonghobae
seonghobae deleted the fix/exception-detail-snapshot-integrity branch August 12, 2026 00:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant