Skip to content

fix(models): reject non-string BatchRequest fields - #104

Merged
seonghobae merged 13 commits into
mainfrom
fix/batch-request-type-boundary
Aug 11, 2026
Merged

seonghobae merged 13 commits into
mainfrom
fix/batch-request-type-boundary

Conversation

@seonghobae

@seonghobae seonghobae commented Aug 10, 2026 •

Copy link
Copy Markdown
Contributor

RCA

BatchRequest is a public package-root data class, but protected main relied on annotations rather than runtime validation. False-valued non-string prompts such as integer 0 could therefore cross TokenCounter.count_tokens() as zero-token text, while non-string model/system/id values could reach unrelated lower-layer failures. The same original validation repair initially copied rejected caller values into ValidationError.value, which could disclose prompt/model/id content through exception rendering.

A later representation audit found a second privacy boundary in this same value object: the generated dataclass repr() exposed user_prompt, system_prompt, and caller-selected id. Generic debugging/assertion/logging surfaces should not become package-owned copies of request content.

Bounded implementation

  • user_prompt, model, and id must be exact strings; system_prompt must be None or an exact string.
  • Empty strings remain accepted for compatibility; content policy remains host/provider owned.
  • Rejected values are represented only as fixed <redacted> validation evidence.
  • user_prompt, system_prompt, and id use dataclass repr=False; model remains visible for useful non-content diagnostics.
  • Constructor shape, direct attribute access, equality semantics, schema, persistence, provider transport, credentials, CLI, workflow, and release authority remain unchanged.
  • repr=False is deliberately not claimed as a serialization boundary: direct attribute access, vars()/__dict__, dataclasses.asdict(), pickling, and caller-defined serializers remain separate caller-controlled surfaces.

Test-first evidence

  • RED 39a37f3159ee618f05629c5548529494bc257a7d established exact runtime type failures.
  • A later RED on predecessor dd300b017e953593356a2fff9f262fc291b74e69 exposed rejected-value confidentiality leakage.
  • The current branch also carries a focused representation regression proving unique prompt/system/id sentinels are absent from repr() and str() while the model remains visible, plus docs/doctoring/batch-request-representation-confidentiality.md with Python 3.14 dataclass primary documentation in APA 7 form.

Predecessor checks are development provenance only.

Current exact staged state

  • Source head: c2be6651d7133459685ccb98cdf38e75ee624d98.
  • Independently resolved protected main: bf2cc2e140dc3ff4a56c3203f80f41bb9fed5d10.
  • GitHub reports Draft and mergeable.
  • CI 31466781421: completed / success.
  • Security Scan 31466781431: completed / success.
  • SAST Semgrep 31466781388: completed / success.
  • Formal reviews: none; unresolved inline review threads: zero at the fresh exact-head inspection.

These PR-triggered workflows still run under protected-main pre-#88 checkout semantics, so terminal success is staged integration evidence rather than final exact contributor-head acceptance.

Integration boundary

Keep Draft while repository exact-source governance #88 and its read-only central review-evidence prerequisite remain outside protected main. Do not churn this clean source merely to retrigger unchanged external infrastructure. After that prerequisite materially changes, reacquire every applicable exact-source CI/security/coverage/package/provenance/review/policy gate on the unchanged final source and merge only if live policy is genuinely satisfied.

Summary by CodeRabbit

  • 개선 사항

    • BatchRequest 생성 시 주요 필드의 타입을 엄격히 검증합니다.
    • 잘못된 타입이 입력되면 검증 오류를 반환합니다.
    • 객체를 출력하거나 문자열로 변환할 때 프롬프트와 식별자 등 민감한 정보가 표시되지 않습니다.
    • 모델 정보는 진단을 위해 계속 표시됩니다.
    • 기존 값 접근, 동등성 비교 및 빈 문자열 사용 방식은 유지됩니다.
  • 문서

    • 민감 정보 보호 범위와 호환성 관련 내용을 문서화했습니다.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@opencode-agent[bot], you've reached your PR review limit, so we couldn't start this review.

Next review available in: 33 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 13e0d771-35c8-4168-8fe6-d69178d3b000

📥 Commits

Reviewing files that changed from the base of the PR and between c35757f and 4458713.

📒 Files selected for processing (2)
  • docs/doctoring/batch-request-representation-confidentiality.md
  • tests/test_batch_request_type_boundary.py
📝 Walkthrough

Walkthrough

BatchRequest에 엄격한 타입 검증을 추가했습니다. user_prompt, system_prompt, id는 기본 표현에서 제외했습니다. 검증 오류와 객체 표현에 민감한 값이 포함되지 않는지 테스트와 문서로 확인했습니다.

Changes

BatchRequest 경계 동작

Layer / File(s) Summary
타입 검증 및 표현 계약
pg_llm_batch/models.py
user_prompt, model, id에는 정확한 문자열을 요구합니다. system_prompt에는 None 또는 정확한 문자열을 요구합니다. 프롬프트와 식별자는 repr() 및 기본 str()에서 제외합니다.
경계 동작 검증 및 문서화
tests/test_batch_request_type_boundary.py, docs/doctoring/batch-request-representation-confidentiality.md
비문자열 입력 거부, 오류의 민감값 비노출, 표현의 민감값 비노출, 빈 문자열 보존을 검증합니다. 표현 변경의 범위와 호환성 조건을 문서화합니다.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 28.57% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 제목은 BatchRequest 필드의 비문자열 값 거부라는 주요 변경을 명확하고 간결하게 설명합니다.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches 💡 1
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/batch-request-type-boundary

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown
⚠️ Action not completed

Review rate limited.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@docs/doctoring/batch-request-representation-confidentiality.md`:
- Around line 21-24: The Compatibility section’s statement that accepted values
are unchanged is inaccurate. Update the documentation to state that the
constructor signature is unchanged but runtime accepted values are narrowed:
user_prompt, model, and id require exact str values, while system_prompt accepts
only None or exact str; preserve the existing notes about equality and
interfaces.

In `@tests/test_batch_request_type_boundary.py`:
- Around line 10-29: Extend
test_batch_request_rejects_non_string_optional_and_identifier_fields and the
related BatchRequest validation tests to pass a custom str subclass for each of
user_prompt, model, system_prompt, and id, asserting ValidationError and the
corresponding field detail; ensure the tests require exact string types rather
than accepting isinstance(value, str).
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: bb0b545f-f4ce-4ad8-8bab-0ed722af2249

📥 Commits

Reviewing files that changed from the base of the PR and between 8963916 and c35757f.

📒 Files selected for processing (3)
  • docs/doctoring/batch-request-representation-confidentiality.md
  • pg_llm_batch/models.py
  • tests/test_batch_request_type_boundary.py

Comment thread docs/doctoring/batch-request-representation-confidentiality.md
Comment thread tests/test_batch_request_type_boundary.py
@seonghobae
seonghobae merged commit df75ea3 into main Aug 11, 2026
33 checks passed
@seonghobae
seonghobae deleted the fix/batch-request-type-boundary branch August 11, 2026 23:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant