Skip to content

feat(export): emit schema-validated NewsDOM JSONL - #820

Draft
seonghobae wants to merge 7 commits into
developfrom
feature/add-export-jsonl-4506681705681529659
Draft

feat(export): emit schema-validated NewsDOM JSONL#820
seonghobae wants to merge 7 commits into
developfrom
feature/add-export-jsonl-4506681705681529659

Conversation

@seonghobae

@seonghobae seonghobae commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

NewsDOM JSON을 article/body-block 단위 JSONL로 내보내는 CLI를 추가합니다. 이 exporter의 입력 권위는 별도 ad-hoc dict 규칙이 아니라 기존 newsdom_api.schemas.ParseResponse입니다.

RED → causal repair

초기 head 00e9b784badedd398d176667bb1da40bd8029c50은 page/article가 dict가 아니면 조용히 건너뛰고 body_blocks가 list가 아니어도 iterable이면 그대로 순회했습니다. 따라서 canonical ParseResponse에 맞지 않는 입력을 성공으로 처리하면서 row를 누락하거나 문자열을 문자 단위 body block으로 내보낼 수 있었습니다.

RED 22631eef6e918b3ec53031a6f412899e016a92ff는 세 실제 schema 위반을 고정합니다: non-object page, non-object article, string-valued body_blocks. 세 경우 모두 export 전에 ValidationError가 발생하고 output file이 생성되지 않아야 합니다. 기존 구현은 이 계약을 만족할 수 없습니다.

Repair 165b0a0b1678fbb29732ccc15bc77bf9dd5c7e07은 JSON syntax parsing 뒤 ParseResponse.model_validate(data)를 호출하고 typed PageNode/ArticleNode만 순회합니다. 검증은 output file open보다 먼저 끝나므로 schema-invalid DOM은 부분/빈 JSONL을 성공 산출물처럼 남기지 않습니다. document_id, page_number, article_id, headline, ordered body_blocks의 의미도 canonical schema에서 직접 소비하며 중복 UL/validation contract를 만들지 않습니다.

Current exact state — 2026-09-07

  • protected/base: develop@e06b1f3fb10903569124af011da213951e6e2473
  • exact head: 165b0a0b1678fbb29732ccc15bc77bf9dd5c7e07
  • lifecycle: Draft / fresh exact-head gates pending
  • changed paths: tools/export_jsonl.py, tests/test_tools_export_jsonl.py, CHANGELOG.md

Valid articles preserve body-block order and emit one row per block; an article with no body block emits one metadata row with body_block_index=null and empty body_block_text, matching the feature's current documented row contract. Invalid JSON syntax retains the existing ValueError boundary; schema-invalid but syntactically valid JSON now fails closed through Pydantic validation.

Do not treat the original 100% coverage sentence as current-head evidence. The branch advanced after the generated claim. Before Ready/merge, the unchanged exact head must obtain applicable test/coverage, Security, SAST, CodeQL and governance GREEN, zero valid unresolved current-head findings, and then-live independent review. Exporting very large NewsDOM files still uses whole-file JSON parsing; if buyer workloads make that a material memory/latency gap, profile representative/right-cleared inputs before introducing a streaming parser or Rust hot path.

No self-approval, gate weakening, scanner suppression, force push, destructive rebase, mutable sibling contract, or source-neutral CI retrigger.

@google-labs-jules

Copy link
Copy Markdown

👋 Jules, reporting for duty! I'm here to lend a hand with this pull request.

When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down.

I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job!

For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with @jules. You can find this option in the Pull Request section of your global Jules UI settings. You can always switch back!

New to Jules? Learn more at jules.google/docs.


For security, I will only act on instructions from the user who triggered this task.

@coderabbitai

coderabbitai Bot commented Sep 6, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@seonghobae seonghobae changed the title ✨ 기능: JSONL 내보내기 도구 추가 feat(export): emit schema-validated NewsDOM JSONL Sep 6, 2026
@seonghobae
seonghobae marked this pull request as draft September 6, 2026 22:00
@seonghobae seonghobae added area: api API, protocol, event, or external contract enhancement New feature or request priority: medium Normal-priority or P2 work status: draft Draft pull request type: feature New or expanded product capability labels Sep 6, 2026 — with ChatGPT Codex Connector
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: api API, protocol, event, or external contract enhancement New feature or request priority: medium Normal-priority or P2 work status: draft Draft pull request type: feature New or expanded product capability

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant