Skip to content

TSA-140: [BE] Keep structured output in sync with corrected text - #326

Merged
Yehor-Makarenko merged 42 commits into
mainfrom
feat/TSA-140-sync-structured-output-with-corrected-text
Sep 23, 2026
Merged

Yehor-Makarenko merged 42 commits into
mainfrom
feat/TSA-140-sync-structured-output-with-corrected-text

Conversation

@Yehor-Makarenko

Copy link
Copy Markdown
Collaborator

Closes #140

Description

Fixes an issue where human corrections in the verification UI were not propagated to the structured representation of a page. Previously, fixing plain text left edited_structured as null, allowing the original misread entities from structured to leak into the lexicon and poison the context for subsequent pages.

Key Changes

  • Re-derivation on Correction: When a page is verified with action: "correct", edited_structured is now re-derived from the corrected text via a secondary text-only model call using the preset's output_schema.
  • Schema Validation & Fallback: Re-derived JSON is validated against the preset's schema. Presets without an output_schema bypass re-derivation gracefully.
  • Fallback Resolution (edited ?? original): Updated queries and data readers across the codebase to prioritize edited fields via COALESCE / fallback logic (edited_text ?? text and edited_structured ?? structured).
  • Budget Tracking: Model calls for re-derivation are charged directly against the document budget.
  • Immutability: Preserved original text and structured outputs to maintain auditability of the model's initial reading.

Acceptance Criteria

  • Correcting a surname in the text puts the corrected form into the lexicon, not the original
  • The next pages receive the corrected form in their context
  • A page confirmed unchanged behaves exactly as before — no extra call, no extra cost
  • The re-derivation cost is charged against the document budget like any other model call
  • A preset without an output schema still verifies and exports normally
  • text and structured are never overwritten — the model's original reading stays available

Notes

  • Initially, I planned to implement the lexicon update directly within this scope, but upon investigation, it turned out to be covered by task [BE] Update the lexicon when a page is verified #296. This PR focuses on providing the re-derived edited_structured payload.

@Yehor-Makarenko Yehor-Makarenko added this to the transcripta-release-4 milestone Sep 17, 2026
@Yehor-Makarenko Yehor-Makarenko self-assigned this Sep 17, 2026
@Yehor-Makarenko Yehor-Makarenko added backend Backend application feature New feature (auto-applied by the feature template) labels Sep 17, 2026
Comment thread apps/backend/src/modules/pages/page.service.ts Outdated
Comment thread apps/backend/src/modules/pages/page.service.ts Outdated
Comment thread apps/backend/src/modules/jobs/rederive-structured/rederive-structured.ts Outdated
arturobdev
arturobdev previously approved these changes Sep 23, 2026
@Yehor-Makarenko
Yehor-Makarenko removed this pull request from stack #340 September 23, 2026 15:11
@Yehor-Makarenko
Yehor-Makarenko merged commit 9964508 into main Sep 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend Backend application feature New feature (auto-applied by the feature template)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[BE] Keep structured output in sync with corrected text

4 participants