Skip to content

TSA-296: [BE] Update the lexicon when a page is verified - #339

Merged
yurii-tymoshevskyi merged 7 commits into
mainfrom
feat/TSA-296-update-lexicon-when-page-is-verified
Sep 24, 2026
Merged

yurii-tymoshevskyi merged 7 commits into
mainfrom
feat/TSA-296-update-lexicon-when-page-is-verified

Conversation

@arturobdev

@arturobdev arturobdev commented Sep 19, 2026 •

Copy link
Copy Markdown
Collaborator

Closes #296

Summary

  • Added lexicon entry repository and domain entities.
  • Added entity extraction from transcription text and structured output.
  • Updates lexicon entries on confirm and correct.
  • Uses corrected transcription values when available.
  • skip does not contribute to the lexicon.
  • Lexicon updates run inside the same transaction as page verification.
  • Populates lexiconAdded with the entries affected by the verification.
  • Respects preset.settings.minDistinctPages.

Testing

  • Confirming a page creates lexicon entries.
  • Repeated words across different pages increment freq and distinct_pages.
  • Entries become available in context after reaching minDistinctPages.
  • Correcting a page uses the corrected text / structured output instead of the original values.
  • Replaying the same correct action does not change lexicon counters.
  • Skipping a page does not modify the lexicon.
  • lexiconAdded contains the entries added or updated by the action.
  • buildContext() includes learned lexicon entries in later-page context.

Lexicon entries after verifying multiple pages:

Lexicon entries

lexiconAdded returned by the verify endpoint:

LexiconAdded return

Learned lexicon entries included in later-page context:

Learned lexicon

@arturobdev arturobdev added this to the transcripta-release-4 milestone Sep 19, 2026
@arturobdev arturobdev self-assigned this Sep 19, 2026
@arturobdev
arturobdev added this pull request to stack #340 September 19, 2026 02:25
@arturobdev arturobdev added backend Backend application feature New feature (auto-applied by the feature template) labels Sep 19, 2026
Comment thread apps/backend/src/modules/pages/page.service.ts Outdated
@arturobdev
arturobdev force-pushed the feat/TSA-296-update-lexicon-when-page-is-verified branch 3 times, most recently from 29663c6 to 73ebc54 Compare September 23, 2026 10:48
@arturobdev arturobdev moved this to In Review / Code Review in bsa-2026-transcripta Sep 23, 2026
@arturobdev
arturobdev force-pushed the feat/TSA-296-update-lexicon-when-page-is-verified branch 3 times, most recently from 52ddcdc to 6c2534d Compare September 23, 2026 14:06
@Yehor-Makarenko
Yehor-Makarenko removed this pull request from stack #340 September 23, 2026 15:11
Comment thread apps/backend/src/modules/lexicon/lexicon-extractor/lexicon-extractor.module.ts Outdated
Comment thread apps/backend/src/modules/lexicon/lexicon-extractor/lexicon-extractor.module.ts Outdated
@RomanNabukhotniidev

Copy link
Copy Markdown
Collaborator

god job!

@arturobdev
arturobdev changed the base branch from feat/TSA-140-sync-structured-output-with-corrected-text to main September 23, 2026 22:24
@arturobdev
arturobdev dismissed RomanNabukhotniidev’s stale review September 23, 2026 22:24

The base branch was changed.

@arturobdev
arturobdev force-pushed the feat/TSA-296-update-lexicon-when-page-is-verified branch 3 times, most recently from 4a7d941 to 282bd83 Compare September 24, 2026 14:27
@arturobdev
arturobdev force-pushed the feat/TSA-296-update-lexicon-when-page-is-verified branch from 282bd83 to c02ff15 Compare September 24, 2026 14:50

@yurii-tymoshevskyi yurii-tymoshevskyi left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@yurii-tymoshevskyi
yurii-tymoshevskyi merged commit 0a20880 into main Sep 24, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend Backend application feature New feature (auto-applied by the feature template)

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

[BE] Update the lexicon when a page is verified

3 participants