Skip to content

engine: window one-shot Parakeet TDT transcription past encoder capacity - #817

Open
rshemet wants to merge 1 commit into
mainfrom
parakeet-longform-oneshot
Open

rshemet wants to merge 1 commit into
mainfrom
parakeet-longform-oneshot

Conversation

@rshemet

@rshemet rshemet commented Sep 24, 2026

Copy link
Copy Markdown
Collaborator

Problem

The one-shot cactus_transcribe path feeds the entire clip into a single fixed encoder window and silently drops every mel frame past capacity (~20-21 s): copy_frames = std::min(source_frames, expected_frames) in transcribe_parakeet_tdt. Long recordings come back with only the first ~20 s of text. Reported downstream as Pebble MOB-11009 (+14 duplicates) and MOB-14307 (reproducible on demand).

Fix

When the input exceeds the selected encoder window (and no caller stream state is present), slice the precomputed mel features into overlapping windows — 8 s left context, 1 s uncommitted look-ahead — and carry decoder state across them with a local ParakeetTdtStreamState, reusing the streaming path's word-boundary confirm/rollback scheme. Final window runs is_final=true. Audio that fits one window takes the literally unchanged code path.

  • No FFI change: no new exports, no signature changes, same weights bundles.
  • Vocab bias stays off for internal windows (one-shot never applied it), gated via a new internal flag on the state struct.
  • should_stop is honored at each window boundary and per-frame inside the decode loop.

Validation

Backtested via the transcribe CLI (public cactus_transcribe FFI) with the v2.2.0 parakeet-tdt-0.6b-v3-cq4 bundle, baseline captured pre-patch:

File Baseline Patched Time
Customer recording 63 s (MOB-14307) 76 words, truncated mid-sentence 157 words, complete 0.34 s -> 0.89 s
Two more 63 s customer clips ~76 words ~157 words ~0.9 s
test_long.wav 75.7 s 90 words 220 words 0.37 s -> 1.13 s
test.wav 20.03 s (exactly at capacity) 52 words byte-identical output noise
record.wav 4.5 s 22 words byte-identical output noise
  • Zero repeated 4-grams across seams; the sentence spanning the old 20 s cliff reads correctly.
  • 96.7% bag-of-words overlap with the independent streaming API on the 75.7 s fixture.
  • cactus_stop fired 150 ms into a 75 s file returns in ~270 ms with a partial result.
  • New test_stt case transcription_long_form guards the regression (fails on truncated output).

Follow-ups (not in this PR)

  • parakeet_flush in stream.cpp can build an over-capacity window on a long undecoded tail (same silent truncation).
  • stream.cpp never passes should_stop into transcribe_parakeet_tdt.
  • Whether TDT vocab bias should apply uniformly in one-shot.

The file/one-shot path fed the whole clip into one fixed encoder window and
silently dropped every mel frame past capacity (~20 s), truncating long
transcriptions. When the input exceeds the selected encoder window, slice the
precomputed mel features into overlapping windows (8 s left context, 1 s
uncommitted look-ahead) and carry decoder state across them with the streaming
path's word-boundary confirm/rollback scheme. Audio that fits one window takes
the unchanged code path, byte-identical output. Vocab bias stays off for the
internal windows to match one-shot behavior.

Backtested against real customer recordings (63 s) and test_long.wav (75.7 s):
full-duration transcripts, no seam duplication, 96.7% word overlap with the
streaming API, ~linear cost in window count, prompt cactus_stop cancellation.
Short fixtures byte-identical pre/post.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant