fix(youtube): auto-detect caption language instead of hardcoding English - #1
Merged
Merged
Conversation
The yt-dlp provider requested `--sub-langs en.*` for every video, so a non-English channel was either skipped or, worse, distilled from YouTube's English auto-translation — losing the speaker's actual voice. Probe each video's tracks (`yt-dlp -J`) and pick the original language via a new pure, unit-tested `pickSubtitleLang()`: explicit override → original language (manual track first) → any human-authored track → the original auto-caption track. Machine-translated tracks (keyed `<target>-<source>`, e.g. `en-zh-TW`) are avoided. `MASK_YT_SUBLANGS` overrides with an explicit yt-dlp --sub-langs expression when a specific language is wanted. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The yt-dlp provider requested
--sub-langs en.*for every video. For a non-English channel this meant transcripts were either dropped, or — worse — distilled from YouTube's English auto-translation, losing the speaker's actual voice. (Found while distilling a Mandarin lecture channel: the mask came out in translated English instead of the speaker's Chinese.)Fix
Auto-detect the caption language instead of hardcoding it:
yt-dlp -J(subtitles,automatic_captions,language).pickSubtitleLang()chooses, in order:MASK_YT_SUBLANGS),language), human-authored track first,<target>-<source>, e.g.en-zh-TW) are avoided so a Chinese lecture isn't distilled from its English auto-translation. Whenlanguageis absent, the original is inferred as the source shared across the<target>-<source>translation keys.MASK_YT_SUBLANGSremains as an escape hatch: an explicit yt-dlp--sub-langsexpression (comma lists /zh.*wildcards) passed through verbatim.Design notes
parseVtt/isVideoUrlpattern — the live yt-dlp path stays thin and the injectable-provider contract is unchanged (existing fakes still work).yt-dlp -Jmetadata call per video (correctness over speed).Tests
pickSubtitleLangcases (original-with-manual, original-auto-over-translations, infer-without-language, English-original / lone-auto, override + null).bun test test/youtube.test.ts→ 10 pass.bun run typecheckandbun run buildgreen. Full suite unchanged except the new passing tests (any local failures are pre-existing Windows-only path issues; CI runs on Linux).🤖 Generated with Claude Code