feat: Add video/podcast transcription to fetch phase#30
Open
StartupBros wants to merge 1 commit intoalexknowshtml:mainfrom
Open
feat: Add video/podcast transcription to fetch phase#30StartupBros wants to merge 1 commit intoalexknowshtml:mainfrom
StartupBros wants to merge 1 commit intoalexknowshtml:mainfrom
Conversation
Add tiered transcript extraction to the fetch phase: 1. yt-dlp captions (fast, no audio processing) 2. Whisper audio transcription (fallback when no captions) 3. Graceful placeholder (when tools not installed) Full transcripts are stored as separate files in .state/transcripts/ to keep the pending JSON small — a 215K char transcript takes 89 bytes in JSON. Processing reads only the first 20K chars needed for summarization. New exports: findYtDlp, findWhisper, parseJson3Transcript, parseVttTranscript, fetchTranscriptContent Also fixes podcast URL classification (was falling through to article type, fetching useless SPA HTML). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
StartupBros
force-pushed
the
feat/video-podcast-transcription
branch
from
84cc82e to
291d70f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add tiered transcript extraction to the fetch phase for video and podcast bookmarks:
status: needs_transcriptwhen tools aren't installedBoth yt-dlp and Whisper are optional — zero new required dependencies.
Architecture
Full transcripts are stored as separate files (
.state/transcripts/{id}.txt) instead of being inlined in the pending JSON. A 215K char transcript (1-hour talk) takes 89 bytes in the JSON. During processing, Claude reads only the first ~20K characters for summarization. Files are cleaned up after processing.Changes
src/processor.js— 6 new exports:findYtDlp,findWhisper,parseJson3Transcript,parseVttTranscript,fetchTranscriptContent, plus podcast URL classification fix and transcript file storagesrc/config.js— Config keys:ytdlpPath,whisperPath,whisperModel,transcribeTimeouts.claude/commands/process-bookmarks.md—transcribeaction reads transcript files, creates rich knowledge files with key takeawaysREADME.md— Transcription docs, install instructions, config referencetest/— 23 new tests, 2 new fixturesTesting
76 tests pass, 0 failures. Covers tool detection, subtitle parsing (json3 + VTT), transcript extraction, podcast URL classification, and config defaults/overrides. Integration verified with YouTube, Vimeo, and SoundCloud content.