Skip to content

✨ Inbox: drop audio and video files and transcribe them - #18476

Merged
kubestellar-prow[bot] merged 1 commit into
v5from
feat/inbox-upload-media
Nov 16, 2026
Merged

kubestellar-prow[bot] merged 1 commit into
v5from
feat/inbox-upload-media

Conversation

@clubanderson

Copy link
Copy Markdown
Member

Fifth and last PR of drag-and-drop ingest: audio and video files. Builds on the recording pipeline (#18408) and its local-file transcription (#18432).

What it does

  • Accepted: mp3, m4a, aac, wav, ogg, opus, webm (audio) and mp4, mov (video). Each extension must match the file's actual container signature (ID3 or MPEG frame sync, ADTS, ftyp, RIFF/WAVE, Ogg, EBML); a mismatch, or media without an extension, is refused. Audio/video links stay refused.
  • Upload: the file is streamed straight to connector-state/uploads/<id>/original (0700/0600), never held in memory, capped by the recording limit inbox.recording.max_bytes (200 MiB) and the uploads quota. The browser applies the same cap per file (settings.uploads.max_media_bytes).
  • Item: a pending upload item (upload_kind: audio|video) with a pending Knowledge fact (attr_upload_content: pending) that the machine transcript later fills, like a recording's. The Inbox shows its own "added without text" note.
  • Playback: the upload source implements AudioSource and AudioOpener, so ▶ Play streams the file from disk through the existing same-origin /api/inbox/upload/<id>/audio/stream handler (Range supported). Video keeps only its audio track: no video playback.
  • Transcription: with inbox.transcription enabled and consented, the transcription is queued as soon as the file is added (audited as inbox_transcribe, actor = who added it). It reads the file locally through AudioOpener (✨ Inbox: auto-transcribe hive recordings #18432), so with whisper the audio never leaves the hive. Otherwise 🎧 Transcribe shows the usual reason.
  • Retention: the original is deleted after inbox.uploads.media_retention_days, which now defaults to inbox.recording.retention_days (5 days); the transcript, item and fact stay.

Fix for the earlier upload PRs (#18411, #18418, #18425, #18426)

Upload requests now get a 20-minute read window (http.ResponseController.SetReadDeadline) instead of the dashboard server's 30-second ReadTimeout. Without it, a 25 MB document from the earlier PRs could be cut off mid-upload on a slow link, and a 200 MB recording could never arrive. Request processing is bounded at 25 minutes; text extraction keeps its own 30 s limit and link fetches their 60 s limit.

Tests

  • Go: container sniffing table (accepted, mismatched, missing extension, left to the text pipeline); streaming to the store with sizes, SHA-256 and 0600 perms; pending item and fact; local playback via OpenLocalAudio and the same-origin link; recording-cap and quota refusals that leave nothing behind; retention following the recording default with the item kept; transcription queued on add; HTTP upload of an mp3 played back through the stream handler, and a fake .mp3 refused with 415.
  • JS (node): media files use the recording cap while documents keep 25 MB; upload audio items show their pending note and keep the player.

No new dependencies.

🤖 Generated with Claude Code

Audio/video (mp3, m4a, aac, wav, ogg, opus, webm, mp4, mov) is checked by
container signature, streamed to the upload store under the recording
size cap and the uploads quota, played from disk through the generic
same-origin stream (AudioOpener) and auto-transcribed when transcription
is on. Originals follow the recordings retention; the transcript, item
and fact stay. Video keeps only its audio. Upload requests get a 20-min
read window.

Signed-off-by: Andy Anderson <andy@clubanderson.com>
@kubestellar-prow kubestellar-prow Bot added dco-signoff: yes Indicates the PR's author has signed the DCO. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Nov 16, 2026
@github-actions

github-actions Bot commented Nov 16, 2026 •

Copy link
Copy Markdown
Contributor

Open PR overlap check

Base branch: v5

Conflicts are checked with every PR merged onto the current tip of v5, so a PR that only needs a rebase is not reported as conflicting with its siblings.

PR Title Relation Files
#18471 ✨ inbox: mail-in files as email items with an owner approval queue (Betty C2 part 5) same files dashboard/openapi.json
src/docs/api-reference/knowledge.md
src/pkg/dashboard/static-src/js/inbox.js
src/pkg/dashboard/static/index.html
src/pkg/inbox/service.go

@clubanderson clubanderson added lgtm Indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Nov 16, 2026
@kubestellar-prow

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

Approval requirements bypassed by manually added approval.

This pull-request has been approved by:

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@hivecommons-hive

Copy link
Copy Markdown
Contributor

Hive status — updated in place; edits do not notify. Full history is in the Activity Monitor.

@clubanderson this PR is held at stage/071-needs-author and the hive will not merge it yet.

Failing checks:

  • mutation

Next: the hive does not push to your branch. Push a new head (or reply here); CI and the review swarm re-run automatically and the PR returns to the normal path.

@kubestellar-prow
kubestellar-prow Bot merged commit 8389f7f into v5 Nov 16, 2026
101 of 108 checks passed
@kubestellar-prow
kubestellar-prow Bot deleted the feat/inbox-upload-media branch November 16, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has signed the DCO. lgtm Indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant