Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions changelog.d/added-inbox-recording-tab-audio.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
- Inbox ● Record can record a meeting tab's audio together with the microphone in Chrome and Edge on a computer: you pick the tab and tick "Share tab audio", the video track is dropped, the two are mixed in the browser (Web Audio) and both get a level meter. The consent notice then says remote participants are recorded too and must have been told. Where the browser cannot share tab audio (Safari, Firefox, phones) the option is hidden with the reason, and a share without an audio track warns and records the microphone only. The dashboard document's Permissions-Policy gains `display-capture=(self)`; `/terminal/` and `/api/` keep it closed.
9 changes: 6 additions & 3 deletions src/deploy/nginx.conf
Original file line number Diff line number Diff line change
Expand Up @@ -32,10 +32,13 @@ http {
# browser voice notes (Web Speech API; transcription is done by the browser
# vendor's speech service, audio never reaches Hive). The ttyd terminal
# iframe (/terminal/) never needs the microphone and keeps microphone=().
# The dashboard document also allows display capture for its own origin so Inbox ● Record
# can capture a meeting tab's audio (getDisplayMedia; the user picks the
# tab and ticks "share audio"); /terminal/ keeps display capture closed.
# /api/ responses are not documents and keep the fully closed policy.
map $uri $hive_document_permissions_policy {
default "geolocation=(), camera=(), microphone=(self)";
~^/terminal/ "geolocation=(), camera=(), microphone=()";
default "geolocation=(), camera=(), microphone=(self), display-capture=(self)";
~^/terminal/ "geolocation=(), camera=(), microphone=(), display-capture=()";
}

# Compress JSON API responses (70-90% reduction)
Expand Down Expand Up @@ -100,7 +103,7 @@ http {
# through browser header intersection (issues #3315, #3822, #3941).
add_header X-Content-Type-Options "nosniff" always;
add_header Referrer-Policy "strict-origin-when-cross-origin" always;
add_header Permissions-Policy "geolocation=(), camera=(), microphone=()" always;
add_header Permissions-Policy "geolocation=(), camera=(), microphone=(), display-capture=()" always;

proxy_intercept_errors on;
# Intercept only GATEWAY-origin failures: 502 (upstream
Expand Down
43 changes: 43 additions & 0 deletions src/docs/inbox-recording.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,49 @@ API answers them 403. Each member may have one recording in progress at a
time. An owner can abort anyone's recording; everyone else can only stop or
abort their own.

## Recording a meeting tab

Choose **Microphone and a meeting tab's audio** in the ● Record dialog to
record a video meeting running in another browser tab together with your own
voice. The browser asks which tab to share: pick the meeting tab and tick
**Share tab audio**. The hive keeps only the audio (the video track is
dropped at once) and mixes it with the microphone in the browser. The
recorder panel shows a level meter for each.

If the shared tab has no audio track (Share tab audio was not ticked, or a
window or screen was shared instead), the recorder warns and records the
microphone only. Stopping the share in the browser bar keeps the recording
going with the microphone. The dashboard's `Permissions-Policy` allows
display capture for its own origin only (`display-capture=(self)`); the
terminal and API responses keep it closed.

## Browser support

| Browser | Microphone | Microphone + tab audio | Format |
| --- | --- | --- | --- |
| Chrome, Edge, Brave (desktop) | yes | yes | WebM/Opus |
| Firefox (desktop) | yes | no (Firefox shares no tab audio) | WebM/Opus |
| Safari (macOS) | yes | no (Safari shares no tab audio) | MP4/AAC (fragmented) |
| Safari, Chrome (iPhone, iPad) | yes, while the screen stays on | no (no tab sharing) | MP4/AAC |
| Chrome (Android) | yes, while the page stays in front | no (no tab sharing) | WebM/Opus |

Where tab audio is not available the option is hidden and the dialog says
why. ● Record itself is hidden where the browser has no `MediaRecorder` or no
microphone access (the dashboard must be served over https).

### Phones and tablets

- **Keep the screen open.** iOS stops capture when the screen locks or
another app takes the microphone; the recorder then ends the recording with
what it has. The recorder asks for a Wake Lock so the screen stays on, and
warns when the page is hidden.
- Android stops capture when the browser goes to the background on many
devices; keep the dashboard in front.
- A network drop does not lose audio: chunks wait in the browser (IndexedDB)
and upload when the connection is back. If the page is reloaded or closed,
the queued chunks upload and the recording is finished on the next visit
(what had not been recorded yet is, of course, not there).

## Consent

The dashboard shows a consent notice before **every** recording, and a red
Expand Down
5 changes: 4 additions & 1 deletion src/docs/security-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -513,7 +513,10 @@ importance:
`geolocation=(), camera=(), microphone=(self)` (`src/deploy/nginx.conf`,
`map $uri $hive_document_permissions_policy`); the ttyd terminal iframe
(`/terminal/`) and `/api/` responses keep `microphone=()`, and no other origin
is ever granted the microphone.
is ever granted the microphone. The same document policy carries
`display-capture=(self)` for Inbox ● Record's tab audio
([inbox-recording.md](inbox-recording.md)); `/terminal/` and `/api/` keep
`display-capture=()`.

Audit: `session_created`, `session_started` (the outcome of the background
start), `session_closed`, `session_renamed`, `session_input` and
Expand Down
18 changes: 13 additions & 5 deletions src/pkg/config/gateway_nginx_dualstack_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package config
import (
"os"
"regexp"
"strings"
"testing"

"github.com/hivecommons/hive/internal/testutil"
Expand Down Expand Up @@ -216,12 +217,12 @@ func TestGatewayAuthRateLimiting(t *testing.T) {
}

// Permissions-Policy values the gateway emits. The dashboard document allows
// the microphone for its own origin only (Sessions voice notes use the
// browser's Web Speech API); the ttyd terminal iframe and /api/ responses keep
// the fully closed policy.
// the microphone (Sessions voice notes, Inbox ● Record) and display capture
// (● Record's meeting tab audio) for its own origin only; the ttyd terminal
// iframe and /api/ responses keep the fully closed policy.
const (
gatewayDocumentPermissionsPolicy = `geolocation=(), camera=(), microphone=(self)`
gatewayClosedPermissionsPolicy = `geolocation=(), camera=(), microphone=()`
gatewayDocumentPermissionsPolicy = `geolocation=(), camera=(), microphone=(self), display-capture=(self)`
gatewayClosedPermissionsPolicy = `geolocation=(), camera=(), microphone=(), display-capture=()`
)

// TestGatewayPermissionsPolicyMicrophoneScope pins the microphone scope of
Expand All @@ -245,6 +246,13 @@ func TestGatewayPermissionsPolicyMicrophoneScope(t *testing.T) {
if regexp.MustCompile(`microphone=\*|microphone=\(\s*\*`).MatchString(conf) {
t.Error("nginx.conf must never allow the microphone for every origin")
}
if regexp.MustCompile(`display-capture=\*|display-capture=\(\s*\*`).MatchString(conf) {
t.Error("nginx.conf must never allow display capture for every origin")
}
// display-capture=(self) appears only in the dashboard document default.
if n := strings.Count(conf, "display-capture=(self)"); n != 1 {
t.Errorf("display-capture=(self) must appear exactly once (the dashboard document), found %d", n)
}

apiBlock := regexp.MustCompile(`(?ms)^\s*location\s+/api/\s*\{.*?add_header\s+Permissions-Policy\s+"([^"]+)"\s+always;`).FindStringSubmatch(conf)
if apiBlock == nil || apiBlock[1] != gatewayClosedPermissionsPolicy {
Expand Down
100 changes: 99 additions & 1 deletion src/pkg/dashboard/inbox_recorder.test.js
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ const escapeHtml = s => String(s == null ? '' : s).replace(/&/g, '&').replac

const FNS = ['inboxRecSupport', 'inboxRecRoleCanRecord', 'inboxRecAvailable', 'inboxRecFmtClock', 'inboxRecLevel', 'inboxRecSplit',
'inboxRecConsentText', 'inboxRecDefaultTitle', 'inboxRecConsentHtml', 'inboxRecMemoryQueue', 'inboxRecUploader', 'inboxRecorderMachine',
'inboxRecResumePending', 'inboxRecUploadText', 'inboxRecMeterHtml', 'inboxRecPanelHtml'];
'inboxRecResumePending', 'inboxRecUploadText', 'inboxRecMeterHtml', 'inboxRecPanelHtml', 'inboxRecTabSupport', 'inboxRecSourcesHtml', 'inboxRecCaptureTab'];
const lib = new Function('escapeHtml', consts() + '\n' + FNS.map(fnSrc).join('\n') + '\nreturn {' + FNS.join(',') + ', CONSENT: INBOX_REC_CONSENT_VERSION, ACK: INBOX_REC_CONSENT_ACK };')(escapeHtml);

const flush = async (n = 20) => { for (let i = 0; i < n; i++) await new Promise(r => setImmediate(r)); };
Expand Down Expand Up @@ -299,3 +299,101 @@ test('a live recorder holds a lock and releases it when done', async () => {
assert.strictEqual(m.state, 'done');
assert.deepStrictEqual(held, []);
});

// ── Tab audio (mic + a meeting tab, Chromium only) ──
const CHROME_UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Safari/537.36';
const SAFARI_UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/19.1 Safari/605.1.15';
const FIREFOX_UA = 'Mozilla/5.0 (Macintosh; Intel Mac OS X 15.7; rv:145.0) Gecko/20100101 Firefox/145.0';
const ANDROID_UA = 'Mozilla/5.0 (Linux; Android 15) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/142.0.0.0 Mobile Safari/537.36';
const withDisplay = ua => ({ navigator: { userAgent: ua, mediaDevices: { getUserMedia() {}, getDisplayMedia() {} } } });

test('tab audio is offered on desktop Chromium only, and explained elsewhere', () => {
assert.deepStrictEqual(lib.inboxRecTabSupport(withDisplay(CHROME_UA)), { tab: true, reason: '' });
assert.strictEqual(lib.inboxRecTabSupport({ navigator: { userAgent: SAFARI_UA, userAgentData: undefined, mediaDevices: { getDisplayMedia() {} } } }).tab, false);
for (const ua of [SAFARI_UA, FIREFOX_UA, ANDROID_UA]) {
const s = lib.inboxRecTabSupport(withDisplay(ua));
assert.strictEqual(s.tab, false, ua);
assert.ok(s.reason.length > 0, 'a reason is given for ' + ua);
}
assert.match(lib.inboxRecTabSupport(withDisplay(FIREFOX_UA)).reason, /Chrome or Edge on a computer/);
assert.match(lib.inboxRecTabSupport({ navigator: { userAgent: CHROME_UA, mediaDevices: {} } }).reason, /cannot share a tab/);
const brands = { navigator: { userAgent: '', userAgentData: { brands: [{ brand: 'Microsoft Edge' }], mobile: false }, mediaDevices: { getDisplayMedia() {} } } };
assert.strictEqual(lib.inboxRecTabSupport(brands).tab, true);

const hidden = lib.inboxRecSourcesHtml({ tab: false, reason: 'needs Chrome' });
assert.doesNotMatch(hidden, /value="tab"/, 'no tab option where unsupported');
assert.match(hidden, /data-inbox-rec-tab-unsupported>needs Chrome</);
const shown = lib.inboxRecSourcesHtml({ tab: true, reason: '' });
assert.match(shown, /value="tab"/);
assert.match(shown, /value="mic" data-change-action="inboxRecSourceChange" checked/);
});

// fakeDisplay returns a getDisplayMedia stream with video and (optionally)
// audio tracks, recording what happened to them.
const fakeDisplay = (withAudio) => {
const video = Object.assign(fakeTrack(), { kind: 'video' });
const audio = Object.assign(fakeTrack(), { kind: 'audio' });
const tracks = withAudio ? [video, audio] : [video];
const stream = {
video, audio, asked: null,
getTracks: () => tracks.slice(), getVideoTracks: () => tracks.filter(t => t.kind === 'video'), getAudioTracks: () => tracks.filter(t => t.kind === 'audio'),
removeTrack: t => { const i = tracks.indexOf(t); if (i >= 0) tracks.splice(i, 1); },
};
return stream;
};
class FakeAudioContext {
constructor() { this.connected = []; FakeAudioContext.last = this; this.mixed = { mixed: true, getTracks: () => [] }; }
createMediaStreamDestination() { return { stream: this.mixed }; }
createMediaStreamSource(s) { const ctx = this; return { connect: d => ctx.connected.push([s, d]) }; }
close() { this.closed = true; }
}

test('mic + tab: the tab is requested with audio and video, video dropped, both mixed into the recorder', async () => {
const display = fakeDisplay(true);
let asked = null;
const mic = fakeStream();
let analysed = null;
const { m, hive } = machine({
navigator: { mediaDevices: { getUserMedia: async () => mic, getDisplayMedia: async (c) => { asked = c; return display; } } },
AudioContext: FakeAudioContext, captureTab: lib.inboxRecCaptureTab, analyse: (mm, captured) => { analysed = captured; return { mic: {}, tab: {} }; },
});
await m.start({ title: 'Call', consentVersion: lib.CONSENT, tab: true });
assert.strictEqual(m.state, 'recording');
assert.deepStrictEqual(asked, { audio: true, video: true });
assert.ok(display.video.stopped, 'the video track is stopped');
assert.deepStrictEqual(display.getVideoTracks(), [], 'and removed');
assert.strictEqual(FakeRecorder.last.stream, FakeAudioContext.last.mixed, 'the recorder records the mix');
assert.deepStrictEqual(FakeAudioContext.last.connected.map(c => c[0]), [mic, display], 'mic and tab both feed the mix');
assert.strictEqual(hive.start.includes_tab_audio, true);
assert.strictEqual(analysed.tab, display);
assert.match(lib.inboxRecPanelHtml(m), /data-inbox-rec-meter="mic"[\s\S]*data-inbox-rec-meter="tab"/, 'two level meters');
m.stop();
await flush();
assert.ok(display.audio.stopped, 'the tab capture is released on stop');
});

test('a shared tab without audio warns and records the microphone only', async () => {
const mic = fakeStream();
const { m, hive } = machine({
navigator: { mediaDevices: { getUserMedia: async () => mic, getDisplayMedia: async () => fakeDisplay(false) } },
AudioContext: FakeAudioContext, captureTab: lib.inboxRecCaptureTab,
});
await m.start({ title: 'Call', consentVersion: lib.CONSENT, tab: true });
assert.strictEqual(m.state, 'recording');
assert.match(m.warning, /no audio: share a browser tab and tick “Share tab audio”/);
assert.strictEqual(hive.start.includes_tab_audio, false);
assert.strictEqual(FakeRecorder.last.stream, mic);
assert.match(lib.inboxRecPanelHtml(m), /role="alert">The shared tab has no audio/);
});

test('cancelling the tab picker records the microphone only', async () => {
const mic = fakeStream();
const { m, hive } = machine({
navigator: { mediaDevices: { getUserMedia: async () => mic, getDisplayMedia: async () => { const e = new Error('cancel'); e.name = 'NotAllowedError'; throw e; } } },
AudioContext: FakeAudioContext, captureTab: lib.inboxRecCaptureTab,
});
await m.start({ title: 'Call', consentVersion: lib.CONSENT, tab: true });
assert.strictEqual(m.state, 'recording');
assert.match(m.warning, /No tab was shared/);
assert.strictEqual(hive.start.includes_tab_audio, false);
});
Loading
Loading