diff --git a/docs/releases/2.0.2-Beta.1-release-notes.md b/docs/releases/2.0.2-Beta.1-release-notes.md new file mode 100644 index 000000000..25985780d --- /dev/null +++ b/docs/releases/2.0.2-Beta.1-release-notes.md @@ -0,0 +1,62 @@ +# OpenLess 2.0.2-Beta.1+build.20261017 + +本次为 **Beta 测试版**，不替换当前正式版 2.0.1。比较范围：上一公开 Beta **`v2.0.0-Beta.7-tauri` → `v2.0.2-Beta.1-tauri`**，包含期间正式版的打包改进及本轮胶囊、本地识别更新。 + +## 功能更新 + +- **多屏胶囊可自行选择**：在「设置 → 录音与输入 → 多屏胶囊」选择「鼠标所在屏幕」或「所有屏幕显示」。支持 macOS、Windows，默认跟随鼠标；设置会保存，恢复配置后也会应用。所有屏幕模式使用额外窗口与图形资源，只有显式选择后才创建镜像。 + - PR：[ #1222](https://github.com/Open-Less/openless/pull/1222)；提交者与代码署名：[@appergb](https://github.com/appergb)。 + +## Bug 修复 + +- **模型加载时点停止迟迟不响应**：本地模型的加载与麦克风状态解耦，麦克风就绪即可录音；点击停止先释放麦克风并进入转写，不再等模型加载完才处理停止。取消后不回放已取消音频，尚未执行的加载不再启动；加载失败会明确反馈，不伪造识别成功。 + - PR：[ #1222](https://github.com/Open-Less/openless/pull/1222)；提交者与代码署名：[@appergb](https://github.com/appergb)。 +- **本地语音资源占用突增**：限制 TextStream 的 MLX 空闲缓存和内存工作集指导值；SenseVoice 固定单 CPU 推理线程，避免忙闲预算变化触发整模型重载和双份驻留。本地推理仍会占用模型所需内存，CPU 预算为可协作的平均约束，不是瞬时硬限制。 + - PR：[ #1222](https://github.com/Open-Less/openless/pull/1222)；提交者与代码署名：[@appergb](https://github.com/appergb)。 +- **多屏显示与切换**：默认不再在每个屏幕重复显示胶囊；选择所有屏幕后仍只使用同一识别会话。镜像不显示学习建议或落字失败卡片；卡片始终由主胶囊处理。切回单屏、隐藏胶囊或停止后，镜像隐藏并停止动画，不再接收实时胶囊帧。 + - PR：[ #1222](https://github.com/Open-Less/openless/pull/1222)；提交者与代码署名：[@appergb](https://github.com/appergb)。 + +## 架构改动 + +- **本地加载与校验生命周期**：通过宿主任务调度异步加载实验模型；TextStream 对文件身份和元数据未变的已校验权重使用进程内缓存。首次使用、文件变化或新进程仍校验完整 SHA-256，未知 worker 不会被自动覆盖。 + - PR：[ #1222](https://github.com/Open-Less/openless/pull/1222)；提交者与代码署名：[@appergb](https://github.com/appergb)。 +- **有界镜像与配置契约**：两种模式使用同一原生显示策略，处理负坐标与混合 DPI；macOS 镜像采用与主胶囊一致的非激活面板。镜像窗口有限复用，避免每次录音反复创建 WebView；新配置具有旧版默认值、序列化及同步注册校验。 + - PR：[ #1222](https://github.com/Open-Less/openless/pull/1222)；提交者与代码署名：[@appergb](https://github.com/appergb)。 + +## 构建与分发改动 + +- **新 Beta 版本与回归门禁**：应用版本统一为 `2.0.2-Beta.1+build.20261017`，使用新标签，不移动旧 Beta7 或正式版标签。增加两种模式保存/加载、无效值拒绝、模式切换缓存失效，以及 macOS/Windows 设置页的 Chromium/WebKit 回归。 + - PR：[ #1222](https://github.com/Open-Less/openless/pull/1222)；提交者与代码署名：[@appergb](https://github.com/appergb)。 +- **继承已公开正式版的 Windows 打包与安装验证改进**：MSI 数字版本兼容、CI Restart Manager 隔离、成功返回值 3010 处理，以及独立固定版本的安装验证工具。Beta 提供 NSIS，不提供 MSI。 + - PR：[#1206](https://github.com/Open-Less/openless/pull/1206)、[#1213](https://github.com/Open-Less/openless/pull/1213)、[#1215](https://github.com/Open-Less/openless/pull/1215)；提交者与代码署名：[@appergb](https://github.com/appergb)。这些 PR 已合入 beta，不是本轮新写的应用功能。 + +## PR 与参与者 + +| PR | 内容 | PR 提交者 | 代码署名 | +| --- | --- | --- | --- | +| [#1222](https://github.com/Open-Less/openless/pull/1222) | 多屏模式、本地识别修复、回归与本 Beta 版本准备 | [@appergb](https://github.com/appergb) | [@appergb](https://github.com/appergb) | +| [#1206](https://github.com/Open-Less/openless/pull/1206) | 继承的日期 MSI 与定向草稿打包 | [@appergb](https://github.com/appergb) | [@appergb](https://github.com/appergb) | +| [#1213](https://github.com/Open-Less/openless/pull/1213) | 继承的 Windows 安装验证环境隔离 | [@appergb](https://github.com/appergb) | [@appergb](https://github.com/appergb) | +| [#1215](https://github.com/Open-Less/openless/pull/1215) | 继承的 MSI 3010 成功码与验证工具固定 | [@appergb](https://github.com/appergb) | [@appergb](https://github.com/appergb) | + +本轮问题反馈与双模式需求来自用户实际使用反馈；未擅自公开反馈者身份。源码复查及验证由提交工作流完成，GitHub 审查、讨论、合并的具体记录以各 PR 页面为准，不将提交者自查冒称独立批准。继承的三个打包 PR 已由 @appergb 合并；本轮 #1222 的合并者和最终检查在公开前核对。 + +## 验证说明 + +- 本地 Core 1211 项通过，新增偏好回归 3/3；macOS 原生 583 项通过，5 项需外部模型/安装条件的测试明确跳过；浏览器 Chromium/WebKit 完整 116/116。 +- 同一 ASR 源码此前以已有模型及合成语音验证：TextStream 30.46 秒样本转写相同，worker 峰值 RSS 从约 1674 MiB 降至 1341 MiB；此样本没有证明 CPU 瞬时峰值下降。SenseVoice 单线程会牺牲部分解码速度，以降低争用并避免反复重载。 +- 默认单屏、显式多屏、鼠标屏幕选择、DPI、配置保存和模式切换均有回归。自动化检查不等于所有设备和真双屏交互均已验收；模型与宿主仍可能有短时资源突发，欢迎通过本 Beta 反馈实际表现。 +- 包校验和及更新签名在公开前核对，附 `SHA256SUMS`。详细定位和测试边界见[修复记录](https://github.com/Open-Less/openless/blob/beta/docs/reviews/2026-10-16/local-asr-capsule-performance.md)。 + +## 下载与升级 + +- **macOS**：macOS 14 Sonoma 或更新版本；Apple Silicon 选 `aarch64.dmg`，Intel 选 `x64.dmg`。当前包使用 ad-hoc 签名、未做 Apple 公证；拖入 Applications 后，按现有分发方式执行： + + ```sh + xattr -cr /Applications/OpenLess.app + ``` + +- **Windows**：x64 NSIS 安装包，未做发布者证书签名。 +- **Android**：四种 ABI 的签名 APK；多屏胶囊选项仅用于桌面 macOS/Windows。 +- **Linux**：本次不附带 Linux 安装包。 +- Beta 不推送给 Stable 更新源；[当前正式版 2.0.1](https://github.com/Open-Less/openless/releases/tag/v2.0.1-tauri)保持不变。 diff --git a/docs/reviews/2026-10-16/local-asr-capsule-performance.md b/docs/reviews/2026-10-16/local-asr-capsule-performance.md new file mode 100644 index 000000000..cc00f4d64 --- /dev/null +++ b/docs/reviews/2026-10-16/local-asr-capsule-performance.md @@ -0,0 +1,63 @@ +# 本地识别资源占用与多屏胶囊 + +## 范围 + +基线 beta `96278f40`。处理更新后多屏同时显示胶囊、macOS 本地识别启动/停止等待和资源突增；不更改用户模型开关、凭据、录音、权限或已安装 App，不改已公开版本及标签。当前机器已有 SenseVoice 与 TextStream 权重，Gemma 不具备本次真实模型测试条件。 + +## 原因与修正 + +### 停止响应 + +普通听写在麦克风启动后仍 await `buffered.attach()`，本地模型加载/校验未完成时，Core 公开状态仍为 Starting。`stop_dictation_session_with_options` 会等待该状态结束，导致用户已点停止但麦克风不能及时释放、思考状态也迟迟不出现。SenseVoice 合成音频测试中冷加载约 8–10 秒，能解释这类等待；它不是关闭按钮没有绑定。 + +只对实验本地 ASR 使用宿主 TaskSpawner 后台 attach。麦克风已就绪即可进入 Recording；停止先释放 capture 并进入正常识别收尾，结果仍等待真实模型输出，不伪造完成。模型启动失败通过 RecordingFault 反馈，finalizing 后由 finish 返回原始错误；取消前尚未调度的加载不执行，加载中取消后的迟到 session 不回放音频并清理。云端管线和先录音后识别的稳定模式不改变。 + +### CPU、内存与加载 + +- TextStream 每次启动都会重新读取并 SHA-256 校验约 1.9 GB 固定权重。新增仅存在当前进程的校验缓存，文件目录、设备、inode、大小及 mtime/ctime（纳秒）未变才复用；首次、替换、修改或新进程仍全量校验，校验前后发生变化不缓存。没有写入可跨进程复用的信任标记。 +- MLX 默认内存指导上限为推荐工作集的 1.5 倍，空闲缓存默认也可增长到该上限。现在显式设置工作集指导值为物理内存的三分之一、推荐工作集与 4 GiB 的最小值；空闲缓存最多 128 MiB，并在加载/识别片段/最终化后释放不再使用的缓存。它不是 OS 级硬 RSS 限额，活动模型权重和 KV 状态不会被强删。 +- 原 CPU 总预算可能达到 5–7 个逻辑核，线程参数最多 2 不等于全部 native 辅助线程总用量最多 2。现在协作式平均预算另加 2 个逻辑核上限，保留 utility QoS 和可取消让步；单次不可分割 native 调用仍可能短时突发。 +- SenseVoice 先前按采样到的空闲 CPU 调整线程，忙闲变化可能重建整个 ONNX 模型；创建新模型时旧模型还在 slot 内，造成双份驻留峰值。SenseVoice 固定单 CPU 推理线程，已加载实例不因预算变化重建；必须替换的实例在构造新模型前释放。单线程优先降低争用，不承诺更快的批量推理。 +- 补充无文本的模型加载、包校验缓存命中、ready 和解码阶段计时。 +- 受管 TextStream worker 自动升级同时识别已知旧脚本/环境指纹，保留 Python 环境和权重；未知或修改过的脚本不覆盖。lease/status 能识别可升级版本，防止旧 worker 在进入 refresh 前被拦截。 + +### 多屏 + +后续用户明确要求两种模式自行选择：本轮 Beta 在设置增加 `capsuleDisplayMode`，默认 `cursorScreen`，显式选择 `allScreens` 时才镜像。macOS/Windows 共享策略，macOS 镜像转换为非激活面板并使用目标屏幕逻辑坐标；Windows 保留独立原文 rail。设置同步、恢复和主线程缓存均接线，模式切换使镜像缓存失效。默认值、序列化重载、无效值拒绝、原生模式缓存及两个 OS 的双浏览器控件测试通过。 + +复查发现频繁销毁/重建 WKWebView 会累积原生资源，因此改为有限槽位复用：不用的窗口隐藏并投递 Idle 停止动画，实时 capsule 状态只发往可见镜像；不为每块屏幕创建额外识别会话。以下是最初单屏修复的历史说明，新 Beta 已不再强制用户只能单屏。 + +Windows #1183 曾主动增加“主屏固定胶囊 + 其他屏幕镜像”，因此两个屏幕同时显示是该版本的明确行为，不是识别引擎创建两份 UI。现在只显示鼠标所在屏幕上的一个 capsule / transcript rail，兼容路径隐藏已有镜像；鼠标不可读时回退主屏。混合 DPI、负坐标屏幕和卡片模式使用同一定位依据。 + +macOS 镜像创建原本就被 cfg 排除，继续保留现有鼠标屏幕定位；不把 Windows 镜像根因冒称为 macOS 双胶囊已复现。本机没有完成 Windows 真双屏 UI 操作，定位与重复窗口策略由纯几何回归验证，Windows 编译需云端门禁。 + +## 验证 + +- Core 初轮全库：1210 passed / 1 外部依赖测试 ignored；后增“取消已排队的本地加载”后，最终受影响 dictation_engine 回归 34/34。覆盖三类实验模型启动尚未完成时的停止、取消、音频完整性、启动失败、云端与稳定模式原有语义。 +- 正常原生 macOS 全库：582 passed / 5 明确的模型/安装依赖 ignored。SenseVoice 与固定权重 TextStream 校验两项 ignored 测试另以已有模型、合成语音实际执行通过，不下载模型、不使用用户录音。 +- Core Clippy `--lib --tests -- -D warnings` 通过。改动后的 Python worker 13 项测试通过，Black/Ruff 通过，资源策略 helper 的 mypy strict 通过；既有 lifecycle/retirement/capsule replay 契约通过。 +- 原生包缓存回归覆盖同大小文件替换、删除、校验中修改和未知 worker 拒绝；旧 worker receipt 升级保留解释器。 +- 胶囊多屏纯逻辑 9/9，覆盖多屏不创建镜像、负坐标鼠标选择、DPI、边界与拓扑变化。 + +## 合成音频对比（本机开发验证，不是所有设备性能保证） + +TextStream 使用同一 30.46 秒英语合成音频、相同已有权重和平均 2 核预算，直接启动生产 worker；100ms 采样 worker RSS/CPU，输入由独立发送线程供给，不阻断采样。未读取用户音频。 + +| 指标 | 修正前 | 修正后 | +| --- | --- | --- | +| worker 峰值 RSS | 1674.14 MiB | 1340.75 MiB | +| 首个草稿（含启动） | 2.18s | 1.65s | +| 最终结果（含启动） | 9.21s | 8.76s | +| 合成音频转写 | 完整 | 与修正前相同 | + +RSS 降约 20%，采样 CPU 峰值约 88.5% / 90.3%，这次样本不支持“CPU 峰值已下降”的结论；平均预算与线程约束由计时回归验证。 + +SenseVoice 单线程实测：6.092 秒合成音频识别正确，冷加载 9.72s，随后 warm prepare 112ms，解码约 1.03s。此前两线程一次约 544ms；单线程降低争用会增加这段解码时间，不能包装成普遍提速。 + +TextStream 包校验在 debug 未优化 SHA 实测 cold 53.07s、warm 约 50µs，仅说明避免重复全文件读校验有效；不将 debug 耗时当作用户 Release 包的真实耗时。 + +## 本机与磁盘边界 + +磁盘此前开发构建缓存已清理，当前可用空间约 79 GiB；本轮只用一个共享 validation target、2 个构建任务、关闭 incremental/debug info，没有重建多个工作区的 target。未删除未经确认的系统快照。本轮验证结束后清理不再需要的临时构建产物，保留小型日志与对比结果用于复核。 + +当前只是源码修复与验证，不宣称已安装、已重新发布或所有设备问题已消失。 diff --git a/openless-all/app/crates/openless-core/src/cloud_sync_e2ee_documents/registry.rs b/openless-all/app/crates/openless-core/src/cloud_sync_e2ee_documents/registry.rs index bb38496d4..4ac190fef 100644 --- a/openless-all/app/crates/openless-core/src/cloud_sync_e2ee_documents/registry.rs +++ b/openless-all/app/crates/openless-core/src/cloud_sync_e2ee_documents/registry.rs @@ -102,6 +102,13 @@ pub const PREFERENCE_FIELDS: &[PreferenceField] = &[ shape: PreferenceShape::Text, reason: "portable_setting", }, + PreferenceField { + rust_name: "capsule_display_mode", + key: "capsuleDisplayMode", + class: PreferenceClass::Portable, + shape: PreferenceShape::Text, + reason: "portable_setting", + }, PreferenceField { rust_name: "capsule_transcript_enabled", key: "capsuleTranscriptEnabled", @@ -900,6 +907,7 @@ pub fn validate_preference_value(field: &PreferenceField, value: &Value) -> Docu let choices: &[&str] = match field.key { "defaultMode" => &["raw", "light", "structured", "formal"], "capsuleStyle" => &["siri", "classic", "typeless"], + "capsuleDisplayMode" => &["cursorScreen", "allScreens"], "pipelineMode" => &["traditional", "multimodal"], "chineseScriptPreference" => &["auto", "simplified", "traditional"], "outputLanguagePreference" => &["auto", "zhCn", "zhTw", "en", "ja", "ko"], diff --git a/openless-all/app/crates/openless-core/src/dictation_engine.rs b/openless-all/app/crates/openless-core/src/dictation_engine.rs index 00283faf5..376ab6b23 100644 --- a/openless-all/app/crates/openless-core/src/dictation_engine.rs +++ b/openless-all/app/crates/openless-core/src/dictation_engine.rs @@ -39,6 +39,7 @@ pub struct PipelineDictationEngine { transcription: Arc, polisher: Arc, polish_failure_policy: PolishFailurePolicy, + local_startup_spawner: Option>, sessions: Arc>>>, pending_captures: Arc>>>, } @@ -108,11 +109,20 @@ impl PipelineDictationEngine { transcription, polisher, polish_failure_policy: PolishFailurePolicy::UseRawText, + local_startup_spawner: None, sessions: Arc::new(Mutex::new(HashMap::new())), pending_captures: Arc::new(Mutex::new(HashMap::new())), } } + /// Once capture is running, local model loading must not hold the public + /// session in Starting: Stop must release the microphone before inference is ready. + /// Cloud routes and stable (record-first) mode keep their existing behavior. + pub fn with_local_startup_spawner(mut self, spawner: Arc) -> Self { + self.local_startup_spawner = Some(spawner); + self + } + pub fn with_polish_failure_policy(mut self, policy: PolishFailurePolicy) -> Self { self.polish_failure_policy = policy; self @@ -180,6 +190,10 @@ impl DictationEngine for PipelineDictationEngine { let transcription_engine = Arc::clone(&self.transcription); let sessions = Arc::clone(&self.sessions); let pending_captures = Arc::clone(&self.pending_captures); + let local_startup_spawner = self.local_startup_spawner.clone().filter(|_| { + crate::experimental_asr::ExperimentalAsrModel::from_provider(&context.asr.provider_type) + .is_some() + }); Box::pin(async move { let session = Arc::new(PipelineSession::new(Arc::clone(&context))); { @@ -324,7 +338,20 @@ impl DictationEngine for PipelineDictationEngine { "dictation cancelled while the recorder was starting", )); } - if !stable { + if let Some(spawner) = local_startup_spawner.filter(|_| !stable) { + let attaching = buffered.attach(); + let progress = Arc::clone(&buffered.inner.progress); + let session = Arc::clone(&session); + spawner.spawn(Box::pin(async move { + if let Err(error) = attaching.await { + if !session.cancelled.load(Ordering::Acquire) + && !session.finishing.load(Ordering::Acquire) + { + let _ = progress.publish(crate::ports::RecordingEvent::Fatal(error)); + } + } + })); + } else if !stable { if let Err(error) = buffered.attach().await { let recording = session .resources @@ -1434,6 +1461,17 @@ fn attach_buffered_transcription( ) -> BoxFuture<'static, Result, BackendError>> { Box::pin(async move { let _notify = NotifyOnDrop { waiter }; + if matches!( + *inner + .state + .lock() + .expect("buffered transcription lock poisoned"), + BufferedTranscriptionState::Cancelled + ) { + return Err(cancelled_error( + "transcription cancelled before provider startup", + )); + } let downstream = match inner.prepared.start(Arc::clone(&inner.partials)).await { Ok(session) => session, Err(error) => { @@ -2855,6 +2893,165 @@ mod tests { assert!(engine.sessions.lock().unwrap().is_empty()); } + #[tokio::test] + async fn local_model_loading_never_blocks_stop_or_cancel_of_the_microphone() { + use crate::experimental_asr::ExperimentalAsrModel; + for model in ExperimentalAsrModel::ALL { + for cancel in [false, true] { + let stops = Arc::new(AtomicUsize::new(0)); + let pcm = Arc::new(Mutex::new(Vec::new())); + let cancels = Arc::new(AtomicUsize::new(0)); + let entered = Arc::new(tokio::sync::Notify::new()); + let release = Arc::new(tokio::sync::Notify::new()); + let engine = Arc::new( + PipelineDictationEngine::new( + Arc::new(ExposedRecorder { + consumer: Arc::new(Mutex::new(None)), + stops: Arc::clone(&stops), + }), + Arc::new(DelayedTranscriber { + entered: Arc::clone(&entered), + release: Arc::clone(&release), + session: Arc::new(FixtureTranscriptionSession { + pcm: Arc::clone(&pcm), + cancels: Arc::clone(&cancels), + finish_entered: None, + finish_release: None, + }), + }), + noop_polisher(), + ) + .with_local_startup_spawner(Arc::new(crate::TokioTaskSpawner)), + ); + let mut context = (*raw_dictation_context()).clone(); + context.asr.provider_type = model.provider_type().into(); + let id = SessionId::new(); + let progress = Arc::new(RecordingProgress::default()); + tokio::time::timeout( + std::time::Duration::from_secs(1), + engine.start(id, Arc::new(context), progress.clone()), + ) + .await + .expect("capture must be ready before model loading") + .unwrap(); + entered.notified().await; + if cancel { + engine.cancel(id).await.unwrap(); + assert_eq!(stops.load(Ordering::Acquire), 1); + release.notify_one(); + tokio::time::timeout(std::time::Duration::from_secs(1), async { + while cancels.load(Ordering::Acquire) == 0 { + tokio::task::yield_now().await; + } + }) + .await + .unwrap(); + assert!( + pcm.lock().unwrap().is_empty(), + "cancelled capture must never be replayed" + ); + } else { + let finishing = tokio::spawn({ + let engine = engine.clone(); + async move { engine.finish(id, progress).await } + }); + tokio::time::timeout(std::time::Duration::from_secs(1), async { + while stops.load(Ordering::Acquire) == 0 { + tokio::task::yield_now().await; + } + }) + .await + .expect("stop must release capture while model is still loading"); + assert!( + !finishing.is_finished(), + "result still waits for real recognition" + ); + release.notify_one(); + assert_eq!(finishing.await.unwrap().unwrap().raw_text, "raw text"); + assert_eq!(*pcm.lock().unwrap(), vec![1, 0]); + } + } + } + } + + #[tokio::test] + async fn cancelled_queued_local_start_never_loads_the_model() { + #[derive(Default)] + struct Queue(Mutex>>); + impl crate::TaskSpawner for Queue { + fn spawn(&self, task: BoxFuture<'static, ()>) { + self.0.lock().unwrap().push(task); + } + } + let spawner = Arc::new(Queue::default()); + let starts = Arc::new(AtomicUsize::new(0)); + let stops = Arc::new(AtomicUsize::new(0)); + let engine = PipelineDictationEngine::new( + Arc::new(ExposedRecorder { + consumer: Arc::new(Mutex::new(None)), + stops: stops.clone(), + }), + Arc::new(FixtureTranscriber { + starts: starts.clone(), + session: Arc::new(FixtureTranscriptionSession { + pcm: Arc::new(Mutex::new(Vec::new())), + cancels: Arc::new(AtomicUsize::new(0)), + finish_entered: None, + finish_release: None, + }), + }), + noop_polisher(), + ) + .with_local_startup_spawner(spawner.clone()); + let mut context = (*raw_dictation_context()).clone(); + context.asr.provider_type = crate::experimental_asr::ExperimentalAsrModel::TextStream + .provider_type() + .into(); + let id = SessionId::new(); + engine + .start( + id, + Arc::new(context), + Arc::new(RecordingProgress::default()), + ) + .await + .unwrap(); + engine.cancel(id).await.unwrap(); + let tasks = std::mem::take(&mut *spawner.0.lock().unwrap()); + assert_eq!(tasks.len(), 1); + for task in tasks { + task.await; + } + assert_eq!(starts.load(Ordering::Acquire), 0); + assert_eq!(stops.load(Ordering::Acquire), 1); + } + + #[tokio::test] + async fn background_local_start_failure_is_reported_not_silently_recorded_forever() { + let engine = PipelineDictationEngine::new( + Arc::new(ExposedRecorder { + consumer: Arc::new(Mutex::new(None)), + stops: Arc::new(AtomicUsize::new(0)), + }), + Arc::new(FailingTranscriber), + noop_polisher(), + ) + .with_local_startup_spawner(Arc::new(crate::TokioTaskSpawner)); + let mut context = (*raw_dictation_context()).clone(); + context.asr.provider_type = crate::experimental_asr::ExperimentalAsrModel::SenseVoice + .provider_type() + .into(); + let id = SessionId::new(); + let progress = Arc::new(RecordingProgress::default()); + engine + .start(id, Arc::new(context), progress.clone()) + .await + .unwrap(); + // finish must expose the original provider failure, not a fabricated empty result. + let error = engine.finish(id, progress).await.unwrap_err(); + assert_eq!(error.error.code, BackendErrorCode::Provider); + } + #[tokio::test] async fn provider_polish_failure_uses_raw_text_fallback() { let fixture = fixture_engine( diff --git a/openless-all/app/crates/openless-core/src/lib.rs b/openless-all/app/crates/openless-core/src/lib.rs index acbbd2432..0e3bca1c7 100644 --- a/openless-all/app/crates/openless-core/src/lib.rs +++ b/openless-all/app/crates/openless-core/src/lib.rs @@ -331,9 +331,9 @@ pub use remote_input_service::{ pub use selection_voice_intent::SelectionVoiceIntent; pub use settings::*; pub use shared_types::{ - CapsulePayload, CapsuleState, CapsuleStyle, CredentialsStatus, HotkeyMode, HotkeyStatus, - PendingCorrection, PlatformCapabilities, SelectionPolishOutputMode, UserPreferences, - LOCAL_ASR_KEEP_LOADED_FOREVER_SECS, + CapsuleDisplayMode, CapsulePayload, CapsuleState, CapsuleStyle, CredentialsStatus, HotkeyMode, + HotkeyStatus, PendingCorrection, PlatformCapabilities, SelectionPolishOutputMode, + UserPreferences, LOCAL_ASR_KEEP_LOADED_FOREVER_SECS, }; pub use shortcut_types::{ binding_from_legacy_trigger, binding_requires_mouse_hook, binding_requires_side_aware_hook, diff --git a/openless-all/app/crates/openless-core/src/shared_types.rs b/openless-all/app/crates/openless-core/src/shared_types.rs index 2edc9b62d..9ff98cb1f 100644 --- a/openless-all/app/crates/openless-core/src/shared_types.rs +++ b/openless-all/app/crates/openless-core/src/shared_types.rs @@ -398,6 +398,9 @@ pub struct UserPreferences { /// Recording capsule appearance. Preference events sync it to all windows; recording state carries the current style. #[serde(default)] pub capsule_style: CapsuleStyle, + /// Display only under the pointer, or mirror recording on every attached display. + #[serde(default)] + pub capsule_display_mode: CapsuleDisplayMode, #[serde(default = "default_true")] pub capsule_transcript_enabled: bool, #[serde(default = "default_capsule_transcript_font_size")] @@ -965,6 +968,8 @@ struct UserPreferencesWire { show_capsule: bool, #[serde(default)] capsule_style: CapsuleStyle, + #[serde(default)] + capsule_display_mode: CapsuleDisplayMode, #[serde(default = "default_true")] capsule_transcript_enabled: bool, #[serde(default = "default_capsule_transcript_font_size")] @@ -1230,6 +1235,7 @@ impl Default for UserPreferencesWire { launch_at_login: prefs.launch_at_login, show_capsule: prefs.show_capsule, capsule_style: prefs.capsule_style, + capsule_display_mode: prefs.capsule_display_mode, capsule_transcript_enabled: prefs.capsule_transcript_enabled, capsule_transcript_font_size: prefs.capsule_transcript_font_size, mute_during_recording: prefs.mute_during_recording, @@ -1433,6 +1439,7 @@ impl<'de> Deserialize<'de> for UserPreferences { launch_at_login: wire.launch_at_login, show_capsule: wire.show_capsule, capsule_style: wire.capsule_style, + capsule_display_mode: wire.capsule_display_mode, capsule_transcript_enabled: wire.capsule_transcript_enabled, capsule_transcript_font_size: wire.capsule_transcript_font_size.clamp(12, 20), mute_during_recording: wire.mute_during_recording, @@ -1813,6 +1820,7 @@ impl Default for UserPreferences { launch_at_login: false, show_capsule: true, capsule_style: CapsuleStyle::Siri, + capsule_display_mode: CapsuleDisplayMode::CursorScreen, capsule_transcript_enabled: true, capsule_transcript_font_size: default_capsule_transcript_font_size(), mute_during_recording: false, @@ -2621,6 +2629,16 @@ pub enum CapsuleState { Error, } +/// Which attached displays show the recording capsule. Old preferences default +/// to one display, so upgrading never opts users into extra windows/renderers. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)] +#[serde(rename_all = "camelCase")] +pub enum CapsuleDisplayMode { + #[default] + CursorScreen, + AllScreens, +} + /// Recording capsule appearance; serialized values feed preference storage and each Host's window events. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize, Default)] #[serde(rename_all = "camelCase")] diff --git a/openless-all/app/crates/openless-core/tests/capsule_display_preferences.rs b/openless-all/app/crates/openless-core/tests/capsule_display_preferences.rs new file mode 100644 index 000000000..bcc0836ce --- /dev/null +++ b/openless-all/app/crates/openless-core/tests/capsule_display_preferences.rs @@ -0,0 +1,39 @@ +use openless_core::{CapsuleDisplayMode, UserPreferences}; + +#[test] +fn legacy_preferences_default_to_cursor_display_without_extra_windows() { + let prefs: UserPreferences = serde_json::from_str("{}").unwrap(); + assert_eq!(prefs.capsule_display_mode, CapsuleDisplayMode::CursorScreen); + assert_eq!( + UserPreferences::default().capsule_display_mode, + CapsuleDisplayMode::CursorScreen + ); +} + +#[test] +fn both_display_modes_survive_save_reload_and_unrelated_preference_changes() { + for (mode, wire) in [ + (CapsuleDisplayMode::CursorScreen, "cursorScreen"), + (CapsuleDisplayMode::AllScreens, "allScreens"), + ] { + let prefs = UserPreferences { + capsule_display_mode: mode, + ..Default::default() + }; + let value = serde_json::to_value(prefs).unwrap(); + assert_eq!(value["capsuleDisplayMode"], wire); + let mut reloaded: UserPreferences = serde_json::from_value(value).unwrap(); + reloaded.show_capsule = false; + let saved = serde_json::to_vec(&reloaded).unwrap(); + let restored: UserPreferences = serde_json::from_slice(&saved).unwrap(); + assert_eq!(restored.capsule_display_mode, mode); + assert!(!restored.show_capsule); + } +} + +#[test] +fn unknown_display_mode_is_not_silently_accepted() { + assert!( + serde_json::from_str::(r#"{"capsuleDisplayMode":"sometimes"}"#).is_err() + ); +} diff --git a/openless-all/app/e2e/capsule-display-mode.spec.ts b/openless-all/app/e2e/capsule-display-mode.spec.ts new file mode 100644 index 000000000..72a9c2f96 --- /dev/null +++ b/openless-all/app/e2e/capsule-display-mode.spec.ts @@ -0,0 +1,20 @@ +import { expect, test } from '@playwright/test'; + +for (const os of ['mac', 'win']) { + test(`capsule display preference offers both modes and survives remount on ${os}`, async ({ page }) => { + await page.goto(`/?os=${os}`); + await page.getByRole('button', { name: '设置', exact: true }).click(); + const dialog = page.getByRole('dialog', { name: '设置', exact: true }); + const control = dialog.getByRole('combobox', { name: '多屏胶囊', exact: true }); + await expect(control).toContainText('鼠标所在屏幕'); + await control.click(); + await page.getByRole('option', { name: '所有屏幕显示', exact: true }).click(); + await expect(control).toContainText('所有屏幕显示'); + await dialog.getByRole('button', { name: '外观与语言', exact: true }).click(); + await dialog.getByRole('navigation', { name: '设置分类' }).getByRole('button', { name: '录音与输入', exact: true }).click(); + await expect(control).toContainText('所有屏幕显示'); + await control.click(); + await page.getByRole('option', { name: '鼠标所在屏幕', exact: true }).click(); + await expect(control).toContainText('鼠标所在屏幕'); + }); +} diff --git a/openless-all/app/package-lock.json b/openless-all/app/package-lock.json index d0b509182..5fdd0acae 100644 --- a/openless-all/app/package-lock.json +++ b/openless-all/app/package-lock.json @@ -1,12 +1,12 @@ { "name": "openless-app", - "version": "2.0.1+build.20261016", + "version": "2.0.2-Beta.1+build.20261017", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "openless-app", - "version": "2.0.1+build.20261016", + "version": "2.0.2-Beta.1+build.20261017", "dependencies": { "@base-ui/react": "^1.6.0", "@shadcn/react": "^0.2.1", diff --git a/openless-all/app/package.json b/openless-all/app/package.json index 79e57b2f0..1f7b2bb0e 100644 --- a/openless-all/app/package.json +++ b/openless-all/app/package.json @@ -1,7 +1,7 @@ { "name": "openless-app", "private": true, - "version": "2.0.1+build.20261016", + "version": "2.0.2-Beta.1+build.20261017", "type": "module", "scripts": { "pretest": "npm run build", diff --git a/openless-all/app/scripts/desktop-version-contract.test.mjs b/openless-all/app/scripts/desktop-version-contract.test.mjs index 3c5010c57..7118db033 100644 --- a/openless-all/app/scripts/desktop-version-contract.test.mjs +++ b/openless-all/app/scripts/desktop-version-contract.test.mjs @@ -3,7 +3,7 @@ import { readFile } from 'node:fs/promises'; const app = new URL('../', import.meta.url); const read = async (path) => (await readFile(new URL(path, app), 'utf8')).replace(/\r\n/g, '\n'); -const version = '2.0.1+build.20261016'; +const version = '2.0.2-Beta.1+build.20261017'; const pkg = JSON.parse(await read('package.json')); const lock = JSON.parse(await read('package-lock.json')); const tauri = JSON.parse(await read('src-tauri/tauri.conf.json')); diff --git a/openless-all/app/scripts/local-asr-lifecycle-contract.test.mjs b/openless-all/app/scripts/local-asr-lifecycle-contract.test.mjs index 945cca7e8..b13561430 100644 --- a/openless-all/app/scripts/local-asr-lifecycle-contract.test.mjs +++ b/openless-all/app/scripts/local-asr-lifecycle-contract.test.mjs @@ -10,7 +10,13 @@ assert.equal(existsSync(new URL('../src-tauri/src/asr/local/retired.rs', import. const native = read('src-tauri/src/core_adapters.rs'); assert.doesNotMatch(native, /LocalAsrCache|LocalWhisperCache|LocalQwenEngine|AppleSpeechAsr/); const worker = read('src-tauri/src/asr/local/experimental/textstream.rs'); -assert.match(worker, /installed\(&directory\)[^]*?verify_package[^]*?models\.check_lease[^]*?Command::new\(install::python/); +assert.match(worker, /installed\(&directory\)[^]*?verification\.verify\(Model::TextStream[^]*?models\.check_lease[^]*?Command::new\(install::python/); +const verification = read('src-tauri/src/asr/local/experimental/textstream/package_cache.rs'); +assert.match(verification, /verify_package\(model, directory\)/); +assert.match(verification, /meta\.ino\(\)/); +assert.match(verification, /meta\.ctime_nsec\(\)/); +assert.match(verification, /before != after/); +assert.match(native, /with_local_startup_spawner/); assert.match(worker, /TextStreamChunk\s*\{\s*text: finalized \+ &draft,\s*offset: 0/); const core = read('crates/openless-core/src/local_asr_service.rs'); const activate = core.match(/ fn activate\([^]*?(?=\n fn )/)[0]; diff --git a/openless-all/app/src-tauri/Cargo.lock b/openless-all/app/src-tauri/Cargo.lock index d48d814ca..e92eb6af3 100644 --- a/openless-all/app/src-tauri/Cargo.lock +++ b/openless-all/app/src-tauri/Cargo.lock @@ -3867,7 +3867,7 @@ dependencies = [ [[package]] name = "openless" -version = "2.0.1+build.20261016" +version = "2.0.2-Beta.1+build.20261017" dependencies = [ "anyhow", "arboard", diff --git a/openless-all/app/src-tauri/Cargo.toml b/openless-all/app/src-tauri/Cargo.toml index 98edbbbd7..5794b4863 100644 --- a/openless-all/app/src-tauri/Cargo.toml +++ b/openless-all/app/src-tauri/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "openless" -version = "2.0.1+build.20261016" +version = "2.0.2-Beta.1+build.20261017" license = "AGPL-3.0-only" description = "OpenLess — local voice input that types where your cursor is" authors = ["OpenLess"] diff --git a/openless-all/app/src-tauri/src/asr/local/experimental.rs b/openless-all/app/src-tauri/src/asr/local/experimental.rs index 9e8dc562e..69c8ad351 100644 --- a/openless-all/app/src-tauri/src/asr/local/experimental.rs +++ b/openless-all/app/src-tauri/src/asr/local/experimental.rs @@ -153,7 +153,9 @@ impl ExperimentalNativeRuntime { check()?; let stream = recognizer.create_stream(); stream.accept_waveform(16_000, chunk); + let decode_started = std::time::Instant::now(); recognizer.decode(&stream); + log::info!("[experimental-asr] model=sensevoice stage=decode audio_ms={} elapsed_ms={}", chunk.len() / 16, decode_started.elapsed().as_millis()); let text = stream .get_result() .ok_or_else(|| failure("SenseVoice returned no result"))? @@ -246,11 +248,17 @@ impl ExperimentalAsrRuntime for ExperimentalNativeRuntime { return Ok(()); } openless_core::experimental_asr::verify_package(model, &directory)?; + // A changed thread budget must not construct a second native model + // while the previous model/allocator is still resident. + *slot = None; + let loading = std::time::Instant::now(); let engine = match model { #[cfg(any(target_os = "macos", target_os = "windows"))] Model::SenseVoice => { let mut config = sherpa_onnx::OfflineRecognizerConfig::default(); - config.model_config.num_threads = budget.threads; + // One CPU thread avoids a persistent ONNX worker pool and + // keeps the cached model valid as the idle budget fluctuates. + config.model_config.num_threads = 1; config.model_config.provider = Some("cpu".into()); config.model_config.tokens = Some(directory.join("tokens.txt").to_string_lossy().into()); @@ -279,9 +287,17 @@ impl ExperimentalAsrRuntime for ExperimentalNativeRuntime { if generation.load(Ordering::Acquire) != expected { return Err(failure("Experimental model was disabled while loading")); } + log::info!( + "[experimental-asr] model={model:?} stage=load elapsed_ms={}", + loading.elapsed().as_millis() + ); *slot = Some(Loaded { directory, - cpu_threads: budget.threads, + cpu_threads: if model == Model::SenseVoice { + 1 + } else { + budget.threads + }, engine, }); Ok(()) @@ -438,6 +454,9 @@ mod tests { let started = std::time::Instant::now(); runtime.prepare(model, directory.clone()).await.unwrap(); let load = started.elapsed(); + let warm_started = std::time::Instant::now(); + runtime.prepare(model, directory.clone()).await.unwrap(); + let warm_prepare = warm_started.elapsed(); let wav = std::fs::read( std::env::var_os("OPENLESS_ASR_ENGLISH_TEST_WAV") .expect("provide the English 16kHz PCM smoke-test WAV"), @@ -454,7 +473,7 @@ mod tests { .await .unwrap(); println!( - "{model:?}: load={load:?} inference={:?} audio={:.3}s text={text}", + "{model:?}: load={load:?} warm_prepare={warm_prepare:?} inference={:?} audio={:.3}s text={text}", inference.elapsed(), samples.len() as f64 / 16000.0 ); diff --git a/openless-all/app/src-tauri/src/asr/local/experimental/compute_policy.rs b/openless-all/app/src-tauri/src/asr/local/experimental/compute_policy.rs index 1d8c7b4c5..d141c3fb3 100644 --- a/openless-all/app/src-tauri/src/asr/local/experimental/compute_policy.rs +++ b/openless-all/app/src-tauri/src/asr/local/experimental/compute_policy.rs @@ -22,7 +22,10 @@ impl Budget { } else { 0.25 // Missing telemetry must never mean unlimited CPU. }; - let cores = (logical_cores.max(1) as f64 * idle * IDLE_SHARE).max(0.05); + // Native worker pools (including Core ML/MLX helpers) can exceed the + // requested BLAS/ONNX thread count. Bound their aggregate average too. + let cores = + (logical_cores.max(1) as f64 * idle * IDLE_SHARE).clamp(0.05, MAX_CPU_THREADS as f64); Self { cores, threads: (cores.floor() as i32).clamp(1, MAX_CPU_THREADS), @@ -33,7 +36,7 @@ impl Budget { let cores = std::thread::available_parallelism().map_or(1, |n| n.get()); let budget = Self::from_idle(cores, idle_fraction().unwrap_or(0.25)); log::info!( - "[experimental-asr] CPU budget={:.2} core(s), native threads={}, idle share=80%", + "[experimental-asr] CPU budget={:.2} core(s), native threads={}, idle share=80%, aggregate cap=2 cores", budget.cores, budget.threads ); @@ -200,9 +203,9 @@ impl Drop for BackgroundScope { mod tests { use super::*; #[test] - fn uses_eighty_percent_of_idle_capacity_not_eighty_percent_of_machine() { + fn uses_eighty_percent_of_idle_capacity_with_two_core_aggregate_ceiling() { let budget = Budget::from_idle(10, 0.9); - assert!((budget.cores - 7.2).abs() < 1e-9); + assert_eq!(budget.cores, 2.0); assert_eq!(budget.threads, 2); let busy = Budget::from_idle(10, 0.1); assert!((busy.cores - 0.8).abs() < 1e-9); diff --git a/openless-all/app/src-tauri/src/asr/local/experimental/textstream.rs b/openless-all/app/src-tauri/src/asr/local/experimental/textstream.rs index a031f0883..e419f5329 100644 --- a/openless-all/app/src-tauri/src/asr/local/experimental/textstream.rs +++ b/openless-all/app/src-tauri/src/asr/local/experimental/textstream.rs @@ -6,6 +6,8 @@ mod install; mod process; #[path = "textstream/protocol.rs"] mod protocol; +#[path = "textstream/package_cache.rs"] +mod package_cache; use process::Process; use base64::Engine; @@ -47,6 +49,7 @@ impl Drop for StartupGuard { pub struct Runtime { active: Mutex>>, generation: AtomicU64, + verification: Arc, } impl Runtime { #[cfg(debug_assertions)] @@ -103,10 +106,16 @@ impl Runtime { "TextStream runtime is not installed; explicitly retry its experimental switch", )); } + let started = std::time::Instant::now(); let verify = directory.clone(); + let verification = Arc::clone(&self.verification); let budget = tokio::task::spawn_blocking(move || { let _background = super::compute_policy::BackgroundScope::enter(); - openless_core::experimental_asr::verify_package(Model::TextStream, &verify)?; + let cached = verification.verify(Model::TextStream, &verify)?; + log::info!( + "[experimental-asr] model=textstream stage=verify cached={cached} elapsed_ms={}", + started.elapsed().as_millis() + ); Ok::<_, BackendError>(super::compute_policy::Budget::capture()) }) .await @@ -210,6 +219,10 @@ impl Runtime { .map_err(error)??; models.check_lease(Model::TextStream, generation)?; self.check_start_generation(expected_runtime)?; + log::info!( + "[experimental-asr] model=textstream stage=ready elapsed_ms={}", + started.elapsed().as_millis() + ); guard.0.take(); Ok(Arc::new(Session { process, diff --git a/openless-all/app/src-tauri/src/asr/local/experimental/textstream/install.rs b/openless-all/app/src-tauri/src/asr/local/experimental/textstream/install.rs index 08d3e2cd8..6955b25b3 100644 --- a/openless-all/app/src-tauri/src/asr/local/experimental/textstream/install.rs +++ b/openless-all/app/src-tauri/src/asr/local/experimental/textstream/install.rs @@ -42,13 +42,49 @@ pub fn python(directory: &Path) -> PathBuf { pub fn script(directory: &Path) -> PathBuf { directory.join(".runtime/worker.py") } -pub fn installed(directory: &Path) -> bool { +fn current_worker_installed(directory: &Path) -> bool { python(directory).is_file() && std::fs::read_to_string(directory.join(".runtime/installed")) .is_ok_and(|s| s == marker(directory)) && std::fs::read_to_string(script(directory)).is_ok_and(|s| s == WORKER) } +const PREVIOUS_WORKERS: &[(&str, &str)] = &[ + ( + "f585f75ca2d669b48f75a40fe6a7213f9948ac84ce3c5c9378f2b84dfe77a243", + "6c6e1ce76c0e3e220b35160744c5253b499d364095e294c45d712ed7b0922184", + ), + ( + "64e7ec7b3c3557bea2314d8a99564cf7cfb0402ff1cea586fc3c5366ef5e543b", + "f08166fc2a375a42f9c86bcc75ded43e1bc8b742c4fba4289db4002a6bc464dc", + ), +]; + +/// A known managed worker can be upgraded at recording start without reinstalling +/// Python. Lease/status must recognize it, otherwise start never reaches refresh. +pub fn installed(directory: &Path) -> bool { + if current_worker_installed(directory) { + return true; + } + let root = directory.join(".runtime"); + if !python(directory).is_file() + || !std::fs::read_to_string(root.join("requirements.lock")).is_ok_and(|s| s == REQUIREMENTS) + { + return false; + } + let Ok(receipt) = std::fs::read_to_string(root.join("installed")) else { + return false; + }; + let Ok(worker) = std::fs::read(script(directory)) else { + return false; + }; + let hash = format!("{:x}", Sha256::digest(&worker)); + PREVIOUS_WORKERS.iter().any(|(environment, expected)| { + receipt == format!("{PYTHON}:{UV_SHA256}:{}:{environment}", directory.display()) + && (hash == *expected || worker == WORKER.as_bytes()) + }) +} + struct CommandGuard { child: tokio::process::Child, } @@ -215,7 +251,7 @@ async fn installation_lock(root: &Path) -> Result { /// Upgrade only our worker when the pinned Python/dependency environment is /// unchanged. Never redownload weights or clear a working venv for a script fix. pub async fn refresh_worker(directory: &Path) -> Result { - if installed(directory) { + if current_worker_installed(directory) { return Ok(true); } let root = directory.join(".runtime"); @@ -227,30 +263,13 @@ pub async fn refresh_worker(directory: &Path) -> Result { } fn refresh_worker_locked(directory: &Path) -> Result { - if installed(directory) { + if current_worker_installed(directory) { return Ok(true); } - const PREVIOUS_ENV: &str = "f585f75ca2d669b48f75a40fe6a7213f9948ac84ce3c5c9378f2b84dfe77a243"; - const PREVIOUS_WORKER: &str = - "6c6e1ce76c0e3e220b35160744c5253b499d364095e294c45d712ed7b0922184"; - let root = directory.join(".runtime"); - let previous = format!( - "{PYTHON}:{UV_SHA256}:{}:{PREVIOUS_ENV}", - directory.display() - ); - if !python(directory).is_file() - || !std::fs::read_to_string(root.join("installed")).is_ok_and(|s| s == previous) - || !std::fs::read_to_string(root.join("requirements.lock")).is_ok_and(|s| s == REQUIREMENTS) - { - return Ok(false); - } - let Ok(worker) = std::fs::read(script(directory)) else { - return Ok(false); - }; - // The current script also permits recovery after a crash between renames. - if format!("{:x}", Sha256::digest(&worker)) != PREVIOUS_WORKER && worker != WORKER.as_bytes() { + if !installed(directory) { return Ok(false); } + let root = directory.join(".runtime"); fn replace(path: &Path, bytes: &[u8]) -> Result<(), BackendError> { use std::io::Write; let temporary = path.with_extension(format!("{}.update", uuid::Uuid::new_v4())); @@ -300,6 +319,17 @@ mod worker_update_tests { let root = directory.join(".runtime"); std::fs::write(root.join("requirements.lock"), REQUIREMENTS).unwrap(); std::fs::write(script(directory), WORKER).unwrap(); + for (environment, _) in PREVIOUS_WORKERS { + let receipt = format!("{PYTHON}:{UV_SHA256}:{}:{environment}", directory.display()); + std::fs::write(root.join("installed"), receipt).unwrap(); + assert!( + installed(directory), + "lease must allow a known worker to reach refresh" + ); + assert!(!current_worker_installed(directory)); + assert!(refresh_worker(directory).await.unwrap()); + assert!(current_worker_installed(directory)); + } let previous = format!("{PYTHON}:{UV_SHA256}:{}:f585f75ca2d669b48f75a40fe6a7213f9948ac84ce3c5c9378f2b84dfe77a243", directory.display()); std::fs::write(root.join("installed"), &previous).unwrap(); assert!( diff --git a/openless-all/app/src-tauri/src/asr/local/experimental/textstream/package_cache.rs b/openless-all/app/src-tauri/src/asr/local/experimental/textstream/package_cache.rs new file mode 100644 index 000000000..b19b5afd2 --- /dev/null +++ b/openless-all/app/src-tauri/src/asr/local/experimental/textstream/package_cache.rs @@ -0,0 +1,153 @@ +//! In-process verification memo. A new process or any file identity/metadata +//! change requires full pinned-byte verification again. Never persist trust. +use openless_core::{experimental_asr::ExperimentalAsrModel, BackendError, BackendErrorCode}; +use std::{ + os::unix::fs::MetadataExt, + path::{Path, PathBuf}, + sync::Mutex, +}; + +#[derive(Debug, PartialEq, Eq)] +struct Stamp { + directory: PathBuf, + // dev, inode, size, modification and change time (nanoseconds included). + files: Vec, +} + +#[derive(Debug, PartialEq, Eq)] +struct FileStamp { + device: u64, + inode: u64, + bytes: u64, + modified: (i64, i64), + changed: (i64, i64), +} + +fn stamp(directory: &Path, paths: &[String]) -> Result { + let directory = directory.canonicalize().map_err(failure)?; + let mut files = Vec::with_capacity(paths.len()); + for path in paths { + let meta = std::fs::metadata(directory.join(path)).map_err(failure)?; + if !meta.is_file() { + return Err(failure("model package contains a non-file")); + } + files.push(FileStamp { + device: meta.dev(), + inode: meta.ino(), + bytes: meta.len(), + modified: (meta.mtime(), meta.mtime_nsec()), + changed: (meta.ctime(), meta.ctime_nsec()), + }); + } + Ok(Stamp { directory, files }) +} +fn failure(error: impl std::fmt::Display) -> BackendError { + BackendError::new(BackendErrorCode::Provider, error.to_string()) +} + +#[derive(Default)] +pub(super) struct VerificationCache(Mutex>); +impl VerificationCache { + /// Only call from the blocking preflight worker. Returns true for a cache hit. + pub(super) fn verify( + &self, + model: ExperimentalAsrModel, + directory: &Path, + ) -> Result { + let paths = model + .files() + .into_iter() + .map(|f| f.path) + .collect::>(); + self.verify_with(directory, &paths, || { + openless_core::experimental_asr::verify_package(model, directory) + }) + } + fn verify_with( + &self, + directory: &Path, + paths: &[String], + verify: impl FnOnce() -> Result<(), BackendError>, + ) -> Result { + let before = stamp(directory, paths)?; + let mut cached = self.0.lock().map_err(failure)?; + if cached.as_ref() == Some(&before) { + return Ok(true); + } + *cached = None; + verify()?; + let after = stamp(directory, paths)?; + if before != after { + return Err(failure("model package changed during verification")); + } + *cached = Some(after); + Ok(false) + } +} + +#[cfg(test)] +mod tests { + use super::*; + #[test] + fn unchanged_package_is_cached_but_same_size_replacement_and_deletion_are_not() { + let root = + std::env::temp_dir().join(format!("openless-package-cache-{}", uuid::Uuid::new_v4())); + std::fs::create_dir_all(&root).unwrap(); + let model = root.join("model"); + let paths = vec!["model".to_owned()]; + std::fs::write(&model, b"good").unwrap(); + let cache = VerificationCache::default(); + assert!(!cache.verify_with(&root, &paths, || Ok(())).unwrap()); + assert!(cache + .verify_with(&root, &paths, || panic!("unchanged package rehashed")) + .unwrap()); + std::fs::write(root.join("replacement"), b"evil").unwrap(); + std::fs::rename(root.join("replacement"), &model).unwrap(); + assert!(cache + .verify_with(&root, &paths, || Err(failure("checksum"))) + .is_err()); + assert!(cache.0.lock().unwrap().is_none()); + std::fs::remove_file(&model).unwrap(); + assert!(cache + .verify_with(&root, &paths, || panic!("missing model was trusted")) + .is_err()); + std::fs::remove_dir_all(root).unwrap(); + } + #[test] + #[ignore = "requires existing pinned TextStream weights; never downloads"] + fn existing_textstream_package_is_hashed_once_per_unchanged_process_cache() { + let root = PathBuf::from( + std::env::var_os("OPENLESS_TEXTSTREAM_MODEL_DIR").expect("model directory"), + ); + let cache = VerificationCache::default(); + let start = std::time::Instant::now(); + assert!(!cache + .verify(ExperimentalAsrModel::TextStream, &root) + .unwrap()); + let cold = start.elapsed(); + let start = std::time::Instant::now(); + assert!(cache + .verify(ExperimentalAsrModel::TextStream, &root) + .unwrap()); + println!( + "TextStream pinned preflight cold={cold:?} warm={:?}", + start.elapsed() + ); + } + + #[test] + fn concurrent_package_mutation_is_never_memoized() { + let root = + std::env::temp_dir().join(format!("openless-package-cache-{}", uuid::Uuid::new_v4())); + std::fs::create_dir_all(&root).unwrap(); + std::fs::write(root.join("model"), b"good").unwrap(); + let cache = VerificationCache::default(); + let result = cache.verify_with(&root, &["model".to_owned()], || { + std::fs::write(root.join("model"), b"different").unwrap(); + Ok(()) + }); + assert!(result.is_err()); + assert!(cache.0.lock().unwrap().is_none()); + std::fs::remove_dir_all(root).unwrap(); + } +} diff --git a/openless-all/app/src-tauri/src/asr/local/experimental/textstream/test_worker.py b/openless-all/app/src-tauri/src/asr/local/experimental/textstream/test_worker.py index 0d52271af..c4ac6997d 100644 --- a/openless-all/app/src-tauri/src/asr/local/experimental/textstream/test_worker.py +++ b/openless-all/app/src-tauri/src/asr/local/experimental/textstream/test_worker.py @@ -112,6 +112,23 @@ def test_native_thread_limits_are_bounded_before_imports(self) -> None: ), self.assertRaises(ValueError): worker.configure_cpu_threads() + def test_metal_memory_leaves_room_for_capture_and_bounds_free_cache(self) -> None: + for gib in (4, 8, 16, 64): + total = gib * 1024**3 + memory, cache = worker.metal_memory_limits(total, total * 3 // 4) + self.assertLessEqual(memory, min(4 * 1024**3, total // 3)) + self.assertLessEqual(cache, 128 * 1024**2) + self.assertGreater(cache, 0) + with self.assertRaises(ValueError): + worker.metal_memory_limits(0, 0) + + def test_cpu_capacity_is_capped_even_when_many_cores_are_idle(self) -> None: + with patch.object(os, "cpu_count", return_value=16), patch.dict( + os.environ, {"OPENLESS_CPU_BUDGET_CORES": "12"} + ): + self.assertEqual(worker.configure_cpu_threads(), 2.0) + self.assertEqual(os.environ["OMP_NUM_THREADS"], "2") + def test_draft_is_revisable_and_final_comes_from_decoder(self): snapshots = worker.Snapshots() self.assertEqual(snapshots.update("你", "你好吗")["draft"], "好吗") @@ -131,7 +148,20 @@ def run_fake(self, finish=True, gpu_available=True): lines = json.dumps(request) + "\n" + ('{"type":"finish"}\n' if finish else "") output = io.StringIO() mlx_core = SimpleNamespace( - metal=SimpleNamespace(is_available=lambda: gpu_available), + metal=SimpleNamespace( + is_available=lambda: gpu_available, + device_info=lambda: { + "memory_size": 16 * 1024**3, + "max_recommended_working_set_size": 12 * 1024**3, + }, + ), + set_memory_limit=lambda limit: FakeSession.calls.append( + ("memory_limit", limit) + ), + set_cache_limit=lambda limit: FakeSession.calls.append( + ("cache_limit", limit) + ), + clear_cache=lambda: FakeSession.calls.append(("clear_cache",)), gpu="gpu", set_default_device=lambda device: FakeSession.calls.append( ("device", device) @@ -181,6 +211,13 @@ def test_feed_produces_draft_before_finish_and_final_is_refined(self): self.assertEqual(events[-1]["finalized"], "你好，世界。") self.assertEqual(events[-1]["draft"], "") self.assertIn(("device", "gpu"), FakeSession.calls) + self.assertIn(("memory_limit", 4 * 1024**3), FakeSession.calls) + self.assertIn(("cache_limit", 128 * 1024**2), FakeSession.calls) + self.assertEqual(FakeSession.calls.count(("clear_cache",)), 3) + self.assertLess( + next(i for i, c in enumerate(FakeSession.calls) if c[0] == "cache_limit"), + next(i for i, c in enumerate(FakeSession.calls) if c[0] == "load"), + ) init = next(call for call in FakeSession.calls if call[0] == "init") self.assertIsNone(init[1]["language"], "do not hard-code English") diff --git a/openless-all/app/src-tauri/src/asr/local/experimental/textstream/worker.py b/openless-all/app/src-tauri/src/asr/local/experimental/textstream/worker.py index 37dba3113..7677307fb 100644 --- a/openless-all/app/src-tauri/src/asr/local/experimental/textstream/worker.py +++ b/openless-all/app/src-tauri/src/asr/local/experimental/textstream/worker.py @@ -27,6 +27,17 @@ MAX_AUDIO_BYTES = 32_000 * 600 +def metal_memory_limits(memory_bytes: int, recommended_bytes: int) -> tuple[int, int]: + """Reserve RAM for the host; MLX's memory limit is a guideline, not an OS quota.""" + if memory_bytes <= 0 or recommended_bytes <= 0: + raise ValueError("Metal memory information is unavailable") + # MLX defaults to 1.5x the recommended working set and retains freed buffers + # up to that same limit. That is inappropriate beside a live recorder/UI. + memory_limit = min(4 * 1024**3, memory_bytes // 3, recommended_bytes) + cache_limit = min(128 * 1024**2, memory_limit // 16) + return memory_limit, cache_limit + + class CpuBudget: """Cooperative CPU-seconds budget; GPU wait is not charged as CPU work.""" @@ -60,6 +71,7 @@ def configure_cpu_threads() -> float: cores = float(os.environ.get("OPENLESS_CPU_BUDGET_CORES", "1")) if not math.isfinite(cores) or not 0 < cores <= (os.cpu_count() or 1): raise ValueError("invalid CPU budget") + cores = min(cores, 2.0) threads = str(max(1, min(2, math.floor(cores)))) for name in ( "OMP_NUM_THREADS", @@ -236,6 +248,13 @@ def emit(event): "TextStream requires Metal GPU; CPU fallback is disabled" ) mx.set_default_device(mx.gpu) + device = mx.metal.device_info() + memory_limit, cache_limit = metal_memory_limits( + int(device["memory_size"]), + int(device["max_recommended_working_set_size"]), + ) + mx.set_memory_limit(memory_limit) + mx.set_cache_limit(cache_limit) import numpy as np from mlx_qwen3_asr import Session from silero_vad_lite import SileroVAD @@ -251,6 +270,7 @@ def emit(event): finalization_mode="accuracy", ) vad = SileroVAD(16000) + mx.clear_cache() budget.checkpoint() emit({"type": "ready", "protocol": 1}) snapshots = Snapshots() @@ -271,6 +291,9 @@ def feed(raw): if heard_speech: with contextlib.redirect_stdout(sys.stderr): state = session.feed_audio(audio, state) + # Release dead allocations as the rolling window grows; + # live model weights/KV state remain owned by the session. + mx.clear_cache() budget.checkpoint() emit(snapshots.update(state.stable_text, state.text)) @@ -296,6 +319,7 @@ def feed(raw): if heard_speech: with contextlib.redirect_stdout(sys.stderr): state = session.finish_streaming(state) + mx.clear_cache() budget.checkpoint() # finish_streaming's completed text, not our last draft. emit(snapshots.update(state.stable_text, state.text, terminal=True)) diff --git a/openless-all/app/src-tauri/src/capsule_mirrors.rs b/openless-all/app/src-tauri/src/capsule_mirrors.rs index ab1bfe2b6..6234b7fb5 100644 --- a/openless-all/app/src-tauri/src/capsule_mirrors.rs +++ b/openless-all/app/src-tauri/src/capsule_mirrors.rs @@ -1,25 +1,24 @@ -//! Windows: show the recording capsule on every monitor at once (issue #1057). +//! Native capsule display policy: follow the pointer by default, mirror only +//! when the user explicitly chooses all attached displays. //! //! The primary `capsule` / `capsule-rail` windows stay the source of truth: they own the -//! card modes, the layout cache and the event routing. Every other monitor gets a mirror -//! pair `capsule-mirror-` / `capsule-rail-mirror-` that loads the same frontend route -//! and receives the same state events. Mirrors are created lazily, reused, and hidden when -//! the capsule is not visible. +//! card modes, the layout cache and the event routing. Legacy mirror labels are kept +//! for opt-in mirrors; cursor-only mode creates no additional windows. //! //! Pure placement / topology helpers in this module are platform-agnostic and unit-tested -//! on every CI host. Window creation stays Windows-only. +//! on every CI host. Native mirror creation supports macOS and Windows. use std::collections::hash_map::DefaultHasher; use std::hash::{Hash, Hasher}; -#[cfg(target_os = "windows")] +#[cfg(any(target_os = "macos", target_os = "windows"))] use tauri::{AppHandle, Manager, Monitor, WebviewUrl, WebviewWindow, WebviewWindowBuilder}; +use crate::types::CapsuleDisplayMode; pub(crate) const CAPSULE_MIRROR_PREFIX: &str = "capsule-mirror-"; pub(crate) const RAIL_MIRROR_PREFIX: &str = "capsule-rail-mirror-"; -/// Upper bound on mirrored monitors (the primary monitor is not mirrored, so 8 displays -/// are covered in total). Capability globs cover these labels. +/// Legacy slot count used to hide any previously created mirror windows. pub(crate) const MAX_CAPSULE_MIRRORS: usize = 7; /// Physical description of one monitor in virtual-desktop coordinates. @@ -38,7 +37,7 @@ pub(crate) struct MonitorRect { } impl MonitorRect { - #[cfg(target_os = "windows")] + #[cfg(any(target_os = "macos", target_os = "windows"))] pub(crate) fn from_tauri(monitor: &Monitor) -> Self { let position = monitor.position(); let size = monitor.size(); @@ -68,21 +67,23 @@ impl MonitorRect { (self.left, self.top, self.right, self.bottom) } } - - fn is_same_monitor(&self, other: &Self) -> bool { - self.left == other.left - && self.top == other.top - && self.right == other.right - && self.bottom == other.bottom - } } -/// The OS primary monitor, where the primary capsule lives. Stable across foreground changes, -/// unlike the old "monitor of the focused window" placement. +/// Follow the mouse, not the capsule HWND (which can still be on the old screen). +/// Fall back to the primary display if the cursor cannot be queried. #[cfg(target_os = "windows")] -pub(crate) fn primary_monitor_rect( +pub(crate) fn target_monitor_rect( window: &WebviewWindow, ) -> Option { + use windows::Win32::{Foundation::POINT, UI::WindowsAndMessaging::GetCursorPos}; + let monitors = all_monitor_rects(window); + let mut point = POINT::default(); + // SAFETY: GetCursorPos writes only to this initialized POINT. + if unsafe { GetCursorPos(&mut point) }.is_ok() { + if let Some(monitor) = monitor_at_point(&monitors, point.x, point.y) { + return Some(monitor); + } + } window .primary_monitor() .ok() @@ -90,8 +91,40 @@ pub(crate) fn primary_monitor_rect( .map(|monitor| MonitorRect::from_tauri(&monitor)) } +pub(crate) fn monitor_at_point(monitors: &[MonitorRect], x: i32, y: i32) -> Option { + monitors + .iter() + .copied() + .find(|m| x >= m.left && x < m.right && y >= m.top && y < m.bottom) +} + +/// Match the macOS pointer-based primary placement without querying the old window position. +#[cfg(target_os = "macos")] +pub(crate) fn target_monitor_rect( + window: &WebviewWindow, +) -> Option { + if let Some(m) = crate::capsule_target_monitor(window) { + return Some(MonitorRect { + left: m.physical_x, + top: m.physical_y, + right: m.physical_x + m.physical_width as i32, + bottom: m.physical_y + m.physical_height as i32, + work_left: m.work_x, + work_top: m.work_y, + work_right: m.work_x + m.work_width as i32, + work_bottom: m.work_y + m.work_height as i32, + scale: m.scale, + }); + } + window + .primary_monitor() + .ok() + .flatten() + .map(|m| MonitorRect::from_tauri(&m)) +} + /// Every monitor currently connected, in the OS enumeration order. -#[cfg(target_os = "windows")] +#[cfg(any(target_os = "macos", target_os = "windows"))] pub(crate) fn all_monitor_rects(window: &WebviewWindow) -> Vec { window .available_monitors() @@ -122,13 +155,22 @@ pub(crate) fn capsule_placement( (phys_w.max(1) as u32, phys_h.max(1) as u32, x, y) } -/// Every monitor except the primary one, in enumeration order, capped at -/// [`MAX_CAPSULE_MIRRORS`]. -pub(crate) fn mirror_targets(monitors: &[MonitorRect], primary: MonitorRect) -> Vec { +/// Mirror only on explicit opt-in, excluding the display already hosting the primary. +pub(crate) fn mirror_targets( + mode: CapsuleDisplayMode, + monitors: &[MonitorRect], + primary: MonitorRect, +) -> Vec { + if mode != CapsuleDisplayMode::AllScreens { + return Vec::new(); + } monitors .iter() .copied() - .filter(|monitor| !monitor.is_same_monitor(&primary)) + .filter(|m| { + (m.left, m.top, m.right, m.bottom) + != (primary.left, primary.top, primary.right, primary.bottom) + }) .take(MAX_CAPSULE_MIRRORS) .collect() } @@ -184,7 +226,7 @@ pub(crate) fn mirror_label(prefix: &str, index: usize) -> String { } /// Labels of existing mirror windows whose label starts with `prefix`. -#[cfg(target_os = "windows")] +#[cfg(any(target_os = "macos", target_os = "windows"))] pub(crate) fn existing_mirror_labels( app: &AppHandle, prefix: &str, @@ -197,7 +239,7 @@ pub(crate) fn existing_mirror_labels( } /// Returns the capsule mirror with `label`, creating it hidden when it does not exist yet. -#[cfg(target_os = "windows")] +#[cfg(any(target_os = "macos", target_os = "windows"))] pub(crate) fn ensure_capsule_mirror( app: &AppHandle, label: &str, @@ -205,7 +247,7 @@ pub(crate) fn ensure_capsule_mirror( if let Some(window) = app.get_webview_window(label) { return Ok(window); } - WebviewWindowBuilder::new( + let window = WebviewWindowBuilder::new( app, label, WebviewUrl::App("index.html?window=capsule".into()), @@ -221,7 +263,26 @@ pub(crate) fn ensure_capsule_mirror( .focused(false) .visible(false) .accept_first_mouse(true) - .build() + .build()?; + #[cfg(target_os = "macos")] + { + use tauri_nspanel::cocoa::appkit::NSWindowCollectionBehavior; + use tauri_nspanel::WebviewWindowExt; + let panel = match window.to_panel() { + Ok(panel) => panel, + Err(error) => { + let _ = window.close(); + return Err(error); + } + }; + panel.set_style_mask(1 << 7); // Non-activating, like the primary capsule. + panel.set_level(25); + panel.set_collection_behaviour( + NSWindowCollectionBehavior::NSWindowCollectionBehaviorFullScreenAuxiliary + | NSWindowCollectionBehavior::NSWindowCollectionBehaviorCanJoinAllSpaces, + ); + } + Ok(window) } /// Returns the transcript-rail mirror with `label`, creating it hidden when needed. @@ -271,30 +332,68 @@ mod tests { } #[test] - fn mirror_targets_excludes_primary_and_keeps_enumeration_order() { + fn multi_monitor_recording_has_no_duplicate_capsules() { let primary = monitor(0, 0, 1920, 1080, 40, 1.0); let left = monitor(-1920, 0, 1920, 1080, 40, 1.0); let right = monitor(1920, 0, 2560, 1440, 40, 1.5); - let targets = mirror_targets(&[left, primary, right], primary); + let targets = mirror_targets( + CapsuleDisplayMode::CursorScreen, + &[left, primary, right], + primary, + ); - assert_eq!(targets, vec![left, right]); + assert!(targets.is_empty()); + assert_eq!( + mirror_targets( + CapsuleDisplayMode::AllScreens, + &[left, primary, right], + primary + ), + vec![left, right] + ); + assert_eq!( + mirror_targets( + CapsuleDisplayMode::AllScreens, + &[left, primary, right], + right + ), + vec![left, primary] + ); + assert!(mirror_targets(CapsuleDisplayMode::AllScreens, &[primary], primary).is_empty()); + assert_eq!( + monitor_at_point(&[left, primary, right], -20, 50), + Some(left) + ); + assert_eq!( + monitor_at_point(&[left, primary, right], 2000, 50), + Some(right) + ); + assert_eq!( + monitor_at_point(&[left, primary, right], 10, 50), + Some(primary) + ); + assert_eq!(monitor_at_point(&[left, primary, right], 99999, 50), None); } #[test] fn mirror_targets_is_empty_on_single_monitor() { let primary = monitor(0, 0, 1920, 1080, 40, 1.0); - assert!(mirror_targets(&[primary], primary).is_empty()); + assert!(mirror_targets(CapsuleDisplayMode::CursorScreen, &[primary], primary).is_empty()); } #[test] - fn mirror_targets_caps_the_number_of_mirrors() { + fn cursor_only_never_mirrors_and_all_screens_is_bounded() { let primary = monitor(0, 0, 1920, 1080, 0, 1.0); let mut monitors = vec![primary]; for i in 1..=(MAX_CAPSULE_MIRRORS as i32 + 3) { monitors.push(monitor(i * 1920, 0, 1920, 1080, 0, 1.0)); } - assert_eq!(mirror_targets(&monitors, primary).len(), MAX_CAPSULE_MIRRORS); + assert!(mirror_targets(CapsuleDisplayMode::CursorScreen, &monitors, primary).is_empty()); + assert_eq!( + mirror_targets(CapsuleDisplayMode::AllScreens, &monitors, primary).len(), + MAX_CAPSULE_MIRRORS + ); } #[test] diff --git a/openless-all/app/src-tauri/src/coordinator.rs b/openless-all/app/src-tauri/src/coordinator.rs index 2d68b67f9..41eeb689f 100644 --- a/openless-all/app/src-tauri/src/coordinator.rs +++ b/openless-all/app/src-tauri/src/coordinator.rs @@ -1545,9 +1545,13 @@ impl Coordinator { /// After saving, syncs style, window size, and click region without /// waiting for the next recording state event. pub fn sync_capsule_style_from_preferences(&self) { + let preferences = self.inner.backend.get_preferences(); self.inner .host - .cache_capsule_style(self.inner.backend.get_preferences().capsule_style); + .cache_capsule_display_mode(preferences.capsule_display_mode); + self.inner + .host + .cache_capsule_style(preferences.capsule_style); } /// Apply only the Tauri presentation side of the Core-owned vocabulary /// suggestion state. Commands mutate the Core collection first, then pass diff --git a/openless-all/app/src-tauri/src/coordinator/restore_runtime.rs b/openless-all/app/src-tauri/src/coordinator/restore_runtime.rs index 04767889e..5c171d0db 100644 --- a/openless-all/app/src-tauri/src/coordinator/restore_runtime.rs +++ b/openless-all/app/src-tauri/src/coordinator/restore_runtime.rs @@ -58,6 +58,10 @@ impl openless_core::config::RestoreRuntimeEffects for RestoreHost { )))] return Err(failure("restore_host_platform_unsupported")); crate::net::set_use_system_proxy(work_target.use_system_proxy); + coord + .inner + .host + .cache_capsule_display_mode(work_target.capsule_display_mode); coord .inner .host diff --git a/openless-all/app/src-tauri/src/core_adapters.rs b/openless-all/app/src-tauri/src/core_adapters.rs index 25260ff04..2b83bfae2 100644 --- a/openless-all/app/src-tauri/src/core_adapters.rs +++ b/openless-all/app/src-tauri/src/core_adapters.rs @@ -258,11 +258,14 @@ pub(crate) fn backend_dependencies( Arc::clone(&host_recorder), external, )); - let traditional = Arc::new(openless_core::PipelineDictationEngine::new( - Arc::clone(&recorder), - transcription, - Arc::clone(&polisher), - )); + let traditional = Arc::new( + openless_core::PipelineDictationEngine::new( + Arc::clone(&recorder), + transcription, + Arc::clone(&polisher), + ) + .with_local_startup_spawner(Arc::clone(&task_spawner)), + ); let dictation = Arc::new(openless_core::DictationEngineRouter::new(traditional)); let production_omni: Arc = Arc::new( openless_core::SharedOmniDictationEngine::new(Arc::clone(&credential_store), recorder), diff --git a/openless-all/app/src-tauri/src/lib.rs b/openless-all/app/src-tauri/src/lib.rs index 9d2ff41f3..37239aff6 100644 --- a/openless-all/app/src-tauri/src/lib.rs +++ b/openless-all/app/src-tauri/src/lib.rs @@ -3476,17 +3476,16 @@ pub(crate) fn position_capsule_bottom_center_with_style_and_transcript ({},{})", + "[capsule] win position: target mon=({},{})..({},{}) scale={:.2} size=({}x{}) -> ({},{})", mon.left, mon.top, mon.right, mon.bottom, mon.scale, phys_w, phys_h, x, y ); window.set_position(PhysicalPosition::new(x, y))?; diff --git a/openless-all/app/src-tauri/src/tauri_coordinator_host.rs b/openless-all/app/src-tauri/src/tauri_coordinator_host.rs index 315b6def0..e3653cf21 100644 --- a/openless-all/app/src-tauri/src/tauri_coordinator_host.rs +++ b/openless-all/app/src-tauri/src/tauri_coordinator_host.rs @@ -19,7 +19,7 @@ use openless_core::BackendEvent; use parking_lot::Mutex; use tauri::{AppHandle, Emitter, Manager}; -use crate::types::{CapsulePayload, CapsuleState, CapsuleStyle}; +use crate::types::{CapsuleDisplayMode, CapsulePayload, CapsuleState, CapsuleStyle}; static CAPSULE_SUPPRESSED_BY_TOGGLE_LOGGED: AtomicBool = AtomicBool::new(false); static CAPSULE_FIRST_SHOW_LOGGED: AtomicBool = AtomicBool::new(false); @@ -728,14 +728,15 @@ struct CapsuleWindowState { hit_test_mode: AtomicU8, transcript_visible: AtomicBool, style: AtomicU8, + mirror_all_displays: AtomicBool, fallback_card_visible: AtomicBool, fallback_presentation_id: AtomicU64, deferred_payload: Mutex>, snapshot: Mutex, layout_watch_active: AtomicBool, layout_watch_epoch: AtomicU64, - /// Windows: fingerprint of the last mirror sync; `None` forces the next sync to apply. - #[cfg(target_os = "windows")] + /// Native display topology and presentation cache; preference changes invalidate it. + #[cfg(any(target_os = "macos", target_os = "windows"))] mirror_fingerprint: Mutex>, } @@ -748,19 +749,31 @@ impl Default for CapsuleWindowState { hit_test_mode: AtomicU8::new(HIT_TEST_MODE_CAPSULE), transcript_visible: AtomicBool::new(false), style: AtomicU8::new(0), + mirror_all_displays: AtomicBool::new(false), fallback_card_visible: AtomicBool::new(false), fallback_presentation_id: AtomicU64::new(0), deferred_payload: Mutex::new(None), snapshot: Mutex::new(CapsuleSnapshotState::default()), layout_watch_active: AtomicBool::new(false), layout_watch_epoch: AtomicU64::new(0), - #[cfg(target_os = "windows")] + #[cfg(any(target_os = "macos", target_os = "windows"))] mirror_fingerprint: Mutex::new(None), } } } impl CapsuleWindowState { + fn cache_display_mode(&self, mode: CapsuleDisplayMode) { + let all = mode == CapsuleDisplayMode::AllScreens; + let changed = self.mirror_all_displays.swap(all, Ordering::SeqCst) != all; + #[cfg(any(target_os = "macos", target_os = "windows"))] + if changed { + *self.mirror_fingerprint.lock() = None; + } + #[cfg(not(any(target_os = "macos", target_os = "windows")))] + let _ = changed; + } + fn cache_style(&self, style: CapsuleStyle) { self.style.store( match style { @@ -985,35 +998,32 @@ impl TauriCapsuleWindow { pub(crate) fn invalidate_layout(&self) { *self.state.layout.lock() = None; - #[cfg(target_os = "windows")] + #[cfg(any(target_os = "macos", target_os = "windows"))] { *self.state.mirror_fingerprint.lock() = None; } } - /// Windows: makes every mirror capsule / transcript rail match the primary window — - /// visibility, geometry, cursor passthrough and hit-testing — so the capsule shows on - /// all monitors (issue #1057). Queued on the main thread because window creation and - /// show/hide must run there. A no-op when nothing relevant changed. - #[cfg(target_os = "windows")] + /// Apply the user's cursor-only/all-displays choice on the native main thread. + #[cfg(any(target_os = "macos", target_os = "windows"))] pub(crate) fn sync_capsule_mirrors(&self) { let _ = self.run_on_main_thread(|capsule| capsule.apply_capsule_mirrors()); } - #[cfg(not(target_os = "windows"))] + #[cfg(not(any(target_os = "macos", target_os = "windows")))] pub(crate) fn sync_capsule_mirrors(&self) {} - #[cfg(target_os = "windows")] + #[cfg(any(target_os = "macos", target_os = "windows"))] fn apply_capsule_mirrors(&self) { use crate::capsule_mirrors::{ - capsule_placement, ensure_capsule_mirror, ensure_rail_mirror, mirror_label, - mirror_targets, CAPSULE_MIRROR_PREFIX, MAX_CAPSULE_MIRRORS, RAIL_MIRROR_PREFIX, + capsule_placement, ensure_capsule_mirror, mirror_label, mirror_targets, + CAPSULE_MIRROR_PREFIX, MAX_CAPSULE_MIRRORS, RAIL_MIRROR_PREFIX, }; let Some(primary) = self.window() else { return; }; - let Some(primary_rect) = crate::capsule_mirrors::primary_monitor_rect(&primary) else { + let Some(primary_rect) = crate::capsule_mirrors::target_monitor_rect(&primary) else { return; }; let monitors = crate::capsule_mirrors::all_monitor_rects(&primary); @@ -1052,30 +1062,54 @@ impl TauriCapsuleWindow { } let app = self.app.clone(); + let mode = if self.state.mirror_all_displays.load(Ordering::SeqCst) { + CapsuleDisplayMode::AllScreens + } else { + CapsuleDisplayMode::CursorScreen + }; let targets = if visible { - mirror_targets(&monitors, primary_rect) + mirror_targets(mode, &monitors, primary_rect) } else { Vec::new() }; - // Hide every mirror slot that has no monitor to sit on (or when the capsule is hidden). + // Reuse a bounded set of webviews. Recreating WKWebViews on every + // recording/mode switch accumulates native resources on macOS. + let hidden_payload = self + .state + .snapshot + .lock() + .payload + .clone() + .map(|mut payload| { + payload.state = CapsuleState::Idle; + payload + }); + let mut complete = true; for index in targets.len()..MAX_CAPSULE_MIRRORS { for prefix in [CAPSULE_MIRROR_PREFIX, RAIL_MIRROR_PREFIX] { if let Some(window) = app.get_webview_window(&mirror_label(prefix, index)) { - let _ = window.hide(); + if let Err(error) = window.hide() { + log::warn!("[capsule] mirror hide failed: {error}"); + complete = false; + } + // Stop hidden shader/exit animations; future live frames are + // routed only to visible mirror windows. + if let Some(payload) = &hidden_payload { + let _ = app.emit_to(window.label(), "capsule:state", payload); + } } } } let bounds = crate::capsule_window_bounds_for_style_with_transcript_and_translation( style, - false, + cfg!(target_os = "macos") && transcript_visible, translation_active, ); let bottom_gap = crate::capsule_bottom_gap(style); - // Only a fully applied sync may be cached; a failed mirror must be retried by the next sync. - let mut complete = true; + // Only a fully applied sync may be cached; failed operations are retried. for (index, monitor) in targets.iter().enumerate() { let capsule_label = mirror_label(CAPSULE_MIRROR_PREFIX, index); let mirror = match ensure_capsule_mirror(&app, &capsule_label) { @@ -1088,23 +1122,38 @@ impl TauriCapsuleWindow { }; let (phys_w, phys_h, x, y) = capsule_placement(bounds.width, bounds.height, bottom_gap, *monitor); + #[cfg(target_os = "macos")] + let _ = (phys_w, phys_h); // Geometry failures must not be fingerprint-cached: a skipped retry would leave // the mirror stuck at the wrong size/position after a monitor reconnect. - if let Err(error) = mirror.set_size(tauri::PhysicalSize::new(phys_w, phys_h)) { + #[cfg(target_os = "windows")] + let size_result = mirror.set_size(tauri::PhysicalSize::new(phys_w, phys_h)); + #[cfg(target_os = "macos")] + let size_result = mirror.set_size(tauri::LogicalSize::new(bounds.width, bounds.height)); + if let Err(error) = size_result { log::warn!("[capsule] mirror {capsule_label} set_size failed: {error}"); complete = false; continue; } - if let Err(error) = mirror.set_position(tauri::PhysicalPosition::new(x, y)) { + #[cfg(target_os = "windows")] + let position_result = mirror.set_position(tauri::PhysicalPosition::new(x, y)); + #[cfg(target_os = "macos")] + let position_result = mirror.set_position(tauri::LogicalPosition::new( + x as f64 / monitor.scale, + y as f64 / monitor.scale, + )); + if let Err(error) = position_result { log::warn!("[capsule] mirror {capsule_label} set_position failed: {error}"); complete = false; continue; } + #[cfg(target_os = "windows")] let hit_test = if passthrough { clear_capsule_input_region_for_window(&mirror) } else { configure_capsule_hit_test(&mirror, style, transcript_visible, translation_active) }; + #[cfg(target_os = "windows")] if let Err(error) = hit_test { log::warn!("[capsule] mirror {capsule_label} hit-test setup failed: {error}"); complete = false; @@ -1119,26 +1168,29 @@ impl TauriCapsuleWindow { complete = false; } - let rail_label = mirror_label(RAIL_MIRROR_PREFIX, index); - match ensure_rail_mirror(&app, &rail_label) { - Ok(rail) => { - if let Err(error) = position_rail_for_capsule( - &app, - &rail, - tauri::PhysicalPosition::new(x, y), - monitor.scale, - style, - translation_active, - transcript_visible, - ) { - log::warn!("[capsule] mirror {rail_label} positioning failed: {error}"); + #[cfg(target_os = "windows")] + { + let rail_label = mirror_label(RAIL_MIRROR_PREFIX, index); + match crate::capsule_mirrors::ensure_rail_mirror(&app, &rail_label) { + Ok(rail) => { + if let Err(error) = position_rail_for_capsule( + &app, + &rail, + tauri::PhysicalPosition::new(x, y), + monitor.scale, + style, + translation_active, + transcript_visible, + ) { + log::warn!("[capsule] mirror {rail_label} positioning failed: {error}"); + complete = false; + } + } + Err(error) => { + log::warn!("[capsule] mirror {rail_label} create failed: {error}"); complete = false; } } - Err(error) => { - log::warn!("[capsule] mirror {rail_label} create failed: {error}"); - complete = false; - } } } @@ -1286,8 +1338,8 @@ impl TauriCapsuleWindow { ) -> Option { #[cfg(target_os = "windows")] { - // The primary capsule lives on the OS primary monitor (see capsule_mirrors). - if let Some(mon) = crate::capsule_mirrors::primary_monitor_rect(window) { + // Use the same cursor display as placement, including mixed-DPI layouts. + if let Some(mon) = crate::capsule_mirrors::target_monitor_rect(window) { let monitors = crate::capsule_mirrors::all_monitor_rects(window); return Some(CapsuleLayoutState { translation_active, @@ -1649,6 +1701,10 @@ impl TauriCoordinatorHost { self.capsule.snapshot.lock().record_backend_event(event); } + pub(crate) fn cache_capsule_display_mode(&self, mode: CapsuleDisplayMode) { + self.capsule.cache_display_mode(mode); + } + pub(crate) fn cache_capsule_style(&self, style: CapsuleStyle) { self.capsule.cache_style(style); if let Some(capsule) = self.capsule_window() { @@ -1803,13 +1859,18 @@ impl TauriCoordinatorHost { if let Some(app) = self.app() { let _ = app.emit_to("capsule", "capsule:state", payload); let _ = app.emit_to("capsule-rail", "capsule:state", payload); - #[cfg(target_os = "windows")] + #[cfg(any(target_os = "macos", target_os = "windows"))] for prefix in [ crate::capsule_mirrors::CAPSULE_MIRROR_PREFIX, crate::capsule_mirrors::RAIL_MIRROR_PREFIX, ] { for label in crate::capsule_mirrors::existing_mirror_labels(&app, prefix) { - let _ = app.emit_to(label.as_str(), "capsule:state", payload); + if app + .get_webview_window(&label) + .is_some_and(|window| window.is_visible().unwrap_or(false)) + { + let _ = app.emit_to(label.as_str(), "capsule:state", payload); + } } } } @@ -1887,6 +1948,29 @@ impl TauriCoordinatorHost { mod tests { use super::*; + #[test] + fn display_mode_changes_invalidate_mirrors_without_changing_style() { + let state = CapsuleWindowState::default(); + assert!(!state.mirror_all_displays.load(Ordering::SeqCst)); + for mode in [ + CapsuleDisplayMode::AllScreens, + CapsuleDisplayMode::CursorScreen, + ] { + #[cfg(any(target_os = "macos", target_os = "windows"))] + { + *state.mirror_fingerprint.lock() = Some(123); + } + state.cache_display_mode(mode); + assert_eq!( + state.mirror_all_displays.load(Ordering::SeqCst), + mode == CapsuleDisplayMode::AllScreens + ); + #[cfg(any(target_os = "macos", target_os = "windows"))] + assert!(state.mirror_fingerprint.lock().is_none()); + assert_eq!(state.cached_style(), CapsuleStyle::Siri); + } + } + #[test] fn capsule_style_cache_preserves_every_wire_variant() { let state = CapsuleWindowState::default(); diff --git a/openless-all/app/src-tauri/tauri.conf.json b/openless-all/app/src-tauri/tauri.conf.json index 8de1f7cbf..2bf2a484c 100644 --- a/openless-all/app/src-tauri/tauri.conf.json +++ b/openless-all/app/src-tauri/tauri.conf.json @@ -1,7 +1,7 @@ { "$schema": "https://schema.tauri.app/config/2", "productName": "OpenLess", - "version": "2.0.1+build.20261016", + "version": "2.0.2-Beta.1+build.20261017", "identifier": "com.openless.app", "build": { "beforeDevCommand": "npm run dev", diff --git a/openless-all/app/src/i18n/de.ts b/openless-all/app/src/i18n/de.ts index d11649b97..2d05a04f5 100644 --- a/openless-all/app/src/i18n/de.ts +++ b/openless-all/app/src/i18n/de.ts @@ -1521,6 +1521,10 @@ export const de: typeof zhCN = { capsuleTranscriptDesc: 'Erkannten Originaltext statt Audiobalken anzeigen. Ohne Streaming erscheint der Text nach der Erkennung. Deaktivieren stellt die bisherige Anzeige wieder her.', capsuleTranscriptFontSize: 'Transkript-Schriftgröße', + capsuleDisplayModeLabel: "Kapsel auf mehreren Bildschirmen", + capsuleDisplayModeDesc: "Wähle die Bildschirme für die Aufnahmekapsel. Alle Bildschirme benötigen zusätzliche Fenster und Grafikressourcen.", + capsuleDisplayCursorScreen: "Bildschirm unter dem Mauszeiger", + capsuleDisplayAllScreens: "Auf allen Bildschirmen anzeigen", capsuleStyleLabel: 'Kapselstil', capsuleStyleSiri: 'Schimmernder Siri-Stil', capsuleStyleClassic: 'OpenLess-Standardstil', diff --git a/openless-all/app/src/i18n/en.ts b/openless-all/app/src/i18n/en.ts index 71e7e7b33..353b3b7ba 100644 --- a/openless-all/app/src/i18n/en.ts +++ b/openless-all/app/src/i18n/en.ts @@ -1492,6 +1492,10 @@ export const en: typeof zhCN = { capsuleTranscriptDesc: 'Show recognized speech in the capsule instead of audio bars. Turn off to restore the original display. Non-streaming recognition appears when results arrive.', capsuleTranscriptFontSize: 'Transcript font size', + capsuleDisplayModeLabel: "Multi-display capsule", + capsuleDisplayModeDesc: "Choose where the recording capsule appears. Showing it on all displays uses additional windows and graphics resources.", + capsuleDisplayCursorScreen: "Display under the pointer", + capsuleDisplayAllScreens: "Show on all displays", capsuleStyleLabel: 'Capsule style', capsuleStyleSiri: 'Shimmer Siri style', capsuleStyleClassic: 'OpenLess default style', diff --git a/openless-all/app/src/i18n/es.ts b/openless-all/app/src/i18n/es.ts index a955dfd16..2811b88d3 100644 --- a/openless-all/app/src/i18n/es.ts +++ b/openless-all/app/src/i18n/es.ts @@ -1513,6 +1513,10 @@ export const es: typeof zhCN = { capsuleTranscriptDesc: 'Muestra el texto reconocido en lugar de las barras de audio. Sin streaming, aparece al recibir el resultado. Desactiva para restaurar la vista original.', capsuleTranscriptFontSize: 'Tamaño de la transcripción', + capsuleDisplayModeLabel: "Cápsula en varias pantallas", + capsuleDisplayModeDesc: "Elige dónde aparece la cápsula. El modo de todas las pantallas utiliza más ventanas y recursos gráficos.", + capsuleDisplayCursorScreen: "Pantalla bajo el puntero", + capsuleDisplayAllScreens: "Mostrar en todas las pantallas", capsuleStyleLabel: 'Estilo de cápsula', capsuleStyleSiri: 'Estilo luminoso Siri', capsuleStyleClassic: 'Estilo predeterminado de OpenLess', diff --git a/openless-all/app/src/i18n/fr.ts b/openless-all/app/src/i18n/fr.ts index 187401398..84cbb76a6 100644 --- a/openless-all/app/src/i18n/fr.ts +++ b/openless-all/app/src/i18n/fr.ts @@ -1533,6 +1533,10 @@ export const fr: typeof zhCN = { capsuleTranscriptDesc: 'Affiche le texte reconnu à la place des barres audio. Sans streaming, le texte apparaît à réception du résultat. Désactivez pour retrouver l’affichage initial.', capsuleTranscriptFontSize: 'Taille de la transcription', + capsuleDisplayModeLabel: "Capsule sur plusieurs écrans", + capsuleDisplayModeDesc: "Choisissez où afficher la capsule. Le mode tous les écrans utilise davantage de fenêtres et de ressources graphiques.", + capsuleDisplayCursorScreen: "Écran sous le pointeur", + capsuleDisplayAllScreens: "Afficher sur tous les écrans", capsuleStyleLabel: 'Style de capsule', capsuleStyleSiri: 'Style lumineux Siri', capsuleStyleClassic: 'Style par défaut d’OpenLess', diff --git a/openless-all/app/src/i18n/ja.ts b/openless-all/app/src/i18n/ja.ts index cccdcfc1c..4743df55f 100644 --- a/openless-all/app/src/i18n/ja.ts +++ b/openless-all/app/src/i18n/ja.ts @@ -1480,6 +1480,10 @@ export const ja: typeof zhCN = { capsuleTranscriptDesc: '音声バーの代わりに認識した原文を表示します。オフにすると元の表示に戻ります。非ストリーミング認識は結果の受信後に表示します。', capsuleTranscriptFontSize: '文字起こしの文字サイズ', + capsuleDisplayModeLabel: "複数画面のカプセル", + capsuleDisplayModeDesc: "録音カプセルを表示する画面を選択します。すべての画面ではウィンドウと描画リソースの使用量が増えます。", + capsuleDisplayCursorScreen: "マウスポインターのある画面", + capsuleDisplayAllScreens: "すべての画面に表示", capsuleStyleLabel: 'カプセルスタイル', capsuleStyleSiri: '光条 Siri スタイル', capsuleStyleClassic: 'Openless デフォルトスタイル', diff --git a/openless-all/app/src/i18n/ko.ts b/openless-all/app/src/i18n/ko.ts index 6269138e3..1ae922941 100644 --- a/openless-all/app/src/i18n/ko.ts +++ b/openless-all/app/src/i18n/ko.ts @@ -1468,6 +1468,10 @@ export const ko: typeof zhCN = { capsuleTranscriptDesc: '오디오 막대 대신 인식된 원문을 표시합니다. 끄면 기존 표시로 돌아갑니다. 비스트리밍 인식은 결과 수신 후 표시됩니다.', capsuleTranscriptFontSize: '원문 글자 크기', + capsuleDisplayModeLabel: "다중 화면 캡슐", + capsuleDisplayModeDesc: "녹음 캡슐을 표시할 화면을 선택합니다. 모든 화면 모드는 창과 그래픽 리소스를 더 사용합니다.", + capsuleDisplayCursorScreen: "마우스 포인터가 있는 화면", + capsuleDisplayAllScreens: "모든 화면에 표시", capsuleStyleLabel: '캡슐 스타일', capsuleStyleSiri: '시리 광선 스타일', capsuleStyleClassic: 'Openless 기본 스타일', diff --git a/openless-all/app/src/i18n/zh-CN.ts b/openless-all/app/src/i18n/zh-CN.ts index 10d0eb488..6f6146d86 100644 --- a/openless-all/app/src/i18n/zh-CN.ts +++ b/openless-all/app/src/i18n/zh-CN.ts @@ -1421,6 +1421,10 @@ export const zhCN = { capsuleTranscriptDesc: '在胶囊中显示语音识别原文，替换音频条；关闭后恢复原有显示。非流式识别将在结果返回后显示。', capsuleTranscriptFontSize: '原文字号', + capsuleDisplayModeLabel: "多屏胶囊", + capsuleDisplayModeDesc: "选择录音胶囊显示在哪些屏幕。所有屏幕模式会增加窗口与图形资源占用。", + capsuleDisplayCursorScreen: "鼠标所在屏幕", + capsuleDisplayAllScreens: "所有屏幕显示", capsuleStyleLabel: '胶囊样式', capsuleStyleSiri: '流光 Siri 风格', capsuleStyleClassic: 'Openless 默认风格', diff --git a/openless-all/app/src/i18n/zh-TW.ts b/openless-all/app/src/i18n/zh-TW.ts index 65895ad86..caf51beaf 100644 --- a/openless-all/app/src/i18n/zh-TW.ts +++ b/openless-all/app/src/i18n/zh-TW.ts @@ -1422,6 +1422,10 @@ export const zhTW: typeof zhCN = { capsuleTranscriptDesc: '在膠囊中顯示語音辨識原文，取代音訊條；關閉後恢復原有顯示。非串流辨識將於結果傳回後顯示。', capsuleTranscriptFontSize: '原文字級', + capsuleDisplayModeLabel: "多螢幕膠囊", + capsuleDisplayModeDesc: "選擇錄音膠囊顯示在哪些螢幕。所有螢幕模式會增加視窗與圖形資源用量。", + capsuleDisplayCursorScreen: "滑鼠所在螢幕", + capsuleDisplayAllScreens: "所有螢幕顯示", capsuleStyleLabel: '膠囊樣式', capsuleStyleSiri: '流光 Siri 風格', capsuleStyleClassic: 'Openless 預設風格', diff --git a/openless-all/app/src/lib/ipc/mock-data.ts b/openless-all/app/src/lib/ipc/mock-data.ts index 916f17ca2..b27080e99 100644 --- a/openless-all/app/src/lib/ipc/mock-data.ts +++ b/openless-all/app/src/lib/ipc/mock-data.ts @@ -47,6 +47,7 @@ export let mockSettings: UserPreferences = { launchAtLogin: false, showCapsule: true, capsuleStyle: 'siri', + capsuleDisplayMode: 'cursorScreen', capsuleTranscriptEnabled: true, capsuleTranscriptFontSize: 14, muteDuringRecording: false, diff --git a/openless-all/app/src/lib/types.ts b/openless-all/app/src/lib/types.ts index 66a4dd2d0..afea9b3bd 100644 --- a/openless-all/app/src/lib/types.ts +++ b/openless-all/app/src/lib/types.ts @@ -352,6 +352,7 @@ export interface UserPreferences { showCapsule: boolean; /** Recording capsule appearance; synced to the capsule window on save. */ capsuleStyle: CapsuleStyle; + capsuleDisplayMode: 'cursorScreen' | 'allScreens'; capsuleTranscriptEnabled: boolean; capsuleTranscriptFontSize: number; /** Temporarily mutes system output during recording; restores the original mute state on stop/cancel/error. */ diff --git a/openless-all/app/src/pages/settings/RecordingInputSection.tsx b/openless-all/app/src/pages/settings/RecordingInputSection.tsx index 42adc1928..6d9bbc1b5 100644 --- a/openless-all/app/src/pages/settings/RecordingInputSection.tsx +++ b/openless-all/app/src/pages/settings/RecordingInputSection.tsx @@ -458,6 +458,23 @@ export function RecordingInputSection() { )} + {!isAndroid && (os === 'mac' || os === 'win') && ( + + savePrefs({ ...prefs, capsuleDisplayMode: next as 'cursorScreen' | 'allScreens' })} + options={[ + { value: 'cursorScreen', label: t('settings.recording.capsuleDisplayCursorScreen') }, + { value: 'allScreens', label: t('settings.recording.capsuleDisplayAllScreens') }, + ]} + ariaLabel={t('settings.recording.capsuleDisplayModeLabel')} + style={{ maxWidth: 240, minWidth: 200 }} + /> + + )} {!isAndroid && (