Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ jobs:
run: python scripts/sync_catalog.py --check

- name: Python syntax
run: python -m py_compile server.py audio_encoding.py tray_app.py "Kokoro TTS.pyw" tts_catalog.py windows_runtime.py windows_startup.py
run: python -m py_compile server.py audio_encoding.py tray_app.py "Kokoro TTS.pyw" tts_catalog.py windows_protocol.py windows_runtime.py windows_startup.py

- name: Bundled FFmpeg check
run: python -c "from audio_encoding import validate_ffmpeg; validate_ffmpeg()"
Expand Down
132 changes: 105 additions & 27 deletions README.md

Large diffs are not rendered by default.

56 changes: 44 additions & 12 deletions docs/greasyfork-additional-info.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,15 +16,17 @@ Keep the GitHub links in both places: metadata makes them appear in Greasy Fork'
选中网页上的文本后,可以直接:

- `Read`:英文含公式时先读正文,同时后台处理公式;播放到公式处如果还没处理好再等待,然后继续交给 Kokoro TTS 朗读
- `Translate`:调用本机 Ollama 模型翻译,默认 `translategemma:4b`
- `Translate`:默认调用本机 Ollama,也可以明确选择由 Windows 托盘程序配置的项目服务器模型
- `Copy`:不翻译,只复制选中原文;MathJax/MathML/KaTeX 公式会尽量扩展到完整公式框并复制为 LaTeX
- UI 使用原生 DOM API 构建,不使用 `innerHTML` 等 HTML 字符串注入,以兼容 Gemini 等启用 Trusted Types 的页面
- 在设置面板里切换并保存声音、语速、翻译模型和目标语言
- 查看本地 TTS 服务与 Ollama 模型状态
- 使用 **Use project server** 选择可用的项目服务器模型,使用 **Initialize local model** 初始化所选本机 Ollama 模型
- 本地 API 尚未运行时,使用 **Start local service** 打开固定的 `localreadtranslate://start` 操作,再等待服务就绪
- 分别查看本地 API、按需加载的 TTS 与本机/项目服务器 Ollama 模型状态
- 在 Translation 设置栏手动常驻或卸载当前 Ollama 模型;频繁使用时减少首次加载等待,不用时释放显存
- 英文会尽量原样保留,中文会翻成英文;英文含公式的朗读会优先开始正文,公式在后台变成英文口语描述
- MathJax/MathML/LaTeX 会优先提取语义公式;翻译结果会把公式渲染为带上下标的易读公式,而不是显示原始 LaTeX 代码
- 翻译请求可附带附近正文作为本地参考上下文,只用于术语和指代消歧;真正翻译和输出的只有选中内容
- 翻译请求可附带附近正文作为参考上下文,只用于术语和指代消歧;真正翻译和输出的只有选中内容;选择远程模型时该上下文也会发送到对应服务器
- 上下文长度会按模型大小自动裁剪:4B 模型翻译和公式朗读不参考上下文,9B/14B/更大模型会逐级保留更多上下文
- `qwen3:14b`、QwQ、DeepSeek-R1 等推理模型会通过 Ollama `think: false` 关闭思考过程,降低翻译和朗读准备延迟
- 选择 4B 模型时,常见公式会优先使用本地保守字面读法,例如 `D_I` 读作 `D sub I`,`\hat{B}(x)` 读作 `B hat of x`
Expand All @@ -36,44 +38,70 @@ Keep the GitHub links in both places: metadata makes them appear in Greasy Fork'

## 重要:需要本地服务

这个脚本不是单独安装就能工作的云端脚本。它只负责浏览器里的划词按钮和交互,需要你先在电脑上启动本地服务:
当前用户脚本版本为 `1.13.0`。它不是单独安装就能工作的云端脚本:浏览器端始终需要本项目的本地 FastAPI 中介服务。

1. 安装并启动本项目的本地 FastAPI 服务
2. 朗读需要 Kokoro TTS 环境
3. 翻译需要安装 Ollama并拉取本地模型,例如:
1. 按项目 README 完成环境安装,并至少启动本地 FastAPI 服务
2. `Read` 需要 Kokoro TTS 环境;Kokoro 会在第一次朗读时按需加载,不会因仅启动 API 或仅使用远程翻译而占用本地 GPU。
3. 本机翻译需要安装 Ollama 并拉取本地模型,例如:

```powershell
ollama pull translategemma:4b
# 可选更大模型
ollama pull qwen3:14b
```

翻译、朗读稿准备和复杂公式口语化默认都使用 `translategemma:4b`。可在服务端通过 `OLLAMA_TRANSLATE_MODEL`、`OLLAMA_READ_MODEL`、`OLLAMA_FORMULA_MODEL` 覆盖,也可在脚本设置里切换当前翻译/朗读准备模型。如果第一次变慢,通常是 Ollama 正在加载模型。设置栏里的 **Keep loaded** 会用 Ollama `keep_alive: -1m` 常驻当前模型,**Unload** 会用 `keep_alive: 0` 卸载模型释放显存。4B 模型的翻译和公式朗读不参考上下文,公式朗读也会优先采用保守字面规则;14B 模型会保留更多上下文。使用 `qwen3:14b`、QwQ、DeepSeek-R1 等推理模型时,服务端会自动向 Ollama 传入 `think: false`,让翻译和朗读准备直接输出结果。
4. 如果只使用项目服务器翻译,本机可以不安装 Ollama;但必须从托盘菜单 `Remote Service` 保存并连接服务器,然后在网页设置中点击 **Use project server**。

托盘程序会为当前 Windows 用户注册 `localreadtranslate://start`。如果 **Start local service** 无法唤起托盘程序,在项目目录执行:

```powershell
conda run -n kokoro-tts python windows_protocol.py register
```

注册记录使用绝对路径;移动项目后需要重新执行。注册在 `HKCU` 下,不需要管理员权限。网页发起协议操作时,浏览器可能要求确认打开外部应用;协议只支持固定的 `start` 操作,不携带远程凭据或模型参数。

翻译、朗读稿准备和复杂公式口语化默认使用 `translategemma:4b`。可在服务端通过 `OLLAMA_TRANSLATE_MODEL`、`OLLAMA_READ_MODEL`、`OLLAMA_FORMULA_MODEL` 覆盖,也可在脚本设置里切换模型。设置栏里的 **Keep loaded** 会在当前模型所属来源上常驻模型,**Unload** 会从同一来源卸载模型释放显存。4B 模型的翻译和公式朗读不参考上下文,公式朗读也会优先采用保守字面规则;14B 模型会保留更多上下文。使用 `qwen3:14b`、QwQ、DeepSeek-R1 等推理模型时,服务端会自动向 Ollama 传入 `think: false`。
数学符号读法可在项目的 `config/math_glossary.json` 中调整,当前覆盖箭头、上下标、集合、逻辑、求和、积分、偏导等常见论文符号。

## 隐私说明

脚本只请求本机地址
浏览器脚本只请求本机地址

```text
http://127.0.0.1:5000
```

选中文本不会被发送到外部云端服务。朗读和翻译都在你的电脑本地完成。
浏览器不会获得 SSH 密码、密钥路径或远程 Ollama 地址。本机模型模式下,朗读、翻译和允许的上下文都留在本机;当你明确选择项目服务器模型时,选中文本和允许的附近上下文会由本地 FastAPI 中介发送到你配置的服务器。

远程连接支持 SSH 隧道和 Direct API:

- SSH 模式优先使用 SSH agent、默认密钥或指定密钥文件,只有密钥认证失败且填写了密码时才回退到密码认证。客户端加载系统/OpenSSH 主机密钥,并拒绝 `known_hosts` 中不存在的主机。
- 如果填写了 SSH 密码,它会以明文保存在 Git 已忽略的 `tray_settings.json` 中。请保护 Windows 账户和项目目录,优先使用 agent/密钥,并且不要同步、提交或分享该文件。
- Direct API 只支持原生 Ollama API;当前实现不会添加 API key 或其他认证请求头。常见 `http://` 地址传输不加密,只应在可信局域网或 VPN 内使用,不应直接暴露到公网。
- Ollama 请求会绕过环境中的 HTTP 代理,避免局域网请求和选中文本经过无关代理。
- SSH 主机身份采用失败即关闭策略:客户端调用 `load_system_host_keys()` 并使用 Paramiko `RejectPolicy`。未知主机必须先通过可信渠道核对指纹,再加入 `known_hosts`;本机的 `10.12.96.203` 已按此策略实机重连成功。

## 常见问题

### 安装后没有反应

先确认本地服务已启动
先点击设置面板中的 **Start local service**,接受浏览器的外部应用确认,然后检查

```text
http://127.0.0.1:5000/health
```

如果打不开,先运行项目里的 `start.bat` 或推荐的托盘启动器 `Kokoro TTS.bat`。
如果打不开,先运行推荐的托盘启动器 `Kokoro TTS.bat`,或用上面的 `windows_protocol.py register` 命令修复协议。`start.bat` 只启动裸 FastAPI,不负责远程 SSH 隧道或协议唤起;需要项目服务器时必须使用托盘程序
新版 `start.bat` 和 `Kokoro TTS.bat` 会直接定位 `kokoro-tts` 环境里的 Python,不需要先执行 `conda init`;`Kokoro TTS.pyw` 只在 Windows 已有关联 `.pyw` 到 Python 时适合双击。

### **Use project server** 提示没有可用模型

从托盘菜单打开 `Remote Service`,保存并连接服务器,确认检查成功后再回网页点击该按钮。远程主机和凭据不能在油猴脚本里配置。

### **Initialize local model** 提示不能初始化远端模型

这个按钮只用于本机 Ollama。先在模型列表选择一个本机模型并确保 Ollama 正在运行;远端模型请先在托盘中连接,再使用 **Use project server**、**Keep loaded** 或普通翻译请求。

### 翻译健康检测失败

通常是浏览器脚本已更新,但本地后台服务还没重启到最新版。重启本地服务后再刷新网页。
Expand All @@ -82,6 +110,10 @@ http://127.0.0.1:5000/health

Ollama 第一次使用某个模型时需要把模型加载到 GPU/内存,之后同一模型会快很多。

## 更新与发布

仓库中的 `tts-userscript.js` 与浏览器已安装副本是两份文件。发布者应递增 `@version`,运行项目测试并推送,确认 GitHub Raw 地址返回新版本,再从 Tampermonkey 执行“检查用户脚本更新”;Greasy Fork 也必须发布同一个版本号和本附加说明。仅修改本地仓库不会自动更新浏览器脚本。

## 项目地址

- GitHub: https://github.com/Yan-ShiBo/LocalReadTranslate
Expand Down
100 changes: 100 additions & 0 deletions docs/iteration-4-2026-07-18.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# Iteration 4 Release Record — 2026-07-18

**Status:** Implementation and release verification complete.

## Release Goal

Make local/remote translation source control explicit in the userscript, add a one-click Windows path for starting the local mediator, keep remote credentials out of the browser, and document the security and update boundaries accurately.

## Shipped Behavior

### Browser Controls

The userscript is version `1.13.0` and continues to call only `http://127.0.0.1:5000`. Its settings panel now exposes three distinct actions:

- **Use project server** selects an available `remote:` model, persists it, and checks its health. If the local service has no remote choices, the UI directs the user to configure and connect `Remote Service` in the tray.
- **Initialize local model** accepts only a local model reference and sends a source-aware keepalive request. It does not attempt to initialize a remote model.
- **Start local service** opens the fixed `localreadtranslate://start` action and polls local health for about 20 seconds.

The local/remote boundary is explicit: an unavailable selected local model remains selected and reports an error. The script never switches to a `remote:` model because of an ordinary health or translation failure; remote processing begins only after **Use project server** or a manual remote selection.

### Windows Protocol and Tray Ownership

`windows_protocol.py` registers or repairs `localreadtranslate://start` under the current user's `HKCU\Software\Classes\localreadtranslate` tree. Registration needs no administrator rights and stores quoted absolute paths to `pythonw.exe` and `tray_app.py`. Moving the checkout or environment requires:

```powershell
conda run -n kokoro-tts python windows_protocol.py register
```

The application recognizes only the exact `start` action as a protocol start request; alternate hosts/actions, extra paths, query strings, and fragments are not interpreted. It does not transport a server address, credentials, model selection, shell fragment, or arbitrary command.

The tray repairs registration during ordinary startup. A Windows named mutex prevents duplicate tray instances. A first protocol invocation launches the tray and server; when the tray already exists, the new process signals a named auto-reset event and exits, and the existing tray starts its server.

The tray owns remote settings, SSH/API validation, tunnel lifecycle, hidden FastAPI launch, and the credential-free `KOKORO_OLLAMA_SOURCES` environment. `start.bat` remains a bare FastAPI launch and does not provide a remote SSH tunnel.

### Source-Aware Ollama Routing

Plain Ollama model names continue to target local Ollama. Remote choices use `remote:<source-id>:<model-name>` internally. Model discovery, translation, read preparation, formula verbalization, health, keepalive, unload, and pinned-model state resolve the same source-aware reference.

The browser receives display labels and model references, not remote connection details. The server receives only each source's id, display name, and effective base URL; SSH credentials remain in the tray process and settings file.

All Ollama requests bypass ambient HTTP proxy settings. This keeps loopback and trusted-LAN traffic from being redirected through an unrelated proxy.

### Lazy Local TTS

FastAPI startup no longer implies Kokoro initialization. Kokoro loads on the first TTS request, while translation-only use—including project-server translation—can run without allocating local TTS GPU memory. `/health` separates API readiness from `tts_model_loaded`.

## Security and Privacy Boundary

- In local-model mode, selected text and allowed context remain on the local machine.
- After the user selects a project-server model, selected text and allowed context are sent by the local mediator to that configured server.
- The userscript never receives SSH passwords, key paths, remote hosts, or remote Ollama URLs.
- An optional SSH password is persisted as plaintext in the Git-ignored `tray_settings.json`. Users should prefer SSH agent/key authentication, protect the Windows account and project directory, and never sync, commit, or share this file.
- SSH host identity is fail-closed: the client loads system/OpenSSH host keys and uses Paramiko `RejectPolicy`. Unknown hosts must be added to `known_hosts` only after out-of-band fingerprint verification. The configured `10.12.96.203` entry exists on this machine and a real reconnect succeeded.
- Direct API mode targets the native Ollama API and does not add API-key or other authentication headers. Plain HTTP is unencrypted; this mode is for a trusted LAN or VPN, never a publicly exposed Ollama port.
- Any page can ask a browser to open a registered external protocol. The browser confirmation is therefore a user-consent boundary even though this protocol's only accepted operation is the fixed `start` action.

## Tampermonkey Release Flow

The repository copy and the browser-installed copy are separate. For each userscript release:

1. Load the current `tts-userscript.js` into Tampermonkey and validate the affected pages.
2. Increment `@version`; an installed script will not update to a different file with the same version.
3. Run the full Python and Node verification suites plus syntax, dependency, FFmpeg, voice-catalog, and diff checks.
4. Push the tested commit and confirm the GitHub Raw URL serves the new version.
5. Run Tampermonkey's **Check for updates** and confirm the installed version.
6. Publish the same version and updated additional information on Greasy Fork.

## Verification

Automated release commands:

```powershell
conda run -n kokoro-tts python -m pytest tests -v
conda run -n kokoro-tts python -m py_compile server.py audio_encoding.py tray_app.py "Kokoro TTS.pyw" tts_catalog.py windows_protocol.py windows_runtime.py windows_startup.py scripts/sync_catalog.py
node --check tts-userscript.js
node --test tests/userscript-core.test.cjs
conda run -n kokoro-tts python scripts/sync_catalog.py --check
conda run -n kokoro-tts python -c "from audio_encoding import validate_ffmpeg; validate_ffmpeg()"
conda run -n kokoro-tts python -m pip check
git diff --check
```

Final automated results: **Python 196 passed + 15 subtests; Node 33/33 passed.**

Completed real-machine checks:

- first invocation of `localreadtranslate://start` launched the tray/server path;
- invoking the protocol while the tray was already running woke the existing tray through the named event without creating a second instance;
- `/health` returned an API-ready response with `tts_model_loaded=false`, confirming lazy TTS startup;
- SSH reconnected to `10.12.96.203` with the system-known-hosts + `RejectPolicy` policy;
- the remote source health check reported `qwen3:30b` available;
- regression tests confirmed that a failed SSH reconnect cannot publish a stale persisted forwarding port and that an unrelated HTTP service on port 5000 cannot satisfy the userscript's Kokoro health check.

## Documentation Updated

- `README.md` — architecture, startup, remote security, userscript controls, update flow, troubleshooting, and verification.
- `说明.md` — the same operational contract in Chinese.
- `docs/greasyfork-additional-info.md` — install-time requirements, explicit controls, privacy boundaries, and user-facing troubleshooting.
- `docs/superpowers/specs/2026-06-27-remote-ollama-service-design.md` — current protocol, tray, routing, authentication, and error contracts.
- Historical iteration 3 and remote-service implementation plans — archive warnings added without rewriting their recorded versions, paths, or expected counts.
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# Kokoro TTS Third Iteration Implementation Plan

> [!WARNING]
> **Archived implementation plan.** This file preserves the original RED/GREEN steps, user-specific paths, dependency versions, and expected test counts for historical traceability; do not execute it against the current checkout. The former `D:/local-tts-env` path and embedded interpreter paths are historical. Use the current [README](../../../README.md) and [iteration 4 release record](../../iteration-4-2026-07-18.md) instead.

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add continuous WebM/Opus streaming, complete OGG/Opus responses, honest playback progress, tray login auto-start, and full boundary/concurrency coverage while preserving WAV compatibility.
Expand Down
3 changes: 3 additions & 0 deletions docs/superpowers/plans/2026-06-27-remote-ollama-service.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,8 @@
# Remote Ollama Service Implementation Plan

> [!WARNING]
> **Archived implementation plan.** This file preserves the original RED/GREEN steps, paths, versions, and expected test counts for historical traceability; do not execute it against the current checkout. The former `D:/local-tts-env` path no longer describes this project. Use the current [README](../../../README.md), [remote-service design](../specs/2026-06-27-remote-ollama-service-design.md), and [iteration 4 release record](../../iteration-4-2026-07-18.md) instead.

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Add a tray-configured remote Ollama source so the browser can choose either local Ollama models or models exposed through an SSH tunnel to a LAN server.
Expand Down
Loading
Loading