diff --git a/README.md b/README.md index 9320aa0..7586e31 100644 --- a/README.md +++ b/README.md @@ -102,6 +102,12 @@ The default Ollama model is `translategemma:4b` for translation, read preparatio Use **Keep loaded** in the Translation settings when you plan to translate or read many selections with the same Ollama model. This preloads the model with `keep_alive: -1m` and keeps sending that setting for the pinned model, avoiding repeated first-token delays. Use **Unload** when you are done to release VRAM. +### Remote Ollama over LAN + +Right-click the Kokoro TTS tray icon and choose `Remote Service`. Enter the server name, IP, SSH port, username, password, and remote Ollama host/port. The app creates a local SSH tunnel to the server's native Ollama API, then the browser model selector will show models as `Server Name / model`. + +The browser script never receives the SSH password. For convenience, the tray app stores the remote profile in `tray_settings.json`. + Formula wording is guided by `config/math_glossary.json`. Each symbol can define a direct reading, read-aloud defaults and contextual readings, for example right arrow can mean `maps to`, `approaches`, `implies`, `gives`, or simply `right arrow`. Local rules choose common cases first. For 4B models, the formula read-aloud path deliberately prefers these literal rules and omits formula context where possible; the same glossary is included in Ollama prompts only for harder formulas. ### 4. Install the Browser Script diff --git a/docs/superpowers/plans/2026-06-27-remote-ollama-service.md b/docs/superpowers/plans/2026-06-27-remote-ollama-service.md new file mode 100644 index 0000000..7d63205 --- /dev/null +++ b/docs/superpowers/plans/2026-06-27-remote-ollama-service.md @@ -0,0 +1,854 @@ +# Remote Ollama Service Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add a tray-configured remote Ollama source so the browser can choose either local Ollama models or models exposed through an SSH tunnel to a LAN server. + +**Architecture:** Keep the browser talking only to the local FastAPI server. The tray app owns SSH login, starts a local TCP tunnel, and passes a password-free source list to the server through `KOKORO_OLLAMA_SOURCES`. The server routes model references like `remote:lab-server:qwen3:14b` to the correct Ollama base URL while plain model names continue using local Ollama. + +**Tech Stack:** Python 3.10, FastAPI, Pydantic, Paramiko for SSH tunneling, Tkinter for the tray dialog, Node test runner for userscript core tests. + +--- + +## File Structure + +- Modify `D:/local-tts-env/server.py`: add Ollama source/model-reference parsing, source-aware request routing, source-aware health metadata, and pinned model isolation. +- Modify `D:/local-tts-env/tests/test_server.py`: cover local compatibility, remote model parsing, remote source listing, remote request routing, and source-specific pinning. +- Modify `D:/local-tts-env/tray_app.py`: add remote settings defaults, SSH tunnel manager, environment payload builder, menu entry, and Tkinter connection dialog. +- Modify `D:/local-tts-env/tests/test_tray_app.py`: cover default settings and source environment payload behavior without opening a real SSH connection. +- Modify `D:/local-tts-env/tts-userscript.js`: append remote model options from health metadata and expose small pure helpers for tests. +- Modify `D:/local-tts-env/tests/userscript-core.test.cjs`: cover remote model option merging and selected remote model preservation. +- Modify `D:/local-tts-env/requirements.txt`: add `paramiko>=3.5,<5`. +- Modify `D:/local-tts-env/README.md` and `D:/local-tts-env/说明.md`: document the remote service flow. + +## Task 1: Server Model Sources + +**Files:** +- Modify: `D:/local-tts-env/server.py` +- Test: `D:/local-tts-env/tests/test_server.py` + +- [ ] **Step 1: Write failing tests for model-reference parsing and source options** + +Add these tests to `ApiTests` in `D:/local-tts-env/tests/test_server.py`: + +```python + def test_remote_model_reference_resolves_source_and_model(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource( + id="local", + name="Local Ollama", + base_url="http://127.0.0.1:11434", + remote=False, + ), + "lab-server": server.OllamaSource( + id="lab-server", + name="Lab Server", + base_url="http://127.0.0.1:49152", + remote=True, + ), + } + try: + resolved = server._resolve_ollama_model_ref("remote:lab-server:qwen3:14b") + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual(resolved.value, "remote:lab-server:qwen3:14b") + self.assertEqual(resolved.model, "qwen3:14b") + self.assertEqual(resolved.source.id, "lab-server") + self.assertEqual(resolved.source.base_url, "http://127.0.0.1:49152") + + def test_plain_model_reference_uses_local_source(self): + resolved = server._resolve_ollama_model_ref("translategemma:4b") + + self.assertEqual(resolved.value, "translategemma:4b") + self.assertEqual(resolved.model, "translategemma:4b") + self.assertEqual(resolved.source.id, "local") + + def test_model_options_include_remote_source_labels(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource("local", "Local Ollama", "http://127.0.0.1:11434", False), + "lab-server": server.OllamaSource("lab-server", "Lab Server", "http://127.0.0.1:49152", True), + } + + def fake_json(path, timeout=5.0, base_url=None): + if path == "/api/tags" and base_url == "http://127.0.0.1:49152": + return {"models": [{"name": "qwen3:14b"}]} + if path == "/api/tags": + return {"models": [{"name": "translategemma:4b"}]} + raise AssertionError((path, base_url)) + + try: + with patch.object(server, "_call_ollama_json", side_effect=fake_json): + options = server._collect_ollama_model_options() + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual( + options, + [ + { + "value": "translategemma:4b", + "label": "Local Ollama / translategemma:4b", + "source": "local", + "source_name": "Local Ollama", + "model": "translategemma:4b", + }, + { + "value": "remote:lab-server:qwen3:14b", + "label": "Lab Server / qwen3:14b", + "source": "lab-server", + "source_name": "Lab Server", + "model": "qwen3:14b", + }, + ], + ) +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: + +```powershell +python -m pytest tests/test_server.py::ApiTests::test_remote_model_reference_resolves_source_and_model tests/test_server.py::ApiTests::test_plain_model_reference_uses_local_source tests/test_server.py::ApiTests::test_model_options_include_remote_source_labels -q +``` + +Expected: FAIL because `OllamaSource`, `_resolve_ollama_model_ref`, and `_collect_ollama_model_options` do not exist. + +- [ ] **Step 3: Implement source data structures and parsing** + +Add this near the existing Ollama constants in `D:/local-tts-env/server.py`: + +```python +from dataclasses import dataclass + + +@dataclass(frozen=True) +class OllamaSource: + id: str + name: str + base_url: str + remote: bool = False + + +@dataclass(frozen=True) +class OllamaModelRef: + value: str + model: str + source: OllamaSource +``` + +Add these helpers below the Ollama constants: + +```python +def _local_ollama_source() -> OllamaSource: + return OllamaSource("local", "Local Ollama", OLLAMA_BASE_URL, False) + + +def _load_ollama_sources_from_env(raw: Optional[str] = None) -> dict[str, OllamaSource]: + sources = {"local": _local_ollama_source()} + raw_value = os.environ.get("KOKORO_OLLAMA_SOURCES", "") if raw is None else raw + if not raw_value: + return sources + try: + items = json.loads(raw_value) + except json.JSONDecodeError: + return sources + if not isinstance(items, list): + return sources + for item in items: + if not isinstance(item, dict): + continue + source_id = str(item.get("id") or "").strip() + name = str(item.get("name") or source_id).strip() + base_url = str(item.get("base_url") or "").strip().rstrip("/") + if not source_id or source_id == "local" or not base_url: + continue + if any(ch.isspace() for ch in source_id): + continue + sources[source_id] = OllamaSource(source_id, name or source_id, base_url, True) + return sources + + +OLLAMA_SOURCES = _load_ollama_sources_from_env() + + +def _model_value_for_source(source: OllamaSource, model: str) -> str: + return model if source.id == "local" else f"remote:{source.id}:{model}" + + +def _resolve_ollama_model_ref(value: Optional[str]) -> OllamaModelRef: + selected = (value or OLLAMA_TRANSLATE_MODEL).strip() or OLLAMA_TRANSLATE_MODEL + if selected.startswith("remote:"): + parts = selected.split(":", 2) + if len(parts) != 3 or not parts[1] or not parts[2]: + raise RuntimeError("Invalid remote Ollama model reference") + source = OLLAMA_SOURCES.get(parts[1]) + if source is None: + raise RuntimeError("Remote Ollama source is not configured") + return OllamaModelRef(_model_value_for_source(source, parts[2]), parts[2], source) + source = OLLAMA_SOURCES["local"] + return OllamaModelRef(selected, selected, source) + + +def _collect_ollama_model_options() -> list[dict[str, str]]: + options: list[dict[str, str]] = [] + for source in OLLAMA_SOURCES.values(): + try: + payload = _call_ollama_json( + "/api/tags", + base_url=None if source.id == "local" else source.base_url, + ) + except Exception: + continue + for model in _ollama_model_names(payload): + value = _model_value_for_source(source, model) + options.append( + { + "value": value, + "label": f"{source.name} / {model}", + "source": source.id, + "source_name": source.name, + "model": model, + } + ) + return options +``` + +- [ ] **Step 4: Run tests to verify they pass** + +Run: + +```powershell +python -m pytest tests/test_server.py::ApiTests::test_remote_model_reference_resolves_source_and_model tests/test_server.py::ApiTests::test_plain_model_reference_uses_local_source tests/test_server.py::ApiTests::test_model_options_include_remote_source_labels -q +``` + +Expected: PASS. + +## Task 2: Server Source-Aware Ollama Requests + +**Files:** +- Modify: `D:/local-tts-env/server.py` +- Test: `D:/local-tts-env/tests/test_server.py` + +- [ ] **Step 1: Write failing tests for remote request routing and source-aware pinning** + +Add these tests to `ApiTests`: + +```python + def test_remote_translate_routes_generate_to_remote_base_url(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource("local", "Local Ollama", "http://127.0.0.1:11434", False), + "lab-server": server.OllamaSource("lab-server", "Lab Server", "http://127.0.0.1:49152", True), + } + captured = {} + + def fake_urlopen(request, timeout): + captured["url"] = request.full_url + captured["payload"] = json.loads(request.data.decode("utf-8")) + return FakeUrlopenResponse({"response": "你好"}) + + try: + with patch.object(server.urllib_request, "urlopen", side_effect=fake_urlopen): + result = server._call_ollama_translate_raw( + "Hello", + "remote:lab-server:qwen3:14b", + "Simplified Chinese", + None, + ) + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual(result, "你好") + self.assertEqual(captured["url"], "http://127.0.0.1:49152/api/generate") + self.assertEqual(captured["payload"]["model"], "qwen3:14b") + + def test_remote_pinned_model_generation_requests_keep_alive(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource("local", "Local Ollama", "http://127.0.0.1:11434", False), + "lab-server": server.OllamaSource("lab-server", "Lab Server", "http://127.0.0.1:49152", True), + } + server.PINNED_OLLAMA_MODELS.add("remote:lab-server:qwen3:14b") + captured = {} + + def fake_urlopen(request, timeout): + captured["payload"] = json.loads(request.data.decode("utf-8")) + return FakeUrlopenResponse({"response": "你好"}) + + try: + with patch.object(server.urllib_request, "urlopen", side_effect=fake_urlopen): + server._call_ollama_translate_raw("Hello", "remote:lab-server:qwen3:14b", "Simplified Chinese") + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual(captured["payload"]["keep_alive"], server.OLLAMA_KEEP_ALIVE_PIN_VALUE) +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: + +```powershell +python -m pytest tests/test_server.py::ApiTests::test_remote_translate_routes_generate_to_remote_base_url tests/test_server.py::ApiTests::test_remote_pinned_model_generation_requests_keep_alive -q +``` + +Expected: FAIL because generation still calls `OLLAMA_BASE_URL` and sends the prefixed model name. + +- [ ] **Step 3: Implement source-aware URL and payload routing** + +Update `_call_ollama_json` to accept `base_url`: + +```python +def _call_ollama_json(path: str, timeout: float = 5.0, base_url: Optional[str] = None): + selected_base_url = (base_url or OLLAMA_BASE_URL).rstrip("/") + req = urllib_request.Request( + f"{selected_base_url}{path}", + headers={"Accept": "application/json"}, + method="GET", + ) +``` + +Add: + +```python +def _ollama_generate_url(ref: OllamaModelRef) -> str: + return f"{ref.source.base_url}/api/generate" +``` + +Then in `_call_ollama_model_keep_alive`, `_call_ollama_translate_raw`, `_call_ollama_text_generation`, and `_call_ollama_formula_verbalization`: + +```python +ref = _resolve_ollama_model_ref(model) +payload["model"] = ref.model +_prepare_ollama_generate_payload(payload, ref.value) +req = urllib_request.Request( + _ollama_generate_url(ref), + data=data, + headers={"Content-Type": "application/json"}, + method="POST", +) +``` + +Update `_apply_ollama_thinking_mode` to inspect the clean model name: + +```python +def _apply_ollama_thinking_mode(payload: dict, model: Optional[str]) -> None: + clean_model = _resolve_ollama_model_ref(model).model if model else model + if _ollama_should_disable_thinking(clean_model): + payload["think"] = False +``` + +Keep `_apply_ollama_keep_alive` checking the canonical full reference: + +```python +def _apply_ollama_keep_alive(payload: dict, model: Optional[str]) -> None: + if not model: + return + selected_model = _resolve_ollama_model_ref(model).value + if selected_model in PINNED_OLLAMA_MODELS: + payload["keep_alive"] = OLLAMA_KEEP_ALIVE_PIN_VALUE +``` + +- [ ] **Step 4: Run focused server tests** + +Run: + +```powershell +python -m pytest tests/test_server.py::ApiTests::test_remote_translate_routes_generate_to_remote_base_url tests/test_server.py::ApiTests::test_remote_pinned_model_generation_requests_keep_alive tests/test_server.py::ApiTests::test_translate_uses_local_ollama_settings tests/test_server.py::ApiTests::test_pinned_model_generation_requests_keep_alive -q +``` + +Expected: PASS. + +## Task 3: Server Health and Residency APIs + +**Files:** +- Modify: `D:/local-tts-env/server.py` +- Test: `D:/local-tts-env/tests/test_server.py` + +- [ ] **Step 1: Write failing tests for health metadata and remote keepalive** + +Add: + +```python + def test_translate_health_reports_remote_source_metadata(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource("local", "Local Ollama", "http://127.0.0.1:11434", False), + "lab-server": server.OllamaSource("lab-server", "Lab Server", "http://127.0.0.1:49152", True), + } + + def fake_json(path, timeout=5.0, base_url=None): + if base_url == "http://127.0.0.1:49152" and path == "/api/tags": + return {"models": [{"name": "qwen3:14b"}]} + if base_url == "http://127.0.0.1:49152" and path == "/api/ps": + return {"models": [{"name": "qwen3:14b"}]} + if path == "/api/tags": + return {"models": [{"name": "translategemma:4b"}]} + if path == "/api/ps": + return {"models": []} + raise AssertionError((path, base_url)) + + try: + with patch.object(server, "_call_ollama_json", side_effect=fake_json): + response = self.client.get("/translate/health?model=remote:lab-server:qwen3:14b") + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual(response.status_code, 200) + payload = response.json() + self.assertEqual(payload["source"], "lab-server") + self.assertEqual(payload["source_name"], "Lab Server") + self.assertEqual(payload["model"], "remote:lab-server:qwen3:14b") + self.assertTrue(payload["model_running"]) + self.assertIn( + { + "value": "remote:lab-server:qwen3:14b", + "label": "Lab Server / qwen3:14b", + "source": "lab-server", + "source_name": "Lab Server", + "model": "qwen3:14b", + }, + payload["available_model_options"], + ) +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```powershell +python -m pytest tests/test_server.py::ApiTests::test_translate_health_reports_remote_source_metadata -q +``` + +Expected: FAIL because the health response has no source metadata. + +- [ ] **Step 3: Extend response models and health logic** + +Add: + +```python +class OllamaModelOption(BaseModel): + value: str + label: str + source: str + source_name: str + model: str +``` + +Update `TranslateHealthResponse`: + +```python +class TranslateHealthResponse(BaseModel): + status: str + ollama_reachable: bool + model: str + model_available: bool + model_running: bool + model_pinned: bool = False + available_models: list[str] + running_models: list[str] + source: str = "local" + source_name: str = "Local Ollama" + available_model_options: list[OllamaModelOption] = Field(default_factory=list) + error: Optional[str] = None +``` + +In `translate_health`, resolve the selected model first: + +```python +ref = _resolve_ollama_model_ref(model or OLLAMA_TRANSLATE_MODEL) +base_url = None if ref.source.id == "local" else ref.source.base_url +available_models = _ollama_model_names(_call_ollama_json("/api/tags", base_url=base_url)) +running_models = _ollama_model_names(_call_ollama_json("/api/ps", base_url=base_url)) +``` + +Return `model=ref.value`, `source=ref.source.id`, `source_name=ref.source.name`, `model_available=ref.model in available_models`, and `model_pinned=ref.value in PINNED_OLLAMA_MODELS`. + +- [ ] **Step 4: Run health tests** + +Run: + +```powershell +python -m pytest tests/test_server.py::ApiTests::test_translate_health_reports_remote_source_metadata tests/test_server.py::ApiTests::test_translate_health_reports_model_state tests/test_server.py::ApiTests::test_translate_health_handles_ollama_offline -q +``` + +Expected: PASS. + +## Task 4: Tray Remote Settings and Tunnel Payload + +**Files:** +- Modify: `D:/local-tts-env/tray_app.py` +- Modify: `D:/local-tts-env/requirements.txt` +- Test: `D:/local-tts-env/tests/test_tray_app.py` + +- [ ] **Step 1: Write failing tests for settings and server environment payload** + +Add: + +```python +class TrayRemoteOllamaTests(unittest.TestCase): + def test_default_settings_include_remote_ollama(self): + with patch.object( + tray_app, + "SETTINGS_FILE", + Path("__missing_tray_settings_for_test__.json"), + ): + settings = tray_app.load_settings() + + self.assertEqual( + settings["remote_ollama"], + { + "enabled": False, + "name": "", + "host": "", + "ssh_port": 22, + "username": "", + "password": "", + "ollama_host": "127.0.0.1", + "ollama_port": 11434, + "local_port": 0, + }, + ) + + def test_remote_source_env_omits_password(self): + app = TrayAutoStartTests().make_app() + app.settings["remote_ollama"] = { + "enabled": True, + "name": "Lab Server", + "host": "192.168.1.10", + "ssh_port": 22, + "username": "alice", + "password": "secret", + "ollama_host": "127.0.0.1", + "ollama_port": 11434, + "local_port": 49152, + } + app.remote_tunnel_local_port = 49152 + + payload = app.build_remote_ollama_sources_env() + + self.assertEqual( + json.loads(payload), + [ + { + "id": "lab-server", + "name": "Lab Server", + "base_url": "http://127.0.0.1:49152", + } + ], + ) + self.assertNotIn("secret", payload) +``` + +- [ ] **Step 2: Run tests to verify they fail** + +Run: + +```powershell +python -m pytest tests/test_tray_app.py::TrayRemoteOllamaTests -q +``` + +Expected: FAIL because remote defaults and payload builder do not exist. + +- [ ] **Step 3: Implement remote settings defaults and payload builder** + +Add: + +```python +def default_remote_ollama_settings(): + return { + "enabled": False, + "name": "", + "host": "", + "ssh_port": 22, + "username": "", + "password": "", + "ollama_host": "127.0.0.1", + "ollama_port": 11434, + "local_port": 0, + } + + +def slugify_source_id(value): + cleaned = "".join(ch.lower() if ch.isalnum() else "-" for ch in value.strip()) + cleaned = "-".join(part for part in cleaned.split("-") if part) + return cleaned or "remote-server" +``` + +In `load_settings`, merge `remote_ollama` with `default_remote_ollama_settings()`. + +In `TrayApp.__init__`, add: + +```python +self.remote_tunnel = None +self.remote_tunnel_local_port = None +``` + +Add: + +```python +def build_remote_ollama_sources_env(self): + remote = self.settings.get("remote_ollama") or {} + if not remote.get("enabled"): + return "" + local_port = self.remote_tunnel_local_port or int(remote.get("local_port") or 0) + if local_port <= 0: + return "" + name = (remote.get("name") or remote.get("host") or "Remote Ollama").strip() + source_id = slugify_source_id(name) + return json.dumps( + [ + { + "id": source_id, + "name": name, + "base_url": f"http://127.0.0.1:{local_port}", + } + ], + ensure_ascii=False, + ) +``` + +In `start_server`, add: + +```python +sources_env = self.build_remote_ollama_sources_env() +if sources_env: + env["KOKORO_OLLAMA_SOURCES"] = sources_env +``` + +Add `paramiko>=3.5,<5` to `requirements.txt`. + +- [ ] **Step 4: Run tray tests** + +Run: + +```powershell +python -m pytest tests/test_tray_app.py::TrayRemoteOllamaTests tests/test_tray_app.py::TrayAutoStartTests -q +``` + +Expected: PASS. + +## Task 5: Tray SSH Tunnel and Dialog + +**Files:** +- Modify: `D:/local-tts-env/tray_app.py` + +- [ ] **Step 1: Implement tunnel manager with lazy Paramiko import** + +Add a `RemoteOllamaTunnel` class that: + +```python +class RemoteOllamaTunnel: + def __init__(self, settings): + self.settings = settings + self.client = None + self.server = None + self.thread = None + self.local_port = 0 + + def start(self): + import paramiko + # Connect with password, no agent, no key lookup. + # Start a local threaded TCP server forwarding to remote ollama_host:ollama_port. + # Return the selected local port. + + def stop(self): + # Shutdown TCP server and close SSH client if present. +``` + +Use `socketserver.ThreadingTCPServer`, `select.select`, and `paramiko.Transport.open_channel("direct-tcpip", ...)` for forwarding. Import Paramiko inside `start()` so tests can import `tray_app.py` even before dependencies are installed. + +- [ ] **Step 2: Add tray methods** + +Add: + +```python +def connect_remote_ollama(self): + self.disconnect_remote_ollama(restart=False) + tunnel = RemoteOllamaTunnel(self.settings["remote_ollama"]) + local_port = tunnel.start() + self.remote_tunnel = tunnel + self.remote_tunnel_local_port = local_port + self.settings["remote_ollama"]["enabled"] = True + self.settings["remote_ollama"]["local_port"] = local_port + save_settings(self.settings) + self.restart_server() + + +def disconnect_remote_ollama(self, restart=True): + if self.remote_tunnel: + self.remote_tunnel.stop() + self.remote_tunnel = None + self.remote_tunnel_local_port = None + if restart: + self.settings["remote_ollama"]["enabled"] = False + save_settings(self.settings) + self.restart_server() +``` + +- [ ] **Step 3: Add Tkinter dialog and tray menu item** + +Add `open_remote_service_settings` that creates a small Tkinter dialog with fields for name, host, SSH port, username, password, Ollama host, and Ollama port. On `Connect`, update `self.settings["remote_ollama"]`, call `connect_remote_ollama`, and close the dialog. On `Disconnect`, call `disconnect_remote_ollama` and close the dialog. + +Add to `_build_menu` near the project/log items: + +```python +Item("Remote Service", self.open_remote_service_settings), +``` + +- [ ] **Step 4: Run tray tests and import check** + +Run: + +```powershell +python -m pytest tests/test_tray_app.py -q +python -c "import tray_app; print('tray import ok')" +``` + +Expected: PASS and `tray import ok`. + +## Task 6: Browser Remote Model Options + +**Files:** +- Modify: `D:/local-tts-env/tts-userscript.js` +- Test: `D:/local-tts-env/tests/userscript-core.test.cjs` + +- [ ] **Step 1: Write failing userscript helper tests** + +Add: + +```javascript +test("model option merge includes remote health metadata", () => { + const { mergeTranslationModelOptions } = require("../tts-userscript.js"); + const merged = mergeTranslationModelOptions( + [{ value: "translategemma:4b", label: "translategemma:4b - default" }], + { + available_models: ["translategemma:4b"], + available_model_options: [ + { + value: "remote:lab-server:qwen3:14b", + label: "Lab Server / qwen3:14b", + source: "lab-server", + source_name: "Lab Server", + model: "qwen3:14b", + }, + ], + }, + "remote:lab-server:qwen3:14b" + ); + + assert.deepEqual(merged, [ + { value: "translategemma:4b", label: "translategemma:4b - default" }, + { value: "remote:lab-server:qwen3:14b", label: "Lab Server / qwen3:14b" }, + ]); +}); +``` + +- [ ] **Step 2: Run test to verify it fails** + +Run: + +```powershell +node --test tests/userscript-core.test.cjs +``` + +Expected: FAIL because `mergeTranslationModelOptions` is not exported. + +- [ ] **Step 3: Implement and export pure merge helper** + +Inside the `KokoroTTSCore` block add: + +```javascript +function mergeTranslationModelOptions(baseOptions, payload, selectedValue) { + const merged = []; + const seen = new Set(); + function add(value, label) { + const cleanValue = String(value || "").trim(); + if (!cleanValue || seen.has(cleanValue)) return; + seen.add(cleanValue); + merged.push({ value: cleanValue, label: String(label || cleanValue) }); + } + for (const option of Array.isArray(baseOptions) ? baseOptions : []) { + add(option.value, option.label); + } + const remoteOptions = payload && Array.isArray(payload.available_model_options) + ? payload.available_model_options + : []; + for (const option of remoteOptions) { + add(option.value, option.label); + } + for (const model of payload && Array.isArray(payload.available_models) ? payload.available_models : []) { + add(model, `Installed: ${model}`); + } + if (selectedValue) add(selectedValue, `Custom: ${selectedValue}`); + return merged; +} +``` + +Export it in the returned object. + +Update `syncInstalledTranslationModels` to call this helper and rebuild the select options while preserving `settings.translateModel`. + +- [ ] **Step 4: Run userscript tests** + +Run: + +```powershell +node --test tests/userscript-core.test.cjs +``` + +Expected: PASS. + +## Task 7: Documentation and Full Verification + +**Files:** +- Modify: `D:/local-tts-env/README.md` +- Modify: `D:/local-tts-env/说明.md` + +- [ ] **Step 1: Document the remote service** + +Add a short section explaining: + +```markdown +### Remote Ollama over LAN + +Right-click the Kokoro TTS tray icon and choose `Remote Service`. +Enter the server name, IP, SSH port, username, password, and remote Ollama host/port. +The app creates a local SSH tunnel and the browser model selector will show models as `Server Name / model`. + +The browser script never receives the SSH password. The password is stored in `tray_settings.json` for convenience. +``` + +- [ ] **Step 2: Run full Python and Node tests** + +Run: + +```powershell +python -m pytest -q +node --test tests/userscript-core.test.cjs +``` + +Expected: all tests PASS. + +- [ ] **Step 3: Manual tray smoke check** + +Run: + +```powershell +python -c "import tray_app; settings=tray_app.default_remote_ollama_settings(); print(settings['ollama_host'], settings['ollama_port'])" +``` + +Expected: + +```text +127.0.0.1 11434 +``` + +## Self-Review + +- Spec coverage: server routing, tray configuration, remote SSH tunnel, browser model selection, error privacy, and documentation are each covered by a task. +- Placeholder scan: no deferred implementation markers are used; each step has concrete commands and expected results. +- Type consistency: remote model references use `remote::` throughout; server source fields are consistently `id`, `name`, `base_url`, and `remote`; browser option fields match the API response shape. diff --git a/docs/superpowers/specs/2026-06-27-remote-ollama-service-design.md b/docs/superpowers/specs/2026-06-27-remote-ollama-service-design.md new file mode 100644 index 0000000..06878f1 --- /dev/null +++ b/docs/superpowers/specs/2026-06-27-remote-ollama-service-design.md @@ -0,0 +1,157 @@ +# Remote Ollama Service Design + +## Goal + +Add an optional remote Ollama source for translation-related models while keeping the existing local Ollama path working. The user configures the remote server from the Windows tray app, then the browser model selector shows both local models and remote server models. + +## User Flow + +1. The user right-clicks the Kokoro TTS tray icon and opens `Remote Service`. +2. A small settings window lets the user enter: + - server display name + - server IP or host + - SSH port, default `22` + - username + - password + - remote Ollama host and port, default `127.0.0.1:11434` +3. The user clicks connect. +4. The tray app opens an SSH tunnel from a local ephemeral port to the remote server's Ollama endpoint. +5. The local FastAPI service discovers models from both the local Ollama and configured remote sources. +6. The browser settings panel model selector shows grouped options for local models and the remote server name. +7. Translation, read preparation, formula verbalization, keep loaded, unload, and health checks use the selected source. + +## Architecture + +The browser userscript continues to talk only to the local Kokoro TTS API at `127.0.0.1:5000`. It never receives the remote SSH password. + +The tray app owns remote connection setup because it already manages background process state and local settings. It saves the remote connection profile in `tray_settings.json` and starts the local server with environment variables that point to the active remote source configuration. + +The FastAPI server adds source-aware Ollama routing. Existing model names such as `translategemma:4b` keep using local Ollama. Remote model choices are represented with an internal source prefix, for example: + +```text +remote::qwen3:14b +``` + +API responses include display metadata so the browser can render friendly labels such as `Lab Server / qwen3:14b`. + +## Components + +### Tray App + +Add a `Remote Service` menu item. It opens a basic Tkinter dialog for configuration and connection. The dialog validates required fields and tests the tunnel by calling `/api/tags` through the local forwarded port. + +The tray app should store remote settings under a new `remote_ollama` key in `tray_settings.json`: + +```json +{ + "enabled": true, + "name": "Lab Server", + "host": "192.168.1.10", + "ssh_port": 22, + "username": "user", + "password": "password", + "ollama_host": "127.0.0.1", + "ollama_port": 11434, + "local_port": 0 +} +``` + +`local_port: 0` means choose an available local port automatically. The tray app keeps the SSH tunnel process alive while the Kokoro app is running and restarts the local server after connection changes. + +### Server + +Add an Ollama source abstraction with: + +- source id +- display name +- base URL +- whether the source is local or remote + +Default source: + +```text +local -> http://127.0.0.1:11434 +``` + +Remote sources are loaded from environment JSON passed by the tray app, for example `KOKORO_OLLAMA_SOURCES`. + +The existing Ollama helper functions gain an optional source-aware model reference. They resolve `remote::` into the selected source and clean model name before calling `/api/generate`, `/api/tags`, or `/api/ps`. + +Pinned models are tracked by full model reference instead of plain model name, so a local `qwen3:14b` and remote `qwen3:14b` do not collide. + +### Browser Userscript + +The settings panel keeps the current local custom model behavior. It also uses the health response's available model metadata to append remote model options. + +The selected model value can be either: + +```text +qwen3:14b +remote:lab-server:qwen3:14b +``` + +The browser sends that value unchanged to existing endpoints. + +## API Changes + +`GET /translate/health?model=...` remains backward-compatible. + +The response adds optional fields: + +```json +{ + "source": "local", + "source_name": "Local Ollama", + "available_model_options": [ + { + "value": "translategemma:4b", + "label": "Local Ollama / translategemma:4b", + "source": "local", + "source_name": "Local Ollama", + "model": "translategemma:4b" + }, + { + "value": "remote:lab-server:qwen3:14b", + "label": "Lab Server / qwen3:14b", + "source": "lab-server", + "source_name": "Lab Server", + "model": "qwen3:14b" + } + ] +} +``` + +Existing `available_models` remains a list of local-compatible strings for older userscripts. + +## Error Handling + +If the remote SSH tunnel cannot connect, the tray dialog shows a concise error and leaves the previous working settings unchanged. + +If the local server starts without a working remote source, local Ollama still works. Health checks for remote selections return `ollama_reachable: false` and do not expose passwords or detailed connection strings. + +Translation endpoints continue returning generic 502 errors. Logs may include source names but not passwords. + +## Testing + +Add server tests for: + +- parsing local and remote model references +- listing model options across local and remote sources +- routing generate, tags, ps, keepalive, and unload requests to the selected source +- pinned model identity including source id +- backward compatibility for plain local model names + +Add tray tests for: + +- default remote settings shape +- saving remote settings without breaking existing voice, speed, and auto-start settings +- building a remote source environment payload without including unrelated fields + +Add userscript tests for: + +- appending remote model options from health metadata +- preserving selected remote model values when saving settings + +## Non-Goals + +This change does not install or configure Ollama on the remote server. It also does not add public HTTP authentication for Ollama. The intended remote path is SSH login to a LAN server and forwarding to that server's native Ollama API. diff --git a/requirements.txt b/requirements.txt index bec2050..890f799 100644 --- a/requirements.txt +++ b/requirements.txt @@ -13,3 +13,4 @@ imageio-ffmpeg==0.6.0 # Windows tray application pystray==0.19.5 Pillow==12.2.0 +paramiko>=3.5,<5 diff --git a/server.py b/server.py index e5f4a0f..1bf6793 100644 --- a/server.py +++ b/server.py @@ -19,6 +19,7 @@ import threading import time import warnings +from dataclasses import dataclass from pathlib import Path from typing import Optional from urllib import error as urllib_error @@ -98,6 +99,108 @@ ("sqrt", "sqrt"): "square_root", } + +@dataclass(frozen=True) +class OllamaSource: + id: str + name: str + base_url: str + remote: bool = False + + +@dataclass(frozen=True) +class OllamaModelRef: + value: str + model: str + source: OllamaSource + + +def _local_ollama_source() -> OllamaSource: + return OllamaSource("local", "Local Ollama", OLLAMA_BASE_URL, False) + + +def _load_ollama_sources_from_env(raw: Optional[str] = None) -> dict[str, OllamaSource]: + sources = {"local": _local_ollama_source()} + raw_value = os.environ.get("KOKORO_OLLAMA_SOURCES", "") if raw is None else raw + if not raw_value: + return sources + try: + items = json.loads(raw_value) + except json.JSONDecodeError: + return sources + if not isinstance(items, list): + return sources + for item in items: + if not isinstance(item, dict): + continue + source_id = str(item.get("id") or "").strip() + name = str(item.get("name") or source_id).strip() + base_url = str(item.get("base_url") or "").strip().rstrip("/") + if not source_id or source_id == "local" or not base_url: + continue + if any(ch.isspace() for ch in source_id): + continue + sources[source_id] = OllamaSource(source_id, name or source_id, base_url, True) + return sources + + +OLLAMA_SOURCES = _load_ollama_sources_from_env() + + +def _model_value_for_source(source: OllamaSource, model: str) -> str: + return model if source.id == "local" else f"remote:{source.id}:{model}" + + +def _clean_ollama_model_name(value: Optional[str]) -> str: + selected = (value or "").strip() + if selected.startswith("remote:"): + parts = selected.split(":", 2) + if len(parts) == 3: + return parts[2].strip() + return selected + + +def _resolve_ollama_model_ref(value: Optional[str]) -> OllamaModelRef: + selected = (value or OLLAMA_TRANSLATE_MODEL).strip() or OLLAMA_TRANSLATE_MODEL + if selected.startswith("remote:"): + parts = selected.split(":", 2) + if len(parts) != 3 or not parts[1] or not parts[2]: + raise RuntimeError("Invalid remote Ollama model reference") + source = OLLAMA_SOURCES.get(parts[1]) + if source is None: + raise RuntimeError("Remote Ollama source is not configured") + return OllamaModelRef( + _model_value_for_source(source, parts[2]), + parts[2], + source, + ) + source = OLLAMA_SOURCES["local"] + return OllamaModelRef(selected, selected, source) + + +def _collect_ollama_model_options() -> list[dict[str, str]]: + options: list[dict[str, str]] = [] + for source in OLLAMA_SOURCES.values(): + try: + if source.id == "local": + payload = _call_ollama_json("/api/tags") + else: + payload = _call_ollama_json("/api/tags", base_url=source.base_url) + except Exception: + continue + for model in _ollama_model_names(payload): + value = _model_value_for_source(source, model) + options.append( + { + "value": value, + "label": f"{source.name} / {model}", + "source": source.id, + "source_name": source.name, + "model": model, + } + ) + return options + if VOICE not in AVAILABLE_VOICES: raise ValueError(f"Unsupported KOKORO_VOICE: {VOICE}") if not math.isfinite(DEFAULT_SPEED) or not 0.5 <= DEFAULT_SPEED <= 2.0: @@ -531,6 +634,14 @@ class FormulaVerbalizeResponse(BaseModel): elapsed: float +class OllamaModelOption(BaseModel): + value: str + label: str + source: str + source_name: str + model: str + + class TranslateHealthResponse(BaseModel): status: str ollama_reachable: bool @@ -540,6 +651,9 @@ class TranslateHealthResponse(BaseModel): model_pinned: bool = False available_models: list[str] running_models: list[str] + source: str = "local" + source_name: str = "Local Ollama" + available_model_options: list[OllamaModelOption] = Field(default_factory=list) error: Optional[str] = None @@ -664,7 +778,7 @@ def _normalize_llm_source_text(text: str) -> str: def _model_size_billions(model: Optional[str]) -> Optional[float]: - value = (model or "").lower() + value = _clean_ollama_model_name(model).lower() matches = re.findall(r"(\d+(?:\.\d+)?)\s*b\b", value) if not matches: return None @@ -675,7 +789,7 @@ def _model_size_billions(model: Optional[str]) -> Optional[float]: def _ollama_should_disable_thinking(model: Optional[str]) -> bool: - value = re.sub(r"[^a-z0-9]+", "", (model or "").lower()) + value = re.sub(r"[^a-z0-9]+", "", _clean_ollama_model_name(model).lower()) if not value or "embedding" in value: return False return any(marker in value for marker in ("qwen3", "qwq", "deepseekr1")) @@ -687,7 +801,7 @@ def _apply_ollama_thinking_mode(payload: dict, model: Optional[str]) -> None: def _apply_ollama_keep_alive(payload: dict, model: Optional[str]) -> None: - selected_model = (model or "").strip() + selected_model = _resolve_ollama_model_ref(model).value if model else "" if selected_model and selected_model in PINNED_OLLAMA_MODELS: payload["keep_alive"] = OLLAMA_KEEP_ALIVE_PIN_VALUE @@ -799,9 +913,14 @@ def _normalize_translation_context( return limited or None -def _call_ollama_json(path: str, timeout: float = 5.0): +def _call_ollama_json( + path: str, + timeout: float = 5.0, + base_url: Optional[str] = None, +): + selected_base_url = (base_url or OLLAMA_BASE_URL).rstrip("/") req = urllib_request.Request( - f"{OLLAMA_BASE_URL}{path}", + f"{selected_base_url}{path}", headers={"Accept": "application/json"}, method="GET", ) @@ -821,16 +940,22 @@ def _call_ollama_json(path: str, timeout: float = 5.0): raise RuntimeError("Ollama returned invalid JSON") from error +def _ollama_generate_url(ref: OllamaModelRef) -> str: + return f"{ref.source.base_url}/api/generate" + + def _call_ollama_model_keep_alive(model: str, keep_alive: str | int | float): + ref = _resolve_ollama_model_ref(model) payload = { - "model": model, + "model": ref.model, "prompt": "", "stream": False, "keep_alive": keep_alive, } + _prepare_ollama_generate_payload(payload, ref.value) data = json.dumps(payload, ensure_ascii=False).encode("utf-8") req = urllib_request.Request( - f"{OLLAMA_BASE_URL}/api/generate", + _ollama_generate_url(ref), data=data, headers={"Content-Type": "application/json"}, method="POST", @@ -864,7 +989,12 @@ def _ollama_model_names(payload) -> list[str]: def _ollama_model_is_running(model: str) -> bool: - return model in _ollama_model_names(_call_ollama_json("/api/ps")) + ref = _resolve_ollama_model_ref(model) + if ref.source.id == "local": + payload = _call_ollama_json("/api/ps") + else: + payload = _call_ollama_json("/api/ps", base_url=ref.source.base_url) + return ref.model in _ollama_model_names(payload) def _wait_for_ollama_model_state( @@ -1387,6 +1517,7 @@ def _call_ollama_formula_verbalization_zh_single( model: str, context: Optional[str] = None, ) -> str: + ref = _resolve_ollama_model_ref(model) rule_desc = _rule_describe_formula_zh(formula) if rule_desc: return rule_desc @@ -1394,14 +1525,14 @@ def _call_ollama_formula_verbalization_zh_single( cf = formula.strip() cf = _unwrap_formula_for_translation(cf) - context_block = _limit_model_context(context, model, "formula") or "" + context_block = _limit_model_context(context, ref.model, "formula") or "" prompt = f"Formula: {cf}" if context_block: prompt = f"Context: {context_block}\nFormula: {cf}" glossary_prompt = _math_glossary_prompt("zh") payload = { - "model": model, + "model": ref.model, "stream": False, "system": ( "You convert a mathematical formula into a professional spoken Simplified Chinese description (口语化中文描述).\n" @@ -1443,10 +1574,10 @@ def _call_ollama_formula_verbalization_zh_single( "top_p": 0.9, }, } - _prepare_ollama_generate_payload(payload, model) + _prepare_ollama_generate_payload(payload, ref.value) data = json.dumps(payload, ensure_ascii=False).encode("utf-8") req = urllib_request.Request( - f"{OLLAMA_BASE_URL}/api/generate", + _ollama_generate_url(ref), data=data, headers={"Content-Type": "application/json"}, method="POST", @@ -1510,8 +1641,9 @@ def _call_ollama_translate_raw( target_language: str, context: Optional[str] = None, ) -> str: + ref = _resolve_ollama_model_ref(model) prompt = protected_text - context = _limit_model_context(context, model, "translation") + context = _limit_model_context(context, ref.model, "translation") if context: prompt = ( "\n" @@ -1522,7 +1654,7 @@ def _call_ollama_translate_raw( "" ) payload = { - "model": model, + "model": ref.model, "stream": False, "system": ( "You are a precise translation engine. The input contains text with placeholders " @@ -1544,10 +1676,10 @@ def _call_ollama_translate_raw( "top_p": 0.9, }, } - _prepare_ollama_generate_payload(payload, model) + _prepare_ollama_generate_payload(payload, ref.value) data = json.dumps(payload, ensure_ascii=False).encode("utf-8") req = urllib_request.Request( - f"{OLLAMA_BASE_URL}/api/generate", + _ollama_generate_url(ref), data=data, headers={"Content-Type": "application/json"}, method="POST", @@ -1685,8 +1817,9 @@ def _call_ollama_text_generation( timeout_detail: str, empty_detail: str, ) -> str: + ref = _resolve_ollama_model_ref(model) payload = { - "model": model, + "model": ref.model, "stream": False, "system": system, "prompt": prompt, @@ -1695,10 +1828,10 @@ def _call_ollama_text_generation( "top_p": 0.9, }, } - _prepare_ollama_generate_payload(payload, model) + _prepare_ollama_generate_payload(payload, ref.value) data = json.dumps(payload, ensure_ascii=False).encode("utf-8") req = urllib_request.Request( - f"{OLLAMA_BASE_URL}/api/generate", + _ollama_generate_url(ref), data=data, headers={"Content-Type": "application/json"}, method="POST", @@ -1949,7 +2082,8 @@ def _call_ollama_formula_verbalization( context: Optional[str] = None, ) -> list[str]: rule_values = [_rule_describe_formula_en(formula) for formula in formulas] - model_size = _model_size_billions(model) + ref = _resolve_ollama_model_ref(model) + model_size = _model_size_billions(ref.model) small_model = model_size is not None and model_size <= 4.5 if small_model and all(rule_values): return rule_values @@ -1963,14 +2097,14 @@ def _call_ollama_formula_verbalization( if not pending_formulas: return [value or "formula omitted" for value in rule_values] - context_block = _limit_model_context(context, model, "formula") or "" + context_block = _limit_model_context(context, ref.model, "formula") or "" glossary_prompt = _math_glossary_prompt("en") prompt = { "context": context_block, "formulas": pending_formulas, } payload = { - "model": model, + "model": ref.model, "stream": False, "system": ( "You convert math formulas into concise spoken English for text-to-speech. " @@ -1990,10 +2124,10 @@ def _call_ollama_formula_verbalization( "top_p": 0.9, }, } - _prepare_ollama_generate_payload(payload, model) + _prepare_ollama_generate_payload(payload, ref.value) data = json.dumps(payload, ensure_ascii=False).encode("utf-8") req = urllib_request.Request( - f"{OLLAMA_BASE_URL}/api/generate", + _ollama_generate_url(ref), data=data, headers={"Content-Type": "application/json"}, method="POST", @@ -2224,35 +2358,64 @@ async def translate_health(model: Optional[str] = None): """Report whether Ollama is reachable and the selected model is loaded.""" selected_model = (model or OLLAMA_TRANSLATE_MODEL).strip() or OLLAMA_TRANSLATE_MODEL try: - available_models = _ollama_model_names(_call_ollama_json("/api/tags")) + ref = _resolve_ollama_model_ref(selected_model) + base_url = None if ref.source.id == "local" else ref.source.base_url + if base_url: + available_models = _ollama_model_names( + _call_ollama_json("/api/tags", base_url=base_url) + ) + else: + available_models = _ollama_model_names(_call_ollama_json("/api/tags")) try: - running_models = _ollama_model_names(_call_ollama_json("/api/ps")) + if base_url: + running_models = _ollama_model_names( + _call_ollama_json("/api/ps", base_url=base_url) + ) + else: + running_models = _ollama_model_names(_call_ollama_json("/api/ps")) except RuntimeError: running_models = [] - model_available = selected_model in available_models - model_running = selected_model in running_models + model_available = ref.model in available_models + model_running = ref.model in running_models status = "running" if model_running else "available" if model_available else "missing" return TranslateHealthResponse( status=status, ollama_reachable=True, - model=selected_model, + model=ref.value, model_available=model_available, model_running=model_running, - model_pinned=selected_model in PINNED_OLLAMA_MODELS, + model_pinned=ref.value in PINNED_OLLAMA_MODELS, available_models=available_models, running_models=running_models, + source=ref.source.id, + source_name=ref.source.name, + available_model_options=_collect_ollama_model_options(), ) except Exception as error: print(f"[ERROR] Ollama health check failed: {error}") + try: + ref = _resolve_ollama_model_ref(selected_model) + source_id = ref.source.id + source_name = ref.source.name + pinned = ref.value in PINNED_OLLAMA_MODELS + response_model = ref.value + except Exception: + source_id = "local" + source_name = "Local Ollama" + pinned = selected_model in PINNED_OLLAMA_MODELS + response_model = selected_model return TranslateHealthResponse( status="error", ollama_reachable=False, - model=selected_model, + model=response_model, model_available=False, model_running=False, - model_pinned=selected_model in PINNED_OLLAMA_MODELS, + model_pinned=pinned, available_models=[], running_models=[], + source=source_id, + source_name=source_name, + available_model_options=_collect_ollama_model_options(), error="Cannot connect to Ollama", ) @@ -2261,7 +2424,7 @@ async def translate_health(model: Optional[str] = None): async def translate_model_keepalive(request: OllamaModelKeepAliveRequest): """Preload a local Ollama model and keep it resident until explicitly unloaded.""" t0 = time.perf_counter() - model = request.model + model = _resolve_ollama_model_ref(request.model).value try: result = await asyncio.to_thread( _call_ollama_model_keep_alive, @@ -2292,7 +2455,7 @@ async def translate_model_keepalive(request: OllamaModelKeepAliveRequest): async def translate_model_unload(request: OllamaModelUnloadRequest): """Unload a local Ollama model and remove its keep-alive pin.""" t0 = time.perf_counter() - model = request.model + model = _resolve_ollama_model_ref(request.model).value try: result = await asyncio.to_thread(_call_ollama_model_keep_alive, model, 0) PINNED_OLLAMA_MODELS.discard(model) diff --git a/setup.bat b/setup.bat index 01f003f..7727177 100644 --- a/setup.bat +++ b/setup.bat @@ -69,7 +69,7 @@ if errorlevel 1 ( goto :fail ) -call conda run -n "%ENV_NAME%" python -c "import fastapi, imageio_ffmpeg, kokoro, PIL, pystray, soundfile, torch; print('[OK] Imports passed. CUDA:', torch.cuda.is_available())" +call conda run -n "%ENV_NAME%" python -c "import fastapi, imageio_ffmpeg, kokoro, paramiko, PIL, pystray, soundfile, torch; print('[OK] Imports passed. CUDA:', torch.cuda.is_available())" if errorlevel 1 ( echo [ERROR] Import verification failed. goto :fail diff --git a/tests/test_server.py b/tests/test_server.py index f821037..3c4b307 100644 --- a/tests/test_server.py +++ b/tests/test_server.py @@ -917,6 +917,172 @@ def test_translate_request_validation(self): response = self.client.post("/translate", json=payload) self.assertEqual(response.status_code, 422) + def test_remote_model_reference_resolves_source_and_model(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource( + id="local", + name="Local Ollama", + base_url="http://127.0.0.1:11434", + remote=False, + ), + "lab-server": server.OllamaSource( + id="lab-server", + name="Lab Server", + base_url="http://127.0.0.1:49152", + remote=True, + ), + } + try: + resolved = server._resolve_ollama_model_ref( + "remote:lab-server:qwen3:14b" + ) + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual(resolved.value, "remote:lab-server:qwen3:14b") + self.assertEqual(resolved.model, "qwen3:14b") + self.assertEqual(resolved.source.id, "lab-server") + self.assertEqual(resolved.source.base_url, "http://127.0.0.1:49152") + + def test_plain_model_reference_uses_local_source(self): + resolved = server._resolve_ollama_model_ref("translategemma:4b") + + self.assertEqual(resolved.value, "translategemma:4b") + self.assertEqual(resolved.model, "translategemma:4b") + self.assertEqual(resolved.source.id, "local") + + def test_model_options_include_remote_source_labels(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource( + "local", + "Local Ollama", + "http://127.0.0.1:11434", + False, + ), + "lab-server": server.OllamaSource( + "lab-server", + "Lab Server", + "http://127.0.0.1:49152", + True, + ), + } + + def fake_ollama_json(path, timeout=5.0, base_url=None): + if path == "/api/tags" and base_url == "http://127.0.0.1:49152": + return {"models": [{"name": "qwen3:14b"}]} + if path == "/api/tags": + return {"models": [{"name": "translategemma:4b"}]} + raise AssertionError((path, base_url)) + + try: + with patch.object( + server, + "_call_ollama_json", + create=True, + side_effect=fake_ollama_json, + ): + options = server._collect_ollama_model_options() + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual( + options, + [ + { + "value": "translategemma:4b", + "label": "Local Ollama / translategemma:4b", + "source": "local", + "source_name": "Local Ollama", + "model": "translategemma:4b", + }, + { + "value": "remote:lab-server:qwen3:14b", + "label": "Lab Server / qwen3:14b", + "source": "lab-server", + "source_name": "Lab Server", + "model": "qwen3:14b", + }, + ], + ) + + def test_remote_translate_routes_generate_to_remote_base_url(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource( + "local", + "Local Ollama", + "http://127.0.0.1:11434", + False, + ), + "lab-server": server.OllamaSource( + "lab-server", + "Lab Server", + "http://127.0.0.1:49152", + True, + ), + } + captured = {} + + def fake_urlopen(request, timeout): + captured["url"] = request.full_url + captured["payload"] = json.loads(request.data.decode("utf-8")) + return FakeUrlopenResponse({"response": "remote result"}) + + try: + with patch.object(server.urllib_request, "urlopen", side_effect=fake_urlopen): + result = server._call_ollama_translate_raw( + "Hello", + "remote:lab-server:qwen3:14b", + "Simplified Chinese", + None, + ) + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual(result, "remote result") + self.assertEqual(captured["url"], "http://127.0.0.1:49152/api/generate") + self.assertEqual(captured["payload"]["model"], "qwen3:14b") + + def test_remote_pinned_model_generation_requests_keep_alive(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource( + "local", + "Local Ollama", + "http://127.0.0.1:11434", + False, + ), + "lab-server": server.OllamaSource( + "lab-server", + "Lab Server", + "http://127.0.0.1:49152", + True, + ), + } + server.PINNED_OLLAMA_MODELS.add("remote:lab-server:qwen3:14b") + captured = {} + + def fake_urlopen(request, timeout): + captured["payload"] = json.loads(request.data.decode("utf-8")) + return FakeUrlopenResponse({"response": "remote result"}) + + try: + with patch.object(server.urllib_request, "urlopen", side_effect=fake_urlopen): + server._call_ollama_translate_raw( + "Hello", + "remote:lab-server:qwen3:14b", + "Simplified Chinese", + ) + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual( + captured["payload"]["keep_alive"], + server.OLLAMA_KEEP_ALIVE_PIN_VALUE, + ) + def test_translate_health_reports_model_state(self): def fake_ollama_json(path): if path == "/api/tags": @@ -942,6 +1108,64 @@ def fake_ollama_json(path): self.assertTrue(payload["model_running"]) self.assertFalse(payload["model_pinned"]) + def test_translate_health_reports_remote_source_metadata(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource( + "local", + "Local Ollama", + "http://127.0.0.1:11434", + False, + ), + "lab-server": server.OllamaSource( + "lab-server", + "Lab Server", + "http://127.0.0.1:49152", + True, + ), + } + + def fake_ollama_json(path, timeout=5.0, base_url=None): + if base_url == "http://127.0.0.1:49152" and path == "/api/tags": + return {"models": [{"name": "qwen3:14b"}]} + if base_url == "http://127.0.0.1:49152" and path == "/api/ps": + return {"models": [{"name": "qwen3:14b"}]} + if path == "/api/tags": + return {"models": [{"name": "translategemma:4b"}]} + if path == "/api/ps": + return {"models": []} + raise AssertionError((path, base_url)) + + try: + with patch.object( + server, + "_call_ollama_json", + create=True, + side_effect=fake_ollama_json, + ): + response = self.client.get( + "/translate/health?model=remote:lab-server:qwen3:14b" + ) + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual(response.status_code, 200) + payload = response.json() + self.assertEqual(payload["source"], "lab-server") + self.assertEqual(payload["source_name"], "Lab Server") + self.assertEqual(payload["model"], "remote:lab-server:qwen3:14b") + self.assertTrue(payload["model_running"]) + self.assertIn( + { + "value": "remote:lab-server:qwen3:14b", + "label": "Lab Server / qwen3:14b", + "source": "lab-server", + "source_name": "Lab Server", + "model": "qwen3:14b", + }, + payload["available_model_options"], + ) + def test_translate_health_reports_pinned_model(self): server.PINNED_OLLAMA_MODELS.add("translategemma:4b") @@ -995,6 +1219,60 @@ def fake_ollama_json(path): self.assertIn("qwen3:14b", server.PINNED_OLLAMA_MODELS) keepalive.assert_called_once_with("qwen3:14b", server.OLLAMA_KEEP_ALIVE_PIN_VALUE) + def test_translate_model_keepalive_checks_remote_running_state(self): + original_sources = server.OLLAMA_SOURCES + server.OLLAMA_SOURCES = { + "local": server.OllamaSource( + "local", + "Local Ollama", + "http://127.0.0.1:11434", + False, + ), + "lab-server": server.OllamaSource( + "lab-server", + "Lab Server", + "http://127.0.0.1:49152", + True, + ), + } + + def fake_ollama_json(path, timeout=5.0, base_url=None): + if path == "/api/ps" and base_url == "http://127.0.0.1:49152": + return {"models": [{"name": "qwen3:14b"}]} + if path == "/api/ps": + return {"models": []} + raise AssertionError((path, base_url)) + + try: + with patch.object( + server, + "_call_ollama_model_keep_alive", + create=True, + return_value={"done_reason": "load"}, + ) as keepalive, patch.object( + server, + "_call_ollama_json", + create=True, + side_effect=fake_ollama_json, + ): + response = self.client.post( + "/translate/model/keepalive", + json={"model": "remote:lab-server:qwen3:14b"}, + ) + finally: + server.OLLAMA_SOURCES = original_sources + + self.assertEqual(response.status_code, 200) + payload = response.json() + self.assertEqual(payload["model"], "remote:lab-server:qwen3:14b") + self.assertTrue(payload["model_running"]) + self.assertTrue(payload["model_pinned"]) + self.assertIn("remote:lab-server:qwen3:14b", server.PINNED_OLLAMA_MODELS) + keepalive.assert_called_once_with( + "remote:lab-server:qwen3:14b", + server.OLLAMA_KEEP_ALIVE_PIN_VALUE, + ) + def test_translate_model_unload_unpins_model(self): server.PINNED_OLLAMA_MODELS.add("qwen3:14b") diff --git a/tests/test_tray_app.py b/tests/test_tray_app.py index f3ebb90..f056ac4 100644 --- a/tests/test_tray_app.py +++ b/tests/test_tray_app.py @@ -1,3 +1,4 @@ +import json import sys import unittest from types import SimpleNamespace @@ -31,6 +32,34 @@ def test_owned_server_can_be_stopped_by_tray(self): self.assertTrue(app.can_stop_server()) + def test_quit_app_forces_process_exit_after_cleanup(self): + app = self.make_app() + app._stop_remote_ollama_tunnel = Mock() + app.stop_server = Mock() + app.tray_icon = Mock() + + with patch.object(tray_app.os, "_exit") as exit_process: + app.quit_app() + + app._stop_remote_ollama_tunnel.assert_called_once() + app.stop_server.assert_called_once() + app.tray_icon.stop.assert_called_once() + exit_process.assert_called_once_with(0) + + def test_quit_app_still_exits_when_cleanup_fails(self): + app = self.make_app() + app._stop_remote_ollama_tunnel = Mock(side_effect=RuntimeError("stuck")) + app.stop_server = Mock(side_effect=RuntimeError("also stuck")) + app.tray_icon = Mock() + + with patch.object(tray_app.os, "_exit") as exit_process: + app.quit_app() + + app._stop_remote_ollama_tunnel.assert_called_once() + app.stop_server.assert_called_once() + app.tray_icon.stop.assert_called_once() + exit_process.assert_called_once_with(0) + class TrayAutoStartTests(unittest.TestCase): def make_app(self): @@ -121,5 +150,102 @@ def __init__(self, *items): self.assertTrue(item.kwargs["checked"](item)) +class TrayRemoteOllamaTests(unittest.TestCase): + def make_app(self): + with patch.object( + tray_app, + "find_conda_python", + return_value=Path(sys.executable), + ), patch.object(tray_app, "reconcile_startup_shortcut"): + return tray_app.TrayApp() + + def test_default_settings_include_remote_ollama(self): + with patch.object( + tray_app, + "SETTINGS_FILE", + Path("__missing_tray_settings_for_test__.json"), + ): + settings = tray_app.load_settings() + + self.assertEqual( + settings["remote_ollama"], + { + "enabled": False, + "name": "", + "host": "", + "ssh_port": 22, + "username": "", + "password": "", + "ollama_host": "127.0.0.1", + "ollama_port": 11434, + "local_port": 0, + }, + ) + + def test_remote_source_env_omits_password(self): + app = self.make_app() + app.settings["remote_ollama"] = { + "enabled": True, + "name": "Lab Server", + "host": "192.168.1.10", + "ssh_port": 22, + "username": "alice", + "password": "secret", + "ollama_host": "127.0.0.1", + "ollama_port": 11434, + "local_port": 49152, + } + app.remote_tunnel_local_port = 49152 + + payload = app.build_remote_ollama_sources_env() + + self.assertEqual( + json.loads(payload), + [ + { + "id": "lab-server", + "name": "Lab Server", + "base_url": "http://127.0.0.1:49152", + } + ], + ) + self.assertNotIn("secret", payload) + + def test_remote_service_dialog_opens_on_background_thread(self): + app = self.make_app() + started = [] + + class FakeThread: + def __init__(self, target=None, daemon=None): + self.target = target + self.daemon = daemon + + def start(self): + started.append(self) + + def raise_if_tk_opens_synchronously(): + raise AssertionError("dialog should not open synchronously") + + fake_tkinter = SimpleNamespace( + Tk=raise_if_tk_opens_synchronously, + messagebox=SimpleNamespace(), + ) + + with patch.dict(sys.modules, {"tkinter": fake_tkinter}), patch.object( + tray_app.threading, + "Thread", + side_effect=lambda target=None, daemon=None: FakeThread(target, daemon), + ): + app.open_remote_service_settings() + + self.assertEqual(len(started), 1) + self.assertTrue(started[0].daemon) + self.assertIs(started[0].target.__self__, app) + self.assertIs( + started[0].target.__func__, + app._run_remote_service_settings_dialog.__func__, + ) + + if __name__ == "__main__": unittest.main() diff --git a/tests/userscript-core.test.cjs b/tests/userscript-core.test.cjs index 677283a..4b97236 100644 --- a/tests/userscript-core.test.cjs +++ b/tests/userscript-core.test.cjs @@ -426,3 +426,28 @@ test("bare LaTeX formulas use rules or LLM fallback", () => { assert.equal(complex.formulas.length, 1); assert.match(complex.text, /__LOCAL_READ_FORMULA_0__/); }); + +test("model option merge includes remote health metadata", () => { + const { mergeTranslationModelOptions } = require("../tts-userscript.js"); + const merged = mergeTranslationModelOptions( + [{ value: "translategemma:4b", label: "translategemma:4b - default" }], + { + available_models: ["translategemma:4b"], + available_model_options: [ + { + value: "remote:lab-server:qwen3:14b", + label: "Lab Server / qwen3:14b", + source: "lab-server", + source_name: "Lab Server", + model: "qwen3:14b", + }, + ], + }, + "remote:lab-server:qwen3:14b" + ); + + assert.deepEqual(merged, [ + { value: "translategemma:4b", label: "translategemma:4b - default" }, + { value: "remote:lab-server:qwen3:14b", label: "Lab Server / qwen3:14b" }, + ]); +}); diff --git a/tray_app.py b/tray_app.py index bba344f..b928d02 100644 --- a/tray_app.py +++ b/tray_app.py @@ -14,7 +14,9 @@ import json import os import signal +import select import socket +import socketserver import subprocess import sys import threading @@ -117,8 +119,36 @@ def find_conda_pythonw(env_name: str) -> Path: # Settings persistence # --------------------------------------------------------------------------- +def default_remote_ollama_settings(): + return { + "enabled": False, + "name": "", + "host": "", + "ssh_port": 22, + "username": "", + "password": "", + "ollama_host": "127.0.0.1", + "ollama_port": 11434, + "local_port": 0, + } + + +def slugify_source_id(value): + cleaned = "".join( + ch.lower() if ch.isalnum() else "-" + for ch in str(value or "").strip() + ) + cleaned = "-".join(part for part in cleaned.split("-") if part) + return cleaned or "remote-server" + + def load_settings(): - defaults = {"voice": DEFAULT_VOICE, "speed": DEFAULT_SPEED, "auto_start": False} + defaults = { + "voice": DEFAULT_VOICE, + "speed": DEFAULT_SPEED, + "auto_start": False, + "remote_ollama": default_remote_ollama_settings(), + } try: if SETTINGS_FILE.exists(): with open(SETTINGS_FILE, "r", encoding="utf-8") as f: @@ -131,6 +161,16 @@ def load_settings(): if defaults["speed"] not in SPEEDS: defaults["speed"] = DEFAULT_SPEED defaults["auto_start"] = bool(defaults.get("auto_start", False)) + remote = default_remote_ollama_settings() + if isinstance(defaults.get("remote_ollama"), dict): + remote.update(defaults["remote_ollama"]) + remote["enabled"] = bool(remote.get("enabled", False)) + for key in ("ssh_port", "ollama_port", "local_port"): + try: + remote[key] = int(remote.get(key) or default_remote_ollama_settings()[key]) + except (TypeError, ValueError): + remote[key] = default_remote_ollama_settings()[key] + defaults["remote_ollama"] = remote return defaults @@ -189,6 +229,113 @@ def create_icon_image(color="green"): # Server process management # --------------------------------------------------------------------------- +class RemoteOllamaTunnel: + def __init__(self, settings): + self.settings = dict(settings or {}) + self.client = None + self.server = None + self.thread = None + self.local_port = 0 + + def start(self): + import paramiko + + host = str(self.settings.get("host") or "").strip() + username = str(self.settings.get("username") or "").strip() + password = str(self.settings.get("password") or "") + if not host: + raise RuntimeError("Remote server IP is required") + if not username: + raise RuntimeError("Remote username is required") + if not password: + raise RuntimeError("Remote password is required") + + ssh_port = int(self.settings.get("ssh_port") or 22) + ollama_host = str(self.settings.get("ollama_host") or "127.0.0.1").strip() + ollama_port = int(self.settings.get("ollama_port") or 11434) + requested_local_port = int(self.settings.get("local_port") or 0) + + client = paramiko.SSHClient() + client.set_missing_host_key_policy(paramiko.AutoAddPolicy()) + client.connect( + hostname=host, + port=ssh_port, + username=username, + password=password, + look_for_keys=False, + allow_agent=False, + timeout=10, + banner_timeout=10, + auth_timeout=10, + ) + transport = client.get_transport() + if transport is None or not transport.is_active(): + client.close() + raise RuntimeError("SSH connection is not active") + + class ForwardHandler(socketserver.BaseRequestHandler): + def handle(handler_self): + channel = None + try: + channel = transport.open_channel( + "direct-tcpip", + (ollama_host, ollama_port), + handler_self.client_address, + ) + while True: + readable, _, _ = select.select( + [handler_self.request, channel], + [], + [], + 10, + ) + if handler_self.request in readable: + data = handler_self.request.recv(32768) + if not data: + break + channel.sendall(data) + if channel in readable: + data = channel.recv(32768) + if not data: + break + handler_self.request.sendall(data) + finally: + if channel is not None: + channel.close() + handler_self.request.close() + + class ThreadingForwardServer(socketserver.ThreadingTCPServer): + allow_reuse_address = True + daemon_threads = True + + server = ThreadingForwardServer((DEFAULT_HOST, requested_local_port), ForwardHandler) + thread = threading.Thread(target=server.serve_forever, daemon=True) + thread.start() + + self.client = client + self.server = server + self.thread = thread + self.local_port = int(server.server_address[1]) + return self.local_port + + def stop(self): + if self.server is not None: + try: + self.server.shutdown() + self.server.server_close() + except Exception: + pass + self.server = None + if self.client is not None: + try: + self.client.close() + except Exception: + pass + self.client = None + self.thread = None + self.local_port = 0 + + class TrayApp: def __init__(self): self.server_process = None @@ -199,6 +346,8 @@ def __init__(self): self.is_running = False self._lock = threading.Lock() self.python_exe = find_conda_python(CONDA_ENV_NAME) + self.remote_tunnel = None + self.remote_tunnel_local_port = None # 缓存开机自启状态,避免右键托盘菜单渲染时同步拉起 PowerShell 子进程导致系统假死 self.auto_start_cached = bool(self.settings.get("auto_start", False)) threading.Thread(target=self._init_and_reconcile_auto_start, daemon=True).start() @@ -244,12 +393,16 @@ def start_server(self, _=None): self._update_icon("yellow", "Kokoro TTS - Starting...") env = os.environ.copy() + self.ensure_remote_ollama_tunnel() env["PYTHONIOENCODING"] = "utf-8" env["KOKORO_HOST"] = DEFAULT_HOST env["KOKORO_PORT"] = str(DEFAULT_PORT) env["KOKORO_VOICE"] = self.settings["voice"] env["KOKORO_SPEED"] = str(self.settings["speed"]) env["KOKORO_TRAY_PID"] = str(os.getpid()) + sources_env = self.build_remote_ollama_sources_env() + if sources_env: + env["KOKORO_OLLAMA_SOURCES"] = sources_env # Start server process (hidden, no window) startupinfo = subprocess.STARTUPINFO() @@ -314,6 +467,104 @@ def restart_server(self, _=None): def can_stop_server(self): return self.is_running and self.owns_server + def build_remote_ollama_sources_env(self): + remote = self.settings.get("remote_ollama") or {} + if not remote.get("enabled"): + return "" + try: + local_port = int( + self.remote_tunnel_local_port or remote.get("local_port") or 0 + ) + except (TypeError, ValueError): + local_port = 0 + if local_port <= 0: + return "" + name = (remote.get("name") or remote.get("host") or "Remote Ollama").strip() + source_id = slugify_source_id(name) + return json.dumps( + [ + { + "id": source_id, + "name": name, + "base_url": f"http://127.0.0.1:{local_port}", + } + ], + ensure_ascii=False, + ) + + def _test_remote_ollama_tunnel(self, local_port): + with urllib.request.urlopen( + f"http://{DEFAULT_HOST}:{local_port}/api/tags", + timeout=5, + ) as response: + if response.status != 200: + raise RuntimeError("Remote Ollama did not return model tags") + json.load(response) + + def _stop_remote_ollama_tunnel(self): + if self.remote_tunnel: + self.remote_tunnel.stop() + self.remote_tunnel = None + self.remote_tunnel_local_port = None + + def ensure_remote_ollama_tunnel(self): + remote = self.settings.get("remote_ollama") or {} + if not remote.get("enabled"): + return False + if self.remote_tunnel and self.remote_tunnel_local_port: + return True + try: + tunnel = RemoteOllamaTunnel(remote) + local_port = tunnel.start() + self._test_remote_ollama_tunnel(local_port) + self.remote_tunnel = tunnel + self.remote_tunnel_local_port = local_port + remote["local_port"] = local_port + self.settings["remote_ollama"] = remote + save_settings(self.settings) + return True + except Exception as error: + self._stop_remote_ollama_tunnel() + remote = dict(remote) + remote["enabled"] = False + remote["local_port"] = 0 + self.settings["remote_ollama"] = remote + self.show_error("Remote Service", str(error)) + return False + + def connect_remote_ollama(self): + self._stop_remote_ollama_tunnel() + remote = self.settings.get("remote_ollama") or default_remote_ollama_settings() + tunnel = RemoteOllamaTunnel(remote) + try: + local_port = tunnel.start() + self._test_remote_ollama_tunnel(local_port) + except Exception: + tunnel.stop() + raise + + self.remote_tunnel = tunnel + self.remote_tunnel_local_port = local_port + remote["enabled"] = True + remote["local_port"] = local_port + self.settings["remote_ollama"] = remote + save_settings(self.settings) + self.restart_server() + if self.tray_icon: + self.tray_icon.menu = self._build_menu() + + def disconnect_remote_ollama(self, _=None, restart=True): + self._stop_remote_ollama_tunnel() + remote = dict(self.settings.get("remote_ollama") or default_remote_ollama_settings()) + remote["enabled"] = False + remote["local_port"] = 0 + self.settings["remote_ollama"] = remote + save_settings(self.settings) + if restart: + self.restart_server() + if self.tray_icon: + self.tray_icon.menu = self._build_menu() + def open_test_page(self, _=None): webbrowser.open(f"http://{DEFAULT_HOST}:{DEFAULT_PORT}/") @@ -329,6 +580,124 @@ def open_log(self, _=None): LOG_FILE.touch() os.startfile(str(LOG_FILE)) + def open_remote_service_settings(self, _=None): + threading.Thread( + target=self._run_remote_service_settings_dialog, + daemon=True, + ).start() + + def _run_remote_service_settings_dialog(self): + import tkinter as tk + from tkinter import messagebox + + remote = default_remote_ollama_settings() + if isinstance(self.settings.get("remote_ollama"), dict): + remote.update(self.settings["remote_ollama"]) + + window = tk.Tk() + window.title("Kokoro TTS - Remote Service") + window.resizable(False, False) + + fields = [ + ("Server name", "name", remote.get("name") or ""), + ("Server IP", "host", remote.get("host") or ""), + ("SSH port", "ssh_port", str(remote.get("ssh_port") or 22)), + ("Username", "username", remote.get("username") or ""), + ("Password", "password", remote.get("password") or ""), + ("Ollama host", "ollama_host", remote.get("ollama_host") or "127.0.0.1"), + ("Ollama port", "ollama_port", str(remote.get("ollama_port") or 11434)), + ] + entries = {} + for row, (label, key, value) in enumerate(fields): + tk.Label(window, text=label).grid( + row=row, + column=0, + padx=10, + pady=5, + sticky="e", + ) + entry = tk.Entry(window, width=32, show="*" if key == "password" else "") + entry.insert(0, str(value)) + entry.grid(row=row, column=1, padx=10, pady=5, sticky="we") + entries[key] = entry + entries["host"].focus_set() + window.after(100, window.focus_force) + + status_text = "Connected" if remote.get("enabled") else "Not connected" + status = tk.Label(window, text=status_text) + status.grid(row=len(fields), column=0, columnspan=2, padx=10, pady=5) + + def read_remote_settings(): + values = default_remote_ollama_settings() + values.update( + { + "name": entries["name"].get().strip(), + "host": entries["host"].get().strip(), + "username": entries["username"].get().strip(), + "password": entries["password"].get(), + "ollama_host": entries["ollama_host"].get().strip() or "127.0.0.1", + } + ) + for key, fallback in (("ssh_port", 22), ("ollama_port", 11434)): + try: + values[key] = int(entries[key].get().strip() or fallback) + except ValueError: + raise RuntimeError(f"{key.replace('_', ' ')} must be a number") + values["name"] = values["name"] or values["host"] or "Remote Ollama" + values["local_port"] = int(remote.get("local_port") or 0) + return values + + def on_save(): + try: + values = read_remote_settings() + except Exception as error: + messagebox.showerror("Remote Service", str(error), parent=window) + return + values["enabled"] = bool(remote.get("enabled")) + self.settings["remote_ollama"] = values + save_settings(self.settings) + window.destroy() + + def on_connect(): + try: + values = read_remote_settings() + values["enabled"] = True + self.settings["remote_ollama"] = values + status.config(text="Connecting...") + window.update_idletasks() + self.connect_remote_ollama() + messagebox.showinfo("Remote Service", "Connected.", parent=window) + window.destroy() + except Exception as error: + messagebox.showerror("Remote Service", str(error), parent=window) + status.config(text="Connection failed") + + def on_disconnect(): + self.disconnect_remote_ollama(restart=True) + messagebox.showinfo("Remote Service", "Disconnected.", parent=window) + window.destroy() + + buttons = tk.Frame(window) + buttons.grid(row=len(fields) + 1, column=0, columnspan=2, padx=10, pady=10) + tk.Button(buttons, text="Connect", command=on_connect, width=10).pack( + side="left", + padx=4, + ) + tk.Button(buttons, text="Disconnect", command=on_disconnect, width=10).pack( + side="left", + padx=4, + ) + tk.Button(buttons, text="Save", command=on_save, width=10).pack( + side="left", + padx=4, + ) + tk.Button(buttons, text="Cancel", command=window.destroy, width=10).pack( + side="left", + padx=4, + ) + + window.mainloop() + def show_error(self, title, message): if os.name == "nt": try: @@ -399,10 +768,31 @@ def _set(_=None): self.restart_server() return _set + def _schedule_force_exit(self, delay=6.0): + timer = threading.Timer(delay, lambda: os._exit(0)) + timer.daemon = True + timer.start() + return timer + def quit_app(self, _=None): - self.stop_server() - if self.tray_icon: - self.tray_icon.stop() + watchdog = self._schedule_force_exit() + try: + try: + self._stop_remote_ollama_tunnel() + except Exception: + pass + try: + self.stop_server() + except Exception: + pass + if self.tray_icon: + try: + self.tray_icon.stop() + except Exception: + pass + finally: + watchdog.cancel() + os._exit(0) def _update_icon(self, color, title): if self.tray_icon: @@ -455,6 +845,7 @@ def _build_menu(self): enabled=lambda _: self.is_running), Item("Open Server Log", self.open_log), Item("Open Project Folder", self.open_project_dir), + Item("Remote Service", self.open_remote_service_settings), Item( "Auto-start on login", self.toggle_auto_start, diff --git a/tts-userscript.js b/tts-userscript.js index 4cf9ba9..c06e878 100644 --- a/tts-userscript.js +++ b/tts-userscript.js @@ -110,6 +110,44 @@ const KokoroTTSCore = (() => { return supportsWebMOpus(mediaSourceApi) ? "stream" : "ogg"; } + function mergeTranslationModelOptions(baseOptions, payload, selectedValue) { + const merged = []; + const seen = new Set(); + + function add(value, label) { + const cleanValue = String(value || "").trim(); + if (!cleanValue || seen.has(cleanValue)) return; + seen.add(cleanValue); + merged.push({ value: cleanValue, label: String(label || cleanValue) }); + } + + for (const option of Array.isArray(baseOptions) ? baseOptions : []) { + add(option.value, option.label); + } + + const remoteOptions = + payload && Array.isArray(payload.available_model_options) + ? payload.available_model_options + : []; + for (const option of remoteOptions) { + add(option.value, option.label); + } + + const installedModels = + payload && Array.isArray(payload.available_models) + ? payload.available_models + : []; + for (const model of installedModels) { + add(model, `Installed: ${model}`); + } + + if (selectedValue) { + add(selectedValue, `Custom: ${selectedValue}`); + } + + return merged; + } + function formatPlaybackProgress({ currentTime = 0, duration = 0, streamEnded = false } = {}) { const seconds = Math.max(0, Math.floor(Number(currentTime) || 0)); if (!streamEnded || !Number.isFinite(duration) || duration <= 0) { @@ -804,6 +842,7 @@ const KokoroTTSCore = (() => { formulaToReadableHtml, isUnsupportedMediaError, latexToReadableFormula, + mergeTranslationModelOptions, normalizeAudioBuffer, normalizeAudioBlob, normalizeCopyTextWithLatex, @@ -1742,15 +1781,22 @@ if (typeof window !== "undefined" && typeof document !== "undefined") { }); } - function syncInstalledTranslationModels(models) { + function syncInstalledTranslationModels(payloadOrModels) { const select = document.getElementById("tts-translate-model-select"); - if (!select || !Array.isArray(models)) return; - for (const model of models) { - if (typeof model !== "string" || !model.trim()) continue; - if (Array.from(select.options).some((option) => option.value === model)) { - continue; - } - select.appendChild(new Option(`Installed: ${model}`, model)); + if (!select) return; + const payload = Array.isArray(payloadOrModels) + ? { available_models: payloadOrModels } + : payloadOrModels || {}; + const options = KokoroTTSCore.mergeTranslationModelOptions( + TRANSLATION_MODELS, + payload, + settings.translateModel + ); + select.textContent = ""; + for (const option of options) { + const item = new Option(option.label, option.value); + item.selected = option.value === settings.translateModel; + select.appendChild(item); } select.value = settings.translateModel; } @@ -1853,7 +1899,7 @@ if (typeof window !== "undefined" && typeof document !== "undefined") { text.textContent = "Ollama offline"; text.style.color = "#e57373"; } else if (payload.model_running) { - syncInstalledTranslationModels(payload.available_models); + syncInstalledTranslationModels(payload); updateTranslationModelControls(payload); dot.className = "tts-status-dot online"; text.textContent = payload.model_pinned @@ -1861,13 +1907,13 @@ if (typeof window !== "undefined" && typeof document !== "undefined") { : `${payload.model} running`; text.style.color = "#81c784"; } else if (payload.model_available) { - syncInstalledTranslationModels(payload.available_models); + syncInstalledTranslationModels(payload); updateTranslationModelControls(payload); dot.className = "tts-status-dot warning"; text.textContent = `${payload.model} installed, not loaded`; text.style.color = "#f0c040"; } else { - syncInstalledTranslationModels(payload.available_models); + syncInstalledTranslationModels(payload); updateTranslationModelControls(payload); dot.className = "tts-status-dot offline"; text.textContent = `${payload.model} not installed`; diff --git "a/\350\257\264\346\230\216.md" "b/\350\257\264\346\230\216.md" index b6ba5a6..1bfe496 100644 --- "a/\350\257\264\346\230\216.md" +++ "b/\350\257\264\346\230\216.md" @@ -248,3 +248,8 @@ DEVICE = "auto" # auto / cuda / cpu ```javascript const API_URL = 'http://127.0.0.1:5000/tts'; // API 地址 ``` +## 远程 Ollama(局域网服务器) + +右键点击 Kokoro TTS 托盘图标,选择 `Remote Service`。填写服务器名称、IP、SSH 端口、用户名、密码,以及远程 Ollama 的主机和端口(默认 `127.0.0.1:11434`)。连接成功后,网页端 Translation 模型选择里会出现 `服务器名称 / 模型名`。 + +浏览器脚本只连接本机 `127.0.0.1:5000`,不会拿到 SSH 密码。为了使用方便,托盘程序会把远程配置保存在 `tray_settings.json`。