diff --git a/CHANGELOG.md b/CHANGELOG.md index 13c48a0a4..ce61df2bb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -23,7 +23,7 @@ Surface summary of the most recent release dates. See the release ledger below f - Refactoring: share lenient JSON parsing; centralize MCP command resolution. ### 2026-07-23 -- Features: surface and repair dead summary-job backlog (#901); recover missed native transcripts; inference settings density + hierarchy refactor; select connected providers as backends + compact rows. +- Features: surface and repair dead summary-job backlog (#901); recover missed native transcripts; inference settings density + hierarchy refactor; select connected providers as backends + compact rows; add the constrained ARM edge runtime profile and MiniLM native embeddings. - Bug fixes: expose macOS physical memory; harden native WASM inference; quiesce background inference on pause; harden sterile OpenCode inference; provider wall completeness, optional keys, OAuth window; support agent-managed models; parse JSON after model preambles; bound llama.cpp inputs; remove invalid hook ownership markers; stop native embedding smoke child within bun's afterEach budget; integrate pi-ai OAuth providers; make maintenance supersession sweep non-fatal without a provider; enable Astro ClientRouter for client-side navigation. - Docs: update roadmap to reflect current priorities (dashboard, desktop, config UX). diff --git a/README.md b/README.md index c8330abe0..50d1c4432 100644 --- a/README.md +++ b/README.md @@ -393,10 +393,13 @@ Requirements: Embeddings (choose one): -- **Built-in** (recommended) — no extra setup, runs locally via ONNX (`nomic-embed-text-v1.5`) +- **Built-in** (recommended) — no extra setup, runs locally via ONNX (`Xenova/all-MiniLM-L6-v2`, 384d) - **Ollama** — alternative local option, requires `nomic-embed-text` model - **OpenAI** — cloud option, requires `OPENAI_API_KEY` +For constrained 64-bit ARM devices, including the Raspberry Pi 3B+, use the +documented [edge installation](./docs/EDGE.md). + ## Contributing New to open source? Start with [Your First PR](./docs/FIRST-PR.md). diff --git a/docs/CLI.md b/docs/CLI.md index 8a0157048..02114de02 100644 --- a/docs/CLI.md +++ b/docs/CLI.md @@ -250,7 +250,7 @@ Wizard steps: - OpenAI API - Skip embeddings 9. **Embedding Model** - Based on provider: - - Built-in: `nomic-embed-text-v1.5` + - Built-in: `Xenova/all-MiniLM-L6-v2` (384d) - Ollama: `nomic-embed-text`, `all-minilm`, `mxbai-embed-large` - OpenAI: text-embedding-3-small, text-embedding-3-large - Ollama selections run preflight checks for binary availability, diff --git a/docs/CONFIGURATION.md b/docs/CONFIGURATION.md index d2c7cbcc2..eec57936c 100644 --- a/docs/CONFIGURATION.md +++ b/docs/CONFIGURATION.md @@ -87,11 +87,14 @@ harnesses: - openclaw - opencode +runtime: + # "standard" or "edge" (fixed-file polling + lazy native embeddings) + profile: standard + embedding: - provider: ollama - model: nomic-embed-text - dimensions: 768 - base_url: http://localhost:11434 + provider: native + model: Xenova/all-MiniLM-L6-v2 + dimensions: 384 promptSubmitTimeoutMs: 1000 # llama.cpp only: truncate inputs to stay below its physical batch limit llamaCppMaxInputTokens: 1400 @@ -210,10 +213,10 @@ Vector embedding configuration for semantic memory search. | Field | Type | Default | Description | |-------|------|---------|-------------| -| `provider` | string | `"ollama"` | `"ollama"` or `"openai"` | -| `model` | string | `"nomic-embed-text"` | Embedding model name | -| `dimensions` | number | `768` | Output vector dimensions | -| `base_url` | string | `"http://localhost:11434"` | Ollama API base URL | +| `provider` | string | `"native"` | `"native"`, `"llama-cpp"`, `"ollama"`, `"openai"`, or `"none"` | +| `model` | string | `"Xenova/all-MiniLM-L6-v2"` | Embedding model name; honored by the native worker | +| `dimensions` | number | `384` | Output vector dimensions (1-65536) | +| `base_url` | string | provider-specific | External provider API base URL | | `api_key` | string | — | API key or `$secret:NAME` reference | | `promptSubmitTimeoutMs` | number | `1000` | Explicit recall embedding timeout retained for compatibility, range 1000-300000 ms | @@ -229,6 +232,15 @@ Recommended Ollama models: | `all-minilm` | 384 | Faster, smaller vectors | | `mxbai-embed-large` | 1024 | Better quality, more resource usage | +### runtime + +`runtime.profile` accepts `standard` (default) or `edge`. The edge profile is +for the documented source-runtime install on constrained 64-bit ARM devices: +it uses fixed-file polling, skips the startup native-embedding probe, unloads +the isolated embedding child process after 30 seconds idle, and leaves +visualization libraries unloaded until their endpoints are called. See +[EDGE.md](./EDGE.md). + Recommended OpenAI models: | Model | Dimensions | Notes | diff --git a/docs/EDGE.md b/docs/EDGE.md new file mode 100644 index 000000000..5f5538882 --- /dev/null +++ b/docs/EDGE.md @@ -0,0 +1,81 @@ +# Edge installation + +Signet's edge profile supports constrained 64-bit ARM Linux devices, with the Raspberry Pi 3B+ as the minimum target. It uses the Bun/source daemon so self-contained native-binary assets are not mapped into the idle process. + +## Install + +Use a 64-bit Raspberry Pi OS image, install Git and Bun, then: + +```bash +git clone https://github.com/Signet-AI/signetai.git +cd signetai +bun install --frozen-lockfile +bun run --filter '@signet/core' build +bun surfaces/cli/src/cli.ts setup +``` + +Add the profile to `~/.agents/agent.yaml`: + +```yaml +runtime: + profile: edge + +embedding: + provider: native + model: Xenova/all-MiniLM-L6-v2 + dimensions: 384 +``` + +Start the source runtime: + +```bash +bun surfaces/cli/src/cli.ts daemon start +``` + +Do not replace that command with the globally installed self-contained `signet` binary on the constrained path. The standard binary prioritizes portability and offline assets; the source edge path prioritizes idle RSS. `SIGNET_RUNTIME_PROFILE=edge` is available as a temporary override. + +## What the profile changes + +- Uses a fixed-file 30-second poller instead of Chokidar/recursive watches. +- Leaves the native embedding model unloaded until first use. +- Runs native embeddings in a child process and terminates it after 30 seconds + idle, allowing the OS to reclaim model and ONNX Runtime RSS completely. +- Loads the cl100k tokenizer and inference/OAuth catalogs only when a request + needs them. +- Loads UMAP and Louvain only when their endpoints are called. +- Avoids pre-spawning the synthesis renderer and threaded extraction helper while idle. +- Uses the configured native model/dimensions; the default is the 384-dimensional MiniLM ONNX model. + +The full memory pipeline remains available. Disable optional pipeline features in `agent.yaml` only if your workload does not need them; the edge profile does not silently change memory semantics. + +## Verify the Pi target + +After startup, wait 60 seconds without opening the embeddings status page, then find the daemon PID: + +```bash +cat ~/.agents/.daemon/pid +``` + +Read RSS: + +```bash +bun scripts/check-edge-runtime.ts +``` + +The idle result must report `idleRssPass: true`. Then run the embedding gate +with a warm model cache: + +```bash +bun scripts/check-edge-runtime.ts --with-embedding +``` + +The Pi 3B+ gate is under five seconds. The command exits non-zero when either +the 100 MB idle limit or five-second embedding limit is missed. Record the +Signet version, OS image, architecture, Bun version, cache state, idle RSS, +and elapsed time when reporting results. + +After 30 seconds without another embedding request, re-check `VmRSS`; it must return below 102400 kB. + +## Compatibility + +Existing installs on the old built-in native default migrate to `Xenova/all-MiniLM-L6-v2`/384d once. Custom native models and external embedding providers are untouched. Stored source memories remain intact; dimension/model mismatches are re-embedded by the existing tracker. diff --git a/docs/UPGRADING.md b/docs/UPGRADING.md index 88d1483b7..0ffd439fe 100644 --- a/docs/UPGRADING.md +++ b/docs/UPGRADING.md @@ -1,4 +1,23 @@ -# Upgrade Guide — Inference Provisioning Cutover (#947) +# Upgrade Guide + +## Native embedding default and edge profile (#921) + +Existing configs using the exact built-in `nomic-embed-text-v1.5`/768d +default migrate once to `Xenova/all-MiniLM-L6-v2`/384d. Custom native models, +custom dimensions, Ollama, llama.cpp, OpenAI, and disabled embeddings are not +changed. Source memories remain intact; the existing embedding tracker +re-embeds rows whose model or dimensions no longer match. + +The model is now loaded from `embedding.model` and validated against +`embedding.dimensions`, so custom native settings are no longer ignored. +Verify that custom native model IDs are valid Transformers.js repositories and +that their configured dimensions match their output. + +The optional `runtime.profile: edge` path is documented in +[EDGE.md](./EDGE.md). It uses the source/Bun daemon rather than the +self-contained release binary. + +## Inference provisioning cutover (#947) **Breaking change.** This release replaces Signet's hand-rolled inference providers with two backends: **Pi** (`@earendil-works/pi-ai`) for every direct diff --git a/docs/specs/INDEX.md b/docs/specs/INDEX.md index 9a11019b2..ecc92cdce 100644 --- a/docs/specs/INDEX.md +++ b/docs/specs/INDEX.md @@ -743,6 +743,7 @@ Legend: | `predictive-memory-scorer` | approved | `docs/specs/approved/predictive-memory-scorer.md` | `memory-pipeline-v2`, `knowledge-architecture-schema`, `session-continuity-protocol` | - | | | `multi-agent-support` | approved | `docs/specs/approved/multi-agent-support.md` | `memory-pipeline-v2` | - | | | `signet-runtime` | approved | `docs/specs/approved/signet-runtime.md` | `memory-pipeline-v2` | - | | +| `edge-runtime-profile` | approved | `docs/specs/approved/edge-runtime-profile.md` | `signet-runtime` | - | Raspberry Pi 3B+ resource contract: source-runtime install, lazy MiniLM embeddings, fixed-file polling, and repeatable RSS/latency gates. | | `model-provider-router` | approved | `docs/specs/approved/model-provider-router.md` | `signet-runtime` | - | Shared inference control plane for per-agent rosters, daemon workloads, native RPC, and OpenAI-compatible gateway routing | | `pipeline-pause-control` | complete | `docs/specs/complete/pipeline-pause-control.md` | `memory-pipeline-v2` | - | CLI and dashboard pause/resume shipped on top of live daemon pause/resume API, paused-mode observability, and local Ollama unload | | `daemon-startup-responsiveness` | planning | `docs/specs/planning/daemon-startup-responsiveness.md` | `memory-pipeline-v2` | - | Incident-driven stub for issue #331: keep /health responsive during large-db startup recovery and recover stale managed daemon processes cleanly | diff --git a/docs/specs/approved/edge-runtime-profile.md b/docs/specs/approved/edge-runtime-profile.md new file mode 100644 index 000000000..51cec7001 --- /dev/null +++ b/docs/specs/approved/edge-runtime-profile.md @@ -0,0 +1,49 @@ +--- +title: "Constrained ARM Edge Runtime Profile" +id: edge-runtime-profile +status: approved +informed_by: + - "https://github.com/Signet-AI/signetai/issues/921" +section: "Runtime" +depends_on: + - "signet-runtime" +success_criteria: + - "A Raspberry Pi 3B+ edge installation settles below 100 MB RSS before the lazily loaded embedding model is used." + - "The built-in embedding provider honors the configured model and dimensions and defaults new and legacy-default installs to Xenova/all-MiniLM-L6-v2 at 384 dimensions." + - "One query embedding completes in low single-digit seconds on a Raspberry Pi 3B+ using the native ONNX Runtime ARM64 backend." + - "The edge profile avoids recursive native file watchers, eagerly loaded visualization libraries, and startup embedding inference." + - "The edge install and measurement procedure is documented and repeatable." +scope_boundary: "Defines the supported source-runtime edge profile for 64-bit Linux ARM. The self-contained compiled binary remains the standard install and may retain embedded release assets." +draft_quality: "implementation contract" +--- + +# Constrained ARM Edge Runtime Profile + +The minimum edge target is a Raspberry Pi 3B+ running a 64-bit Linux userspace. The `edge` profile is an explicit resource contract, not an automatic hardware guess. Operators select it with `runtime.profile: edge` in `agent.yaml` or the `SIGNET_RUNTIME_PROFILE=edge` environment variable. + +The edge distribution runs the daemon from the source/Bun runtime tree. It does not use the self-contained compiled executable because that artifact intentionally embeds dashboard, connector, template, skill, worker, and WebAssembly release assets. Keeping the standard binary self-contained and keeping those bytes out of the constrained process are separate packaging contracts. + +## Runtime behavior + +- Native embeddings default to `Xenova/all-MiniLM-L6-v2` with 384 dimensions. The daemon passes the configured model and dimensions to the embedding worker instead of using hidden constants. +- Existing configs using the exact old native default migrate once to MiniLM. Custom models, non-native providers, and custom dimensions are preserved. Existing source memories are not deleted; the embedding tracker re-embeds model/dimension mismatches. +- Source-mode Transformers.js selects `onnxruntime-node`; its published package includes Linux ARM64 native bindings. Inference remains isolated in the existing worker thread. +- The edge daemon does not probe or load the native model during startup. First embedding use loads it in a process-isolated worker host. Thirty seconds after the last native embedding request, that process is terminated so the operating system reclaims its model/runtime memory. +- The cl100k tokenizer vocabulary and inference/OAuth provider catalogs initialize on first use rather than during daemon module evaluation. +- Workspace change detection polls the fixed canonical configuration/identity files every 30 seconds. It does not instantiate Chokidar or recursively watch the agent tree. +- UMAP and Louvain modules load only when their visualization/repair endpoints are invoked. +- Idle synthesis and extraction helper threads are not pre-spawned; the edge profile uses the existing in-process/on-demand paths while preserving pipeline semantics. + +The standard profile retains Chokidar, the startup provider probe, and a five-minute native embedding idle window. + +## Acceptance measurement + +The supported procedure in `docs/EDGE.md` is the release gate: + +1. Use a Raspberry Pi 3B+ with a 64-bit Linux userspace and no swap activity. +2. Start the source runtime with the edge profile and wait 60 seconds without calling an embedding endpoint. +3. Record daemon RSS from `/proc//status`; it must be below 100 MB. +4. Run one warm query embedding and record wall time; it must complete in fewer than five seconds. +5. Wait for the 30-second unload window and verify RSS returns below 100 MB. + +Measurements must name the Signet version, OS image, architecture, Bun version, and whether the model cache was warm. Results from another machine may inform development but do not substitute for the Pi 3B+ release gate. diff --git a/docs/specs/dependencies.yaml b/docs/specs/dependencies.yaml index acb020805..0d4a219c4 100644 --- a/docs/specs/dependencies.yaml +++ b/docs/specs/dependencies.yaml @@ -228,6 +228,23 @@ specs: - "Signet operates as a standalone runtime channel independent of harness-specific connectors" decision: null + - id: edge-runtime-profile + status: approved + path: docs/specs/approved/edge-runtime-profile.md + hard_depends_on: + - signet-runtime + soft_depends_on: [] + blocks: [] + informed_by: + - https://github.com/Signet-AI/signetai/issues/921 + success_criteria: + - "A Raspberry Pi 3B+ edge installation settles below 100 MB RSS before the lazily loaded embedding model is used" + - "The built-in embedding provider honors the configured model and dimensions and defaults new and legacy-default installs to Xenova/all-MiniLM-L6-v2 at 384 dimensions" + - "One query embedding completes in low single-digit seconds on a Raspberry Pi 3B+ using the native ONNX Runtime ARM64 backend" + - "The edge profile avoids recursive native file watchers, eagerly loaded visualization libraries, and startup embedding inference" + - "The edge install and measurement procedure is documented and repeatable" + decision: null + - id: model-provider-router status: approved path: docs/specs/approved/model-provider-router.md diff --git a/platform/core/src/embedding-defaults.ts b/platform/core/src/embedding-defaults.ts new file mode 100644 index 000000000..6fc7ee4b4 --- /dev/null +++ b/platform/core/src/embedding-defaults.ts @@ -0,0 +1,4 @@ +export const DEFAULT_NATIVE_EMBEDDING_MODEL = "Xenova/all-MiniLM-L6-v2"; +export const DEFAULT_NATIVE_EMBEDDING_DIMENSIONS = 384; +export const DEFAULT_NATIVE_EMBEDDING_IDLE_UNLOAD_MS = 5 * 60 * 1000; +export const EDGE_NATIVE_EMBEDDING_IDLE_UNLOAD_MS = 30 * 1000; diff --git a/platform/core/src/index.ts b/platform/core/src/index.ts index beadb7e4a..1f0a23cdb 100644 --- a/platform/core/src/index.ts +++ b/platform/core/src/index.ts @@ -5,6 +5,12 @@ export { Signet } from "./signet"; export { Database, findSqliteVecExtension, loadSqliteVec } from "./database"; +export { + DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + DEFAULT_NATIVE_EMBEDDING_IDLE_UNLOAD_MS, + DEFAULT_NATIVE_EMBEDDING_MODEL, + EDGE_NATIVE_EMBEDDING_IDLE_UNLOAD_MS, +} from "./embedding-defaults"; export { MEMORY_TYPES, EXTRACTION_STATUSES, diff --git a/platform/daemon/build.ts b/platform/daemon/build.ts index 3df528111..6386091d8 100644 --- a/platform/daemon/build.ts +++ b/platform/daemon/build.ts @@ -27,6 +27,8 @@ const targets: Array<{ { entrypoint: "./src/mcp-stdio.ts", outfile: "./dist/mcp-stdio.js" }, { entrypoint: "./src/index.ts", outfile: "./dist/index.js" }, { entrypoint: "./src/synthesis-render-worker.ts", outfile: "./dist/synthesis-render-worker.js" }, + { entrypoint: "./src/embedding-process.ts", outfile: "./dist/embedding-process.js" }, + { entrypoint: "./src/embedding-worker.ts", outfile: "./dist/embedding-worker.js" }, { entrypoint: "./src/pipeline/extraction-thread.ts", outfile: "./dist/extraction-thread.js" }, ]; diff --git a/platform/daemon/src/config-migration.ts b/platform/daemon/src/config-migration.ts index 07737a8e6..66bd8e385 100644 --- a/platform/daemon/src/config-migration.ts +++ b/platform/daemon/src/config-migration.ts @@ -11,6 +11,7 @@ */ import { existsSync, readFileSync, renameSync, writeFileSync } from "node:fs"; import { join } from "node:path"; +import { DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, DEFAULT_NATIVE_EMBEDDING_MODEL } from "@signet/core"; import { type Document, isMap, isPair, parseDocument } from "yaml"; import { logger } from "./logger"; @@ -407,3 +408,60 @@ export function migrateLegacyRoutingToRegistry(agentsDir: string): void { }); } } + +// --------------------------------------------------------------------------- +// v5: move the built-in embedding default to MiniLM (#921) +// --------------------------------------------------------------------------- + +const LEGACY_NATIVE_MODELS = new Set(["nomic-embed-text-v1.5", "nomic-ai/nomic-embed-text-v1.5"]); + +export function migrateNativeEmbeddingModel(agentsDir: string): void { + const path = findConfigPath(agentsDir); + if (!path) return; + + let text: string; + try { + text = readFileSync(path, "utf-8"); + } catch { + return; + } + const vMatch = /^configVersion:\s*(\d+)/m.exec(text); + if (vMatch && Number(vMatch[1]) >= 5) return; + + const doc = parseDocument(text); + if (doc.errors.length > 0) { + logger.warn("config-migration", "Skipping native embedding migration: agent.yaml has parse errors", { + file: path, + errors: doc.errors.map((error) => error.message).slice(0, 3), + }); + return; + } + + const embedding = doc.getIn(["embedding"], true); + let migrated = false; + if (isMap(embedding)) { + const provider = String(embedding.get("provider") ?? ""); + const model = String(embedding.get("model") ?? ""); + const dimensions = embedding.get("dimensions"); + if ( + (provider === "native" || provider === "local") && + LEGACY_NATIVE_MODELS.has(model) && + (dimensions === undefined || dimensions === 768) + ) { + embedding.set("model", DEFAULT_NATIVE_EMBEDDING_MODEL); + embedding.set("dimensions", DEFAULT_NATIVE_EMBEDDING_DIMENSIONS); + migrated = true; + } + } + + stampConfigVersion(doc, 5); + writeAtomic(path, doc.toString()); + if (migrated) { + logger.info("config-migration", "Migrated native embedding model to the edge default", { + file: path, + model: DEFAULT_NATIVE_EMBEDDING_MODEL, + dimensions: DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + note: "Existing rows are re-embedded by the embedding tracker without deleting source memories.", + }); + } +} diff --git a/platform/daemon/src/context-budget.ts b/platform/daemon/src/context-budget.ts index 4bb33ba0c..9fcac88f9 100644 --- a/platform/daemon/src/context-budget.ts +++ b/platform/daemon/src/context-budget.ts @@ -41,7 +41,6 @@ export function selectWithTokenBudget(rows: Reado } const TRUNCATED_MARKER = "\n[context truncated]"; -const TRUNCATED_MARKER_TOKENS = countTokens(TRUNCATED_MARKER); /** * Truncate `inject` to fit within `mainBudget` tokens. @@ -52,7 +51,8 @@ const TRUNCATED_MARKER_TOKENS = countTokens(TRUNCATED_MARKER); export function applyTokenBudget(inject: string, mainBudget: number): string { if (mainBudget <= 0) return ""; if (countTokens(inject) <= mainBudget) return inject; + const truncatedMarkerTokens = countTokens(TRUNCATED_MARKER); // Budget too tight to fit content + marker — truncate without marker. - if (mainBudget <= TRUNCATED_MARKER_TOKENS) return truncateToTokens(inject, mainBudget); - return truncateToTokens(inject, mainBudget - TRUNCATED_MARKER_TOKENS) + TRUNCATED_MARKER; + if (mainBudget <= truncatedMarkerTokens) return truncateToTokens(inject, mainBudget); + return truncateToTokens(inject, mainBudget - truncatedMarkerTokens) + TRUNCATED_MARKER; } diff --git a/platform/daemon/src/daemon.ts b/platform/daemon/src/daemon.ts index 69e5ed582..225594200 100644 --- a/platform/daemon/src/daemon.ts +++ b/platform/daemon/src/daemon.ts @@ -30,13 +30,17 @@ import { resolveDefaultBasePath, stripSignetBlock, } from "@signet/core"; -import { watch } from "chokidar"; import { Hono } from "hono"; import { resolveDaemonAgentId } from "./agent-id"; import { yieldEvery } from "./async-yield"; import { requirePermission } from "./auth"; import { bindWithRetry } from "./bind-with-retry"; -import { migrateConfig, migrateInferenceProviders, migrateLegacyRoutingToRegistry } from "./config-migration"; +import { + migrateConfig, + migrateInferenceProviders, + migrateLegacyRoutingToRegistry, + migrateNativeEmbeddingModel, +} from "./config-migration"; import { listConnectors } from "./connectors/registry"; import { clearAllPresence } from "./cross-agent"; import { closeDbAccessor, getDbAccessor, getVectorRuntimeStatus, initDbAccessorAsync } from "./db-accessor"; @@ -73,6 +77,7 @@ import { invalidateTraversalCache } from "./pipeline/graph-traversal"; import { stopModelRegistry } from "./pipeline/model-registry"; import { configureLlmConcurrency } from "./pipeline/provider"; import { startReconciler } from "./pipeline/skill-reconciler"; +import { startPollingFileWatcher } from "./polling-file-watcher"; import { type RepairContext, structuralBackfill } from "./repair-actions"; import { logFdSnapshot, startEventLoopMonitor, startFdPollMonitor, stopResourceMonitors } from "./resource-monitor"; import { @@ -106,6 +111,7 @@ import { embeddingTrackerHandle as sharedEmbeddingTrackerHandle, shuttingDown, } from "./routes/state.js"; +import { type RuntimeProfileConfig, loadRuntimeProfile } from "./runtime-profile"; import { startSchedulerWorker } from "./scheduler"; import { getSecret } from "./secrets.js"; import { flushPendingCheckpoints, initCheckpointFlush, pruneCheckpoints } from "./session-checkpoints"; @@ -267,7 +273,7 @@ setupDashboardRoutes(app); // File Watcher // ============================================================================ -let watcher: ReturnType | null = null; +let watcher: { close(): Promise | void } | null = null; let nativeMemoryBridge: NativeMemoryBridgeHandle | null = null; // Fast in-process cache layered on top of the persistent legacy_markdown_imports @@ -1004,40 +1010,30 @@ function stopMemoryImportPoller(): void { memoryImportInFlight = false; } -function startFileWatcher() { +async function startFileWatcher(runtime: RuntimeProfileConfig): Promise { // Do NOT watch the memory/ directory directly — Bun's fs.watch() // opens one O_RDONLY FD per file in a watched directory and never // releases them on close(), leaking ~8 000 FDs with canonical // artifacts present. Canonical artifacts and backups are intentionally // ignored; rare legacy non-artifact memory markdown imports are handled // by the lightweight poller started after daemon readiness. - watcher = watch( - [ - join(AGENTS_DIR, "agent.yaml"), - join(AGENTS_DIR, "AGENT.yaml"), - join(AGENTS_DIR, "config.yaml"), - join(AGENTS_DIR, "AGENTS.md"), - join(AGENTS_DIR, "SOUL.md"), - join(AGENTS_DIR, "MEMORY.md"), - join(AGENTS_DIR, "IDENTITY.md"), - join(AGENTS_DIR, "USER.md"), - join(AGENTS_DIR, "SIGNET-ARCHITECTURE.md"), - join(AGENTS_DIR, ".sigignore"), - join(AGENTS_DIR, "agents"), - ], - { - persistent: true, - ignoreInitial: true, - ignored: createAgentsWatcherIgnoreMatcher(AGENTS_DIR), - }, - ); - - watcher.on("error", (error) => { + const fixedPaths = [ + join(AGENTS_DIR, "agent.yaml"), + join(AGENTS_DIR, "AGENT.yaml"), + join(AGENTS_DIR, "config.yaml"), + join(AGENTS_DIR, "AGENTS.md"), + join(AGENTS_DIR, "SOUL.md"), + join(AGENTS_DIR, "MEMORY.md"), + join(AGENTS_DIR, "IDENTITY.md"), + join(AGENTS_DIR, "USER.md"), + join(AGENTS_DIR, "SIGNET-ARCHITECTURE.md"), + join(AGENTS_DIR, ".sigignore"), + ]; + const onError = (error: unknown) => { logger.error("watcher", "File watcher error", error instanceof Error ? error : new Error(String(error))); - }); - - watcher.on("change", (path) => { - logger.info("watcher", "File changed", { path }); + }; + const onChange = (path: string) => { + logger.info("watcher", "File changed", { path, profile: runtime.profile }); scheduleAutoCommit(path); const base = basename(path); @@ -1079,17 +1075,15 @@ function startFileWatcher() { }), ); } - }); - - watcher.on("unlink", (path) => { + }; + const onUnlink = (path: string) => { logger.info("watcher", "File removed", { path }); if (path.endsWith("SIGNET-ARCHITECTURE.md")) { void ensureArchitectureDoc(); } scheduleAutoCommit(path); - }); - - watcher.on("add", (path) => { + }; + const onAdd = (path: string) => { logger.info("watcher", "File added", { path }); scheduleAutoCommit(path); @@ -1106,7 +1100,33 @@ function startFileWatcher() { }), ); } + }; + + if (runtime.watcher === "poll") { + watcher = await startPollingFileWatcher({ + paths: fixedPaths, + intervalMs: 30_000, + onError, + onEvent(event, path) { + if (event === "change") onChange(path); + else if (event === "add") onAdd(path); + else onUnlink(path); + }, + }); + return; + } + + const { watch } = await import("chokidar"); + const chokidarWatcher = watch([...fixedPaths, join(AGENTS_DIR, "agents")], { + persistent: true, + ignoreInitial: true, + ignored: createAgentsWatcherIgnoreMatcher(AGENTS_DIR), }); + chokidarWatcher.on("error", onError); + chokidarWatcher.on("change", onChange); + chokidarWatcher.on("unlink", onUnlink); + chokidarWatcher.on("add", onAdd); + watcher = chokidarWatcher; } // ============================================================================ @@ -1309,6 +1329,7 @@ function syncAgentRoster(agentsDir: string): void { async function startPipelineRuntime(memoryCfg: ResolvedMemoryConfig, telemetry?: TelemetryCollector): Promise { const pipelinePaused = memoryCfg.pipelineV2.paused; + const runtimeProfile = loadRuntimeProfile(AGENTS_DIR); configureLlmConcurrency(memoryCfg.pipelineV2.worker.maxLlmConcurrency); clearStructuralBackfillTimer(); if (shouldWarnGraphExtractionWritesDisabled(memoryCfg)) { @@ -1323,6 +1344,18 @@ async function startPipelineRuntime(memoryCfg: ResolvedMemoryConfig, telemetry?: model: memoryCfg.embedding.model, dimensions: memoryCfg.embedding.dimensions, }); + if (memoryCfg.embedding.provider === "native") { + const { configureNativeEmbeddingAssets } = await import("./native-embedding"); + configureNativeEmbeddingAssets({ + embeddingWorkerPath: resolveEmbeddedWorkerPath("embedding-worker"), + wasmAssetDir: materializeEmbeddedWasmAssets(), + transformersRuntimeAssetPath: resolveEmbeddedWorkerPath("embedding-worker-transformers-runtime"), + modelId: memoryCfg.embedding.model, + expectedDimensions: memoryCfg.embedding.dimensions, + idleUnloadMs: runtimeProfile.embeddingIdleUnloadMs, + isolateProcess: runtimeProfile.embeddingIsolation === "process", + }); + } reloadAuthState(AGENTS_DIR); if (!transcriptCaptureWorkerHandle) { @@ -1353,13 +1386,24 @@ async function startPipelineRuntime(memoryCfg: ResolvedMemoryConfig, telemetry?: } }); - const routerStatus = await router.status(false); - const statusValue = routerStatus.ok ? routerStatus.value : null; + const hasSummaryConsumers = memoryCfg.pipelineV2.enabled || memoryCfg.dreaming.enabled; + // A disabled background pipeline does not need provider availability at + // startup. Deferring the status probe is important on edge systems because + // credential providers (and their crypto runtimes) otherwise become part + // of the pure-idle footprint. Inference API requests still resolve the same + // router on demand. + const shouldResolveBackgroundInference = !pipelinePaused && hasSummaryConsumers; + const routerStatus = shouldResolveBackgroundInference ? await router.status(false) : null; + const statusValue = routerStatus?.ok ? routerStatus.value : null; const explicitInference = statusValue?.source === "explicit"; const commandExtractionConfigured = memoryCfg.pipelineV2.extraction.provider === "command"; - const commandExtractionMode = memoryCfg.pipelineV2.enabled && commandExtractionConfigured; - const extractionWorkloadConfigured = commandExtractionConfigured || (await router.hasWorkload("memory_extraction")); - const synthesisWorkloadConfigured = await router.hasWorkload("session_synthesis"); + const commandExtractionMode = memoryCfg.pipelineV2.enabled && memoryCfg.pipelineV2.extraction.provider === "command"; + const extractionWorkloadConfigured = + memoryCfg.pipelineV2.enabled && + !pipelinePaused && + (commandExtractionMode || (await router.hasWorkload("memory_extraction"))); + const synthesisWorkloadConfigured = + hasSummaryConsumers && !pipelinePaused && (await router.hasWorkload("session_synthesis")); const extractionDecision = !commandExtractionConfigured && extractionWorkloadConfigured ? await router.explain({ agentId: defaultAgentId, operation: "memory_extraction" }) @@ -1475,7 +1519,6 @@ async function startPipelineRuntime(memoryCfg: ResolvedMemoryConfig, telemetry?: // without synthesis; all other queued work requires an effective route. const isSummarySynthesisAvailable = async (): Promise => (await router.explain({ agentId: defaultAgentId, operation: "session_synthesis" }, true)).ok; - const hasSummaryConsumers = memoryCfg.pipelineV2.enabled || memoryCfg.dreaming.enabled; const summarySynthesisAvailable = hasSummaryConsumers ? await isSummarySynthesisAvailable() : false; if (hasSummaryConsumers && !pipelinePaused && (commandExtractionMode || summarySynthesisAvailable)) { ensureSummaryWorker(getDbAccessor(), { @@ -1496,7 +1539,17 @@ async function startPipelineRuntime(memoryCfg: ResolvedMemoryConfig, telemetry?: } if (memoryCfg.pipelineV2.enabled && !pipelinePaused && extractionAvailable) { - const workerInit: WorkerInit | undefined = memoryCfg.pipelineV2.worker.threadedExtraction + const pipelineRuntimeCfg = + runtimeProfile.profile === "edge" + ? { + ...memoryCfg.pipelineV2, + worker: { + ...memoryCfg.pipelineV2.worker, + threadedExtraction: false, + }, + } + : memoryCfg.pipelineV2; + const workerInit: WorkerInit | undefined = pipelineRuntimeCfg.worker.threadedExtraction ? { dbPath: MEMORY_DB, vecExtensionPath: getVectorRuntimeStatus().extensionPath ?? "", @@ -1523,7 +1576,7 @@ async function startPipelineRuntime(memoryCfg: ResolvedMemoryConfig, telemetry?: startPipeline( getDbAccessor(), - memoryCfg.pipelineV2, + pipelineRuntimeCfg, memoryCfg.embedding, fetchEmbedding, memoryCfg.search, @@ -1533,18 +1586,6 @@ async function startPipelineRuntime(memoryCfg: ResolvedMemoryConfig, telemetry?: telemetry, workerInit, ); - - // Configure the main thread's own native embedding handle with the - // same pre-resolved asset paths. This is not strictly necessary on - // the main thread (it has the asset globals), but it ensures - // consistency and makes the main thread path identical to the - // extraction-thread path (#922). - const { configureNativeEmbeddingAssets } = await import("./native-embedding"); - configureNativeEmbeddingAssets({ - embeddingWorkerPath: resolveEmbeddedWorkerPath("embedding-worker"), - wasmAssetDir: materializeEmbeddedWasmAssets(), - transformersRuntimeAssetPath: resolveEmbeddedWorkerPath("embedding-worker-transformers-runtime"), - }); } else { ensureRetentionWorker(getDbAccessor(), DEFAULT_RETENTION); } @@ -1630,6 +1671,7 @@ async function startPipelineRuntime(memoryCfg: ResolvedMemoryConfig, telemetry?: } }, agentsDir: AGENTS_DIR, + watchFiles: runtimeProfile.watcher !== "poll", }); } @@ -1774,6 +1816,7 @@ process.on("unhandledRejection", (reason) => { // ============================================================================ async function main() { + const runtimeProfile = loadRuntimeProfile(AGENTS_DIR); logger.info("daemon", "Signet Daemon starting"); logger.info("daemon", "Agents directory", { path: AGENTS_DIR }); logger.info("daemon", "Network configured", { port: PORT, host: HOST, bindHost: BIND_HOST }); @@ -1788,19 +1831,23 @@ async function main() { startFdPollMonitor(); const { extensionPath } = getVectorRuntimeStatus(); - const bundled = join(__dirname, "synthesis-render-worker.js"); - const workerPath = existsSync(bundled) - ? bundled - : (resolveEmbeddedWorkerPath("synthesis-render-worker") ?? join(__dirname, "synthesis-render-worker.ts")); let synthWorker: Worker | null = null; - try { - synthWorker = new Worker(workerPath, { type: "module" }); - } catch (err) { - logger.warn( - "daemon", - "synthesis worker creation failed — using sync rendering", - err instanceof Error ? err : undefined, - ); + if (runtimeProfile.profile === "standard") { + const bundled = join(__dirname, "synthesis-render-worker.js"); + const workerPath = existsSync(bundled) + ? bundled + : (resolveEmbeddedWorkerPath("synthesis-render-worker") ?? join(__dirname, "synthesis-render-worker.ts")); + try { + synthWorker = new Worker(workerPath, { type: "module" }); + } catch (err) { + logger.warn( + "daemon", + "synthesis worker creation failed — using sync rendering", + err instanceof Error ? err : undefined, + ); + } + } else { + logger.info("daemon", "Edge profile uses on-demand synthesis rendering"); } let synthWorkerReady = false; if (synthWorker) { @@ -1866,6 +1913,7 @@ async function main() { migrateConfig(AGENTS_DIR); migrateInferenceProviders(AGENTS_DIR); migrateLegacyRoutingToRegistry(AGENTS_DIR); + migrateNativeEmbeddingModel(AGENTS_DIR); } catch (err) { logger.warn("config-migration", "Config migration failed; continuing startup", { error: err instanceof Error ? err.message : String(err), @@ -1876,8 +1924,8 @@ async function main() { scheduleAutoCommit(join(AGENTS_DIR, ".gitignore")); } - startFileWatcher(); - logger.info("watcher", "File watcher started"); + await startFileWatcher(runtimeProfile); + logger.info("watcher", "File watcher started", { profile: runtimeProfile.profile, mode: runtimeProfile.watcher }); logFdSnapshot("post-watcher"); await ensureArchitectureDoc(); @@ -2109,7 +2157,13 @@ async function main() { } const startupCfg = loadMemoryConfig(AGENTS_DIR); - if (startupCfg.embedding.provider !== "none") { + const startupRuntimeProfile = loadRuntimeProfile(AGENTS_DIR); + if (startupCfg.embedding.provider === "native" && !startupRuntimeProfile.probeEmbeddingAtStartup) { + logger.info("daemon", "Native embedding model will load on first use", { + profile: startupRuntimeProfile.profile, + model: startupCfg.embedding.model, + }); + } else if (startupCfg.embedding.provider !== "none") { checkEmbeddingProvider(startupCfg.embedding) .then((embeddingStatus) => { if (!embeddingStatus.available) { diff --git a/platform/daemon/src/embedding-process.ts b/platform/daemon/src/embedding-process.ts new file mode 100644 index 000000000..b5b10fcb9 --- /dev/null +++ b/platform/daemon/src/embedding-process.ts @@ -0,0 +1,43 @@ +/** + * Process-isolated host for the native embedding worker. + * + * Worker threads keep inference off the daemon event loop, but native runtime + * mappings and allocator arenas still belong to the daemon process. The edge + * profile launches this small host as a child process, then runs the existing + * embedding worker inside it. Killing the host after the idle window lets the + * operating system reclaim the model and ONNX runtime RSS completely. + */ + +import { existsSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath } from "node:url"; +import { Worker } from "node:worker_threads"; +import type { EmbeddingWorkerInit, MainToWorkerMessage, WorkerToMainMessage } from "./embedding-worker-protocol"; + +const encodedInit = process.env.SIGNET_EMBEDDING_PROCESS_INIT; +if (!encodedInit) throw new Error("SIGNET_EMBEDDING_PROCESS_INIT is required"); +Reflect.deleteProperty(process.env, "SIGNET_EMBEDDING_PROCESS_INIT"); + +const init = JSON.parse(Buffer.from(encodedInit, "base64url").toString("utf8")) as EmbeddingWorkerInit; +const currentDir = dirname(fileURLToPath(import.meta.url)); +const bundledWorker = join(currentDir, "embedding-worker.js"); +const workerPath = existsSync(bundledWorker) ? bundledWorker : join(currentDir, "embedding-worker.ts"); +const worker = new Worker(workerPath, { workerData: init }); + +worker.on("message", (message: WorkerToMainMessage) => { + process.send?.(message); +}); +worker.on("error", (error) => { + process.send?.({ type: "error", error: error.message, stack: error.stack } satisfies WorkerToMainMessage); +}); +worker.on("exit", (code) => { + process.exit(code); +}); + +process.on("message", (message: MainToWorkerMessage) => { + if (message.type === "shutdown") { + void worker.terminate().finally(() => process.exit(0)); + return; + } + worker.postMessage(message); +}); diff --git a/platform/daemon/src/embedding-tracker.test.ts b/platform/daemon/src/embedding-tracker.test.ts index c3fa59e2e..4b5a6a178 100644 --- a/platform/daemon/src/embedding-tracker.test.ts +++ b/platform/daemon/src/embedding-tracker.test.ts @@ -1,5 +1,6 @@ import { describe, expect, it } from "bun:test"; -import { computeEmbeddingRetryBackoffMs, processEmbeddingCycle } from "./embedding-tracker"; +import type { DbAccessor } from "./db-accessor"; +import { computeEmbeddingRetryBackoffMs, processEmbeddingCycle, startEmbeddingTracker } from "./embedding-tracker"; const cfg = { provider: "ollama", @@ -90,3 +91,27 @@ describe("processEmbeddingCycle", () => { expect(after.failed).toBe(0); }); }); + +describe("startEmbeddingTracker", () => { + it("does not probe or load the provider when no stale rows exist", async () => { + let providerChecks = 0; + const accessor = { + withReadDb(callback: (db: { prepare(): { all(): unknown[] } }) => T): T { + return callback({ prepare: () => ({ all: () => [] }) }); + }, + } as unknown as DbAccessor; + const handle = startEmbeddingTracker( + accessor, + { provider: "native", model: "test", dimensions: 3, base_url: "" }, + { enabled: true, pollMs: 5, batchSize: 8 }, + async () => null, + async () => { + providerChecks++; + return { available: true }; + }, + ); + await Bun.sleep(20); + await handle.stop(); + expect(providerChecks).toBe(0); + }); +}); diff --git a/platform/daemon/src/embedding-tracker.ts b/platform/daemon/src/embedding-tracker.ts index 81741908d..ef12a931d 100644 --- a/platform/daemon/src/embedding-tracker.ts +++ b/platform/daemon/src/embedding-tracker.ts @@ -157,17 +157,22 @@ export function startEmbeddingTracker( if (!running) return; try { - // 1. Check provider health (uses existing 30s cache) + // Query first so an idle, fully-embedded workspace does not load the + // native model merely to prove there is no work. + const staleRows = accessor.withReadDb((db) => { + return listStaleEmbeddingRows(db, embeddingCfg.model, trackerCfg.batchSize) as StaleRow[]; + }); + lastQueueDepth = staleRows.length; + lastCycleAt = new Date().toISOString(); + if (staleRows.length === 0) return; + + // Check provider health only when work exists (uses existing 30s cache). const health = await checkProviderFn(embeddingCfg); if (!health.available) { skippedCycles++; return; } - // 2. Query stale/missing embeddings (read-only) - const staleRows = accessor.withReadDb((db) => { - return listStaleEmbeddingRows(db, embeddingCfg.model, trackerCfg.batchSize) as StaleRow[]; - }); const cycle = await processEmbeddingCycle(staleRows, failures, embeddingCfg, trackerCfg.pollMs, fetchEmbeddingFn); lastQueueDepth = cycle.queueDepth; diff --git a/platform/daemon/src/embedding-worker-handle.test.ts b/platform/daemon/src/embedding-worker-handle.test.ts index 94818108d..55f201ffa 100644 --- a/platform/daemon/src/embedding-worker-handle.test.ts +++ b/platform/daemon/src/embedding-worker-handle.test.ts @@ -120,7 +120,7 @@ describe("embedding-worker-handle", () => { const p = handle.checkAvailable(); await flush(); worker.emit({ type: "status", status: { initialized: true, initializing: false, modelCached: true, error: null } }); - worker.emit({ type: "check_result", id: lastCheckId(worker), available: true }); + worker.emit({ type: "check_result", id: lastCheckId(worker), available: true, error: null }); const status = await p; expect(status.available).toBe(true); @@ -270,6 +270,27 @@ describe("embedding-worker-handle", () => { await expect(handle.embed("after")).rejects.toThrow(/shut down/); }); + it("selects the process-isolated host for the edge profile", async () => { + const { worker, factory: processFactory } = fakePair(); + let capturedPath = ""; + const trackingProcessFactory: EmbeddingWorkerFactory = (path, init, options) => { + capturedPath = path; + return processFactory(path, init, options); + }; + const handle = await createEmbeddingWorkerHandle({ + processFactory: trackingProcessFactory, + isolateProcess: true, + expectedDimensions: DIM, + }); + worker.emit({ type: "ready" }); + handles.push(handle); + + expect(capturedPath).toMatch(/embedding-process\.(?:js|ts)$/); + await handle.stop(); + expect(worker.posted.some((message) => message.type === "shutdown")).toBe(true); + expect(worker.terminated).toBe(true); + }); + it("keeps a timed-out inference worker disabled after the cooldown window", async () => { const { worker, factory } = fakePair(); const handle = await makeHandle(worker, factory, { embedTimeoutMs: 40, cooldownMs: 80 }); diff --git a/platform/daemon/src/embedding-worker-handle.ts b/platform/daemon/src/embedding-worker-handle.ts index b934a734c..8265af930 100644 --- a/platform/daemon/src/embedding-worker-handle.ts +++ b/platform/daemon/src/embedding-worker-handle.ts @@ -21,11 +21,16 @@ * ready handshake with timeout, injectable workerFactory for tests. */ +import { fork } from "node:child_process"; import { existsSync } from "node:fs"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; import { Worker, type WorkerOptions } from "node:worker_threads"; -import { resolveDefaultBasePath } from "@signet/core"; +import { + DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + DEFAULT_NATIVE_EMBEDDING_MODEL, + resolveDefaultBasePath, +} from "@signet/core"; import type { EmbeddingWorkerInit, EmbeddingWorkerStatus, @@ -39,8 +44,6 @@ import { materializeEmbeddedWasmAssets, resolveEmbeddedWorkerPath } from "./nati // Constants (overridable via options for deterministic tests) // --------------------------------------------------------------------------- -const DEFAULT_MODEL_ID = "nomic-ai/nomic-embed-text-v1.5"; -const DEFAULT_DIMENSIONS = 768; const READY_TIMEOUT_MS = 30_000; const INIT_TIMEOUT_MS = 90_000; // first-run model download can take a while const EMBED_TIMEOUT_MS = 15_000; @@ -71,10 +74,17 @@ export interface EmbeddingHandleOptions { /** Test seam: point transformers env.remoteHost at a local blackhole. */ readonly remoteHostOverride?: string; readonly workerFactory?: EmbeddingWorkerFactory; + /** Test seam for the edge child-process adapter. */ + readonly processFactory?: EmbeddingWorkerFactory; readonly readyTimeoutMs?: number; readonly initTimeoutMs?: number; readonly embedTimeoutMs?: number; readonly cooldownMs?: number; + /** + * Run the worker inside a child process so terminating an idle provider + * returns native runtime/model pages to the OS. Used by the edge profile. + */ + readonly isolateProcess?: boolean; /** * Pre-resolved embedding worker executable path. When provided (including * null), skips the native asset registry resolution. Needed when this @@ -128,8 +138,8 @@ export interface EmbeddingWorkerHandle { // --------------------------------------------------------------------------- export async function createEmbeddingWorkerHandle(opts: EmbeddingHandleOptions = {}): Promise { - const modelId = opts.modelId ?? DEFAULT_MODEL_ID; - const dimensions = opts.expectedDimensions ?? DEFAULT_DIMENSIONS; + const modelId = opts.modelId ?? DEFAULT_NATIVE_EMBEDDING_MODEL; + const dimensions = opts.expectedDimensions ?? DEFAULT_NATIVE_EMBEDDING_DIMENSIONS; const readyTimeoutMs = opts.readyTimeoutMs ?? READY_TIMEOUT_MS; const initTimeoutMs = opts.initTimeoutMs ?? INIT_TIMEOUT_MS; const embedTimeoutMs = opts.embedTimeoutMs ?? EMBED_TIMEOUT_MS; @@ -181,7 +191,14 @@ export async function createEmbeddingWorkerHandle(opts: EmbeddingHandleOptions = ? bundled : (resolveEmbeddedWorkerPath("embedding-worker") ?? join(__dirname, "embedding-worker.ts")); const workerOptions = { workerData: init, type: "module" } as const; - const worker = (opts.workerFactory ?? createNodeWorker)(workerPath, init, workerOptions); + const processPath = existsSync(join(__dirname, "embedding-process.js")) + ? join(__dirname, "embedding-process.js") + : join(__dirname, "embedding-process.ts"); + const worker = opts.workerFactory + ? opts.workerFactory(workerPath, init, workerOptions) + : opts.isolateProcess + ? (opts.processFactory ?? createNodeProcessWorker)(processPath, init, workerOptions) + : createNodeWorker(workerPath, init, workerOptions); let nextId = 1; const pending = new Map(); @@ -449,3 +466,38 @@ function createNodeWorker(workerPath: string, _init: EmbeddingWorkerInit, option // (on/postMessage/terminate). Exposed as a factory so tests inject a fake. return new Worker(workerPath, options) as unknown as EmbeddingWorkerLike; } + +function createNodeProcessWorker( + processPath: string, + init: EmbeddingWorkerInit, + _options: WorkerOptions, +): EmbeddingWorkerLike { + const child = fork(processPath, [], { + env: { + ...process.env, + SIGNET_EMBEDDING_PROCESS_INIT: Buffer.from(JSON.stringify(init), "utf8").toString("base64url"), + }, + stdio: ["ignore", "inherit", "inherit", "ipc"], + }); + + return { + on(event, listener) { + if (event === "message") { + child.on("message", listener as (message: unknown) => void); + } else { + child.on(event, listener); + } + return child; + }, + postMessage(message) { + child.send(message); + }, + terminate() { + if (child.exitCode !== null || child.signalCode !== null) return 0; + return new Promise((resolve) => { + child.once("exit", (code) => resolve(code ?? 0)); + child.kill(); + }); + }, + }; +} diff --git a/platform/daemon/src/inference-oauth.test.ts b/platform/daemon/src/inference-oauth.test.ts index af63cc19f..a3fffa1fd 100644 --- a/platform/daemon/src/inference-oauth.test.ts +++ b/platform/daemon/src/inference-oauth.test.ts @@ -70,13 +70,13 @@ describe("inference OAuth", () => { }); test("streams interactive login events and stores credentials only in the daemon", async () => { - expect(listOAuthProviderMetadata()).toContainEqual({ + expect(await listOAuthProviderMetadata()).toContainEqual({ id: PROVIDER_ID, name: "Signet test OAuth", usesCallbackServer: false, }); - const login = startOAuthLogin(PROVIDER_ID); + const login = await startOAuthLogin(PROVIDER_ID); const reader = login.stream.getReader(); const initial = await readUntil(reader, (text) => text.includes('"type":"prompt"')); expect(initial).toContain("https://example.test/login"); @@ -140,7 +140,7 @@ describe("inference OAuth", () => { test("rejects invalid provider ids before touching secret storage", async () => { await expect(loadOAuthCredentials("../../escape")).rejects.toThrow("Invalid OAuth provider id"); - expect(() => startOAuthLogin("missing-provider")).toThrow("Unknown OAuth provider"); + await expect(startOAuthLogin("missing-provider")).rejects.toThrow("Unknown OAuth provider"); }); test("accepts only selection values offered by the OAuth provider", async () => { @@ -155,7 +155,7 @@ describe("inference OAuth", () => { }, }), ); - const login = startOAuthLogin(PROVIDER_ID); + const login = await startOAuthLogin(PROVIDER_ID); const reader = login.stream.getReader(); const initial = await readUntil(reader, (text) => text.includes('"type":"select"')); const selectData = initial diff --git a/platform/daemon/src/inference-oauth.ts b/platform/daemon/src/inference-oauth.ts index ca7163254..c3490dc60 100644 --- a/platform/daemon/src/inference-oauth.ts +++ b/platform/daemon/src/inference-oauth.ts @@ -1,16 +1,21 @@ import { randomUUID } from "node:crypto"; -import { - type OAuthCredentials, - type OAuthLoginCallbacks, - type OAuthPrompt, - type OAuthSelectPrompt, - getOAuthApiKey, - getOAuthProvider, - getOAuthProviders, +import type { + OAuthCredentials, + OAuthLoginCallbacks, + OAuthPrompt, + OAuthSelectPrompt, } from "@earendil-works/pi-ai/oauth"; import { logger } from "./logger"; import { deleteSecretFromActiveProvider, getSecret, putSecret } from "./secrets"; +let oauthRuntime: Promise | null = null; + +function getOAuthRuntime(): Promise { + if (oauthRuntime) return oauthRuntime; + oauthRuntime = import("@earendil-works/pi-ai/oauth"); + return oauthRuntime; +} + const OAUTH_SECRET_PREFIX = "SIGNET_OAUTH_"; const OAUTH_SESSION_TTL_MS = 10 * 60_000; const MAX_ACTIVE_OAUTH_SESSIONS = 8; @@ -93,22 +98,21 @@ function safeError(error: unknown): string { .slice(0, 500); } -export function listOAuthProviderMetadata(): Array<{ - readonly id: string; - readonly name: string; - readonly usesCallbackServer: boolean; -}> { - return getOAuthProviders().map((provider) => ({ +export async function listOAuthProviderMetadata(): Promise< + Array<{ + readonly id: string; + readonly name: string; + readonly usesCallbackServer: boolean; + }> +> { + const runtime = await getOAuthRuntime(); + return runtime.getOAuthProviders().map((provider) => ({ id: provider.id, name: provider.name, usesCallbackServer: provider.usesCallbackServer === true, })); } -export function isOAuthProvider(providerId: string): boolean { - return getOAuthProvider(providerId) !== undefined; -} - export async function loadOAuthCredentials(providerId: string): Promise { const name = secretName(providerId); let raw: string; @@ -132,13 +136,15 @@ export async function loadOAuthCredentials(providerId: string): Promise { - if (!getOAuthProvider(validateProviderId(providerId))) throw new Error(`Unknown OAuth provider: ${providerId}`); + if (!(await getOAuthRuntime()).getOAuthProvider(validateProviderId(providerId))) + throw new Error(`Unknown OAuth provider: ${providerId}`); if (!isOAuthCredentials(credentials)) throw new Error(`Invalid OAuth credentials for ${providerId}`); await putSecret(secretName(providerId), JSON.stringify(credentials)); } export async function disconnectOAuthProvider(providerId: string): Promise { - if (!getOAuthProvider(validateProviderId(providerId))) throw new Error(`Unknown OAuth provider: ${providerId}`); + if (!(await getOAuthRuntime()).getOAuthProvider(validateProviderId(providerId))) + throw new Error(`Unknown OAuth provider: ${providerId}`); return deleteSecretFromActiveProvider(secretName(providerId)); } @@ -154,9 +160,9 @@ export async function resolveOAuthCredential(providerId: string): Promise { const credentials = await loadOAuthCredentials(normalized); if (!credentials) return null; - let result: Awaited>; + let result: Awaited>; try { - result = await getOAuthApiKey(normalized, { [normalized]: credentials }); + result = await (await getOAuthRuntime()).getOAuthApiKey(normalized, { [normalized]: credentials }); } catch (error) { if (!(error instanceof Error) || error.message !== `Failed to refresh OAuth token for ${normalized}`) { throw error; @@ -203,15 +209,15 @@ function cleanupSession(session: OAuthLoginSession, reason?: string): void { session.pending.clear(); } -export function startOAuthLogin( +export async function startOAuthLogin( providerId: string, onCredentialsChanged?: () => void, -): { +): Promise<{ readonly sessionId: string; readonly stream: ReadableStream; -} { +}> { const normalized = validateProviderId(providerId); - const provider = getOAuthProvider(normalized); + const provider = (await getOAuthRuntime()).getOAuthProvider(normalized); if (!provider) throw new Error(`Unknown OAuth provider: ${normalized}`); if (activeSessions.size >= MAX_ACTIVE_OAUTH_SESSIONS) throw new Error("Too many active OAuth login sessions"); diff --git a/platform/daemon/src/inference-provider-factory.ts b/platform/daemon/src/inference-provider-factory.ts index 1f19b8115..2781a8fc8 100644 --- a/platform/daemon/src/inference-provider-factory.ts +++ b/platform/daemon/src/inference-provider-factory.ts @@ -1,7 +1,6 @@ -import { type Api, type Model, type OAuthCredentials, getModels, getProviders } from "@earendil-works/pi-ai"; -import { getOAuthProvider } from "@earendil-works/pi-ai/oauth"; +import type { Api, Model, OAuthCredentials } from "@earendil-works/pi-ai"; import type { PipelineClaudeCodeConfig, RoutingAccountConfig, RoutingConfig } from "@signet/core"; -import { type PiExecutorKind, createPiModelProvider } from "./pipeline/pi-provider"; +import type { PiExecutorKind } from "./pipeline/pi-provider"; import type { AcpxHooksMode, StreamCapableLlmProvider } from "./pipeline/provider"; import { createAcpxProvider } from "./pipeline/provider"; @@ -32,10 +31,12 @@ function catalogModel( providerFamily: string, modelId: string, credential: ResolvedInferenceCredential | undefined, + pi: typeof import("@earendil-works/pi-ai"), + oauth: typeof import("@earendil-works/pi-ai/oauth"), ): Model | undefined { - if (!(getProviders() as readonly string[]).includes(providerFamily)) return undefined; - let models = (getModels as (provider: string) => Model[])(providerFamily); - const oauthProvider = getOAuthProvider(providerFamily); + if (!(pi.getProviders() as readonly string[]).includes(providerFamily)) return undefined; + let models = (pi.getModels as (provider: string) => Model[])(providerFamily); + const oauthProvider = oauth.getOAuthProvider(providerFamily); if (oauthProvider?.modifyModels && credential?.oauthCredentials) { models = oauthProvider.modifyModels(models, credential.oauthCredentials); } @@ -66,14 +67,19 @@ export async function createRoutingProvider(opts: CreateRoutingProviderOptions): const account = target.account ? opts.config.accounts[target.account] : undefined; const providerFamily = account?.providerFamily ?? target.executor; - if (!CUSTOM_PI_EXECUTORS.has(target.executor) && !(getProviders() as readonly string[]).includes(providerFamily)) { + const [pi, oauth, { createPiModelProvider }] = await Promise.all([ + import("@earendil-works/pi-ai"), + import("@earendil-works/pi-ai/oauth"), + import("./pipeline/pi-provider"), + ]); + if (!CUSTOM_PI_EXECUTORS.has(target.executor) && !(pi.getProviders() as readonly string[]).includes(providerFamily)) { throw new Error(`Unsupported routing executor "${target.executor}" for target ${opts.targetId}`); } const credential = await opts.resolveCredential(account); const piModel = CUSTOM_PI_EXECUTORS.has(target.executor) ? undefined - : catalogModel(providerFamily, model.model, credential); + : catalogModel(providerFamily, model.model, credential, pi, oauth); if (!piModel && !CUSTOM_PI_EXECUTORS.has(target.executor)) { throw new Error(`Unknown pi-ai model "${model.model}" for provider "${providerFamily}"`); } diff --git a/platform/daemon/src/inference-router.ts b/platform/daemon/src/inference-router.ts index 40d0ad36f..5993eabad 100644 --- a/platform/daemon/src/inference-router.ts +++ b/platform/daemon/src/inference-router.ts @@ -26,7 +26,7 @@ import { resolveRoutingDecision, validateRoutingReferences, } from "@signet/core"; -import { isOAuthProvider, resolveOAuthCredential } from "./inference-oauth"; +import { resolveOAuthCredential } from "./inference-oauth"; import { type ResolvedInferenceCredential, createRoutingProvider } from "./inference-provider-factory"; import { logger } from "./logger"; import { loadMemoryConfig } from "./memory-config"; @@ -245,12 +245,8 @@ function isRuntimeBlocked(state: RoutingRuntimeState): boolean { ); } -function isOAuthBackedAccount(account: RoutingAccountConfig | undefined): account is RoutingAccountConfig { - return ( - account !== undefined && - isOAuthProvider(account.providerFamily) && - (account.kind === "subscription_session" || !account.credentialRef) - ); +function isOAuthBackedAccount(account: RoutingAccountConfig | undefined): boolean { + return account?.kind === "subscription_session"; } function buildPromptFromMessages(messages: ReadonlyArray<{ readonly role: string; readonly content: string }>): string { @@ -592,7 +588,7 @@ export class InferenceRouter { private async resolveCredential( account: RoutingAccountConfig | undefined, ): Promise { - if (isOAuthBackedAccount(account)) { + if (account && isOAuthBackedAccount(account)) { const oauth = await resolveOAuthCredential(account.providerFamily); if (oauth) return { apiKey: oauth.apiKey, oauthCredentials: oauth.credentials }; } diff --git a/platform/daemon/src/memory-config.test.ts b/platform/daemon/src/memory-config.test.ts index c701a18e3..79a70fa2b 100644 --- a/platform/daemon/src/memory-config.test.ts +++ b/platform/daemon/src/memory-config.test.ts @@ -2,9 +2,11 @@ import { afterEach, describe, expect, it } from "bun:test"; import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { tmpdir } from "node:os"; import { join } from "node:path"; +import { DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, DEFAULT_NATIVE_EMBEDDING_MODEL } from "@signet/core"; import { DEFAULT_LLAMACPP_MAX_INPUT_TOKENS, DEFAULT_PIPELINE_V2, + MAX_EMBEDDING_MODEL_LENGTH, MAX_LLAMACPP_MAX_INPUT_TOKENS, MAX_PROMPT_SUBMIT_EMBEDDING_TIMEOUT_MS, MIN_LLAMACPP_MAX_INPUT_TOKENS, @@ -95,12 +97,33 @@ describe("loadMemoryConfig", () => { const agentsDir = makeTempAgentsDir(); const cfg = loadMemoryConfig(agentsDir); expect(cfg.embedding.provider).toBe("native"); - expect(cfg.embedding.model).toBe("nomic-embed-text-v1.5"); - expect(cfg.embedding.dimensions).toBe(768); + expect(cfg.embedding.model).toBe(DEFAULT_NATIVE_EMBEDDING_MODEL); + expect(cfg.embedding.dimensions).toBe(DEFAULT_NATIVE_EMBEDDING_DIMENSIONS); expect(cfg.embedding.promptSubmitTimeoutMs).toBe(MIN_PROMPT_SUBMIT_EMBEDDING_TIMEOUT_MS); expect(cfg.embedding.llamaCppMaxInputTokens).toBe(DEFAULT_LLAMACPP_MAX_INPUT_TOKENS); }); + it("bounds invalid embedding dimensions to the provider default", () => { + const agentsDir = makeTempAgentsDir(); + writeFileSync( + join(agentsDir, "agent.yaml"), + "embedding:\n provider: native\n model: Xenova/all-MiniLM-L6-v2\n dimensions: -1\n", + ); + expect(loadMemoryConfig(agentsDir).embedding.dimensions).toBe(DEFAULT_NATIVE_EMBEDDING_DIMENSIONS); + }); + + it("bounds embedding model identifiers before passing config to a worker process", () => { + const agentsDir = makeTempAgentsDir(); + writeFileSync( + join(agentsDir, "agent.yaml"), + `embedding: + provider: native + model: ${"m".repeat(MAX_EMBEDDING_MODEL_LENGTH + 1)} +`, + ); + expect(loadMemoryConfig(agentsDir).embedding.model).toBe(DEFAULT_NATIVE_EMBEDDING_MODEL); + }); + it("deliberately enables the bounded freshness prior by default and allows opt-out", () => { const defaultDir = makeTempAgentsDir(); const defaults = loadMemoryConfig(defaultDir); diff --git a/platform/daemon/src/memory-config.ts b/platform/daemon/src/memory-config.ts index f3db62631..207336166 100644 --- a/platform/daemon/src/memory-config.ts +++ b/platform/daemon/src/memory-config.ts @@ -1,6 +1,8 @@ import { existsSync, readFileSync } from "node:fs"; import { join } from "node:path"; import { + DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + DEFAULT_NATIVE_EMBEDDING_MODEL, DEFAULT_PIPELINE_TIMEOUT_MS, DEFAULT_PROVIDER_RATE_LIMIT, type DreamingConfig, @@ -279,6 +281,9 @@ export const DEFAULT_OPENAI_BASE_URL = "https://api.openai.com/v1"; export const DEFAULT_PROMPT_SUBMIT_EMBEDDING_TIMEOUT_MS = 1000; export const MIN_PROMPT_SUBMIT_EMBEDDING_TIMEOUT_MS = 1000; export const MAX_PROMPT_SUBMIT_EMBEDDING_TIMEOUT_MS = 300000; +export const MIN_EMBEDDING_DIMENSIONS = 1; +export const MAX_EMBEDDING_DIMENSIONS = 65_536; +export const MAX_EMBEDDING_MODEL_LENGTH = 512; export interface ResolvedMemoryConfig { embedding: EmbeddingConfig; @@ -1122,8 +1127,8 @@ export function loadMemoryConfig(agentsDir: string): ResolvedMemoryConfig { const defaults: ResolvedMemoryConfig = { embedding: { provider: "native", - model: "nomic-embed-text-v1.5", - dimensions: 768, + model: DEFAULT_NATIVE_EMBEDDING_MODEL, + dimensions: DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, base_url: "", promptSubmitTimeoutMs: DEFAULT_PROMPT_SUBMIT_EMBEDDING_TIMEOUT_MS, llamaCppMaxInputTokens: DEFAULT_LLAMACPP_MAX_INPUT_TOKENS, @@ -1178,8 +1183,33 @@ export function loadMemoryConfig(agentsDir: string): ResolvedMemoryConfig { const rawProvider = String(emb.provider); defaults.embedding.provider = rawProvider === "local" ? "native" : (rawProvider as "native" | "llama-cpp" | "ollama" | "openai"); - defaults.embedding.model = (emb.model as string | undefined) ?? defaults.embedding.model; - defaults.embedding.dimensions = Number.parseInt(String(emb.dimensions ?? "768"), 10); + const configuredModel = typeof emb.model === "string" ? emb.model.trim() : ""; + if (configuredModel && configuredModel.length <= MAX_EMBEDDING_MODEL_LENGTH) { + defaults.embedding.model = configuredModel; + } else if (configuredModel) { + logger.warn("config", "Ignoring invalid embedding.model", { + length: configuredModel.length, + maxLength: MAX_EMBEDDING_MODEL_LENGTH, + fallback: defaults.embedding.model, + }); + } + if (emb.dimensions !== undefined) { + const dimensions = Number(emb.dimensions); + if ( + Number.isSafeInteger(dimensions) && + dimensions >= MIN_EMBEDDING_DIMENSIONS && + dimensions <= MAX_EMBEDDING_DIMENSIONS + ) { + defaults.embedding.dimensions = dimensions; + } else { + logger.warn("config", "Ignoring invalid embedding.dimensions", { + value: emb.dimensions, + min: MIN_EMBEDDING_DIMENSIONS, + max: MAX_EMBEDDING_DIMENSIONS, + fallback: defaults.embedding.dimensions, + }); + } + } const explicitBaseUrl = (typeof emb.base_url === "string" ? emb.base_url : undefined) ?? (typeof emb.endpoint === "string" ? emb.endpoint : undefined); diff --git a/platform/daemon/src/memory-lineage.ts b/platform/daemon/src/memory-lineage.ts index 238fca2c9..a7a36ee38 100644 --- a/platform/daemon/src/memory-lineage.ts +++ b/platform/daemon/src/memory-lineage.ts @@ -5,8 +5,6 @@ import { mkdir, readFile, readdir, rename, stat, writeFile } from "node:fs/promi import { basename, join } from "node:path"; import type { LlmProvider } from "@signet/core"; import { resolveDefaultBasePath } from "@signet/core"; -import { Tiktoken } from "js-tiktoken/lite"; -import cl100k_base from "js-tiktoken/ranks/cl100k_base"; import { getAgentScope } from "./agent-id"; import { yieldEvery } from "./async-yield"; import type { WriteDb } from "./db-accessor"; @@ -15,6 +13,7 @@ import { logger } from "./logger"; import { MEMORY_HEAD_MAX_TOKENS } from "./memory-head"; import { buildAgentScopeClause } from "./memory-search"; import { NATIVE_MEMORY_BRIDGE_SOURCE_NODE_ID } from "./native-memory-constants"; +import { countTokens } from "./pipeline/tokenizer"; import { isNoiseSession, isTempProject } from "./session-noise"; import { canonicalTranscriptRelativePath } from "./transcript-jsonl"; @@ -37,7 +36,6 @@ export const NOISE_PURGE_REASON = "automatic projection cleanup for temp/test se const REINDEX_BATCH_SIZE = 50; const BASE32 = "abcdefghijklmnopqrstuvwxyz234567"; -let projTok: Tiktoken | null = null; const purgeSeen = new Set(); // Incremental index cache: outer key = agentId or "*" for global, inner key = absolute path, value = stat fingerprint @@ -49,12 +47,6 @@ const lastChangedManifestsByAgent = new Map>(); // Tracks which manifest rel paths were referenced in the previous ledger render per agent const prevLedgerRefsByAgent = new Map>(); -function getProjectionTokenizer(): Tiktoken { - if (projTok) return projTok; - projTok = new Tiktoken(cl100k_base); - return projTok; -} - export type ArtifactKind = "summary" | "transcript" | "compaction" | "manifest"; type SentenceQuality = "ok" | "fallback"; @@ -300,7 +292,7 @@ function pickAnchor(body: string, project: string | null, harness: string | null } function tokenCount(text: string): number { - return getProjectionTokenizer().encode(text).length; + return countTokens(text); } function joinParts(parts: ReadonlyArray): string { diff --git a/platform/daemon/src/native-embedding-model-migration.test.ts b/platform/daemon/src/native-embedding-model-migration.test.ts new file mode 100644 index 000000000..b76e26f4e --- /dev/null +++ b/platform/daemon/src/native-embedding-model-migration.test.ts @@ -0,0 +1,59 @@ +import { afterEach, describe, expect, it } from "bun:test"; +import { mkdtempSync, readFileSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, DEFAULT_NATIVE_EMBEDDING_MODEL } from "@signet/core"; +import { migrateNativeEmbeddingModel } from "./config-migration"; + +const dirs: string[] = []; + +afterEach(() => { + for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); +}); + +function migrate(contents: string): string { + const dir = mkdtempSync(join(tmpdir(), "signet-embedding-migration-")); + dirs.push(dir); + writeFileSync(join(dir, "agent.yaml"), contents); + migrateNativeEmbeddingModel(dir); + return readFileSync(join(dir, "agent.yaml"), "utf8"); +} + +describe("native embedding model migration", () => { + it("moves only the exact legacy native default to MiniLM", () => { + const result = migrate(`configVersion: 4 +embedding: + provider: native + model: nomic-embed-text-v1.5 + dimensions: 768 +`); + expect(result).toContain(`model: ${DEFAULT_NATIVE_EMBEDDING_MODEL}`); + expect(result).toContain(`dimensions: ${DEFAULT_NATIVE_EMBEDDING_DIMENSIONS}`); + expect(result).toMatch(/^configVersion: 5/m); + }); + + it("preserves custom native models and dimensions", () => { + const result = migrate(`configVersion: 4 +embedding: + provider: native + model: owner/custom-model + dimensions: 512 +`); + expect(result).toContain("model: owner/custom-model"); + expect(result).toContain("dimensions: 512"); + }); + + it("is idempotent", () => { + const first = migrate(`configVersion: 4 +embedding: + provider: native + model: nomic-ai/nomic-embed-text-v1.5 + dimensions: 768 +`); + const dir = mkdtempSync(join(tmpdir(), "signet-embedding-migration-idempotent-")); + dirs.push(dir); + writeFileSync(join(dir, "agent.yaml"), first); + migrateNativeEmbeddingModel(dir); + expect(readFileSync(join(dir, "agent.yaml"), "utf8")).toBe(first); + }); +}); diff --git a/platform/daemon/src/native-embedding.test.ts b/platform/daemon/src/native-embedding.test.ts index 16b8d3eae..c8a50f302 100644 --- a/platform/daemon/src/native-embedding.test.ts +++ b/platform/daemon/src/native-embedding.test.ts @@ -54,15 +54,28 @@ class FakeWorker implements EmbeddingWorkerLike { // and reset — the properties callers rely on. describe("native-embedding facade (worker-backed)", () => { let worker: FakeWorker; + let init: EmbeddingWorkerInit | null; beforeEach(() => { worker = new FakeWorker(); + init = null; const factory: EmbeddingWorkerFactory = ( _path: string, - _init: EmbeddingWorkerInit, + workerInit: EmbeddingWorkerInit, _options: WorkerOptions, - ): EmbeddingWorkerLike => worker; + ): EmbeddingWorkerLike => { + init = workerInit; + return worker; + }; __setEmbeddingWorkerFactoryForTests(factory); + configureNativeEmbeddingAssets({ + embeddingWorkerPath: null, + wasmDir: null, + transformersRuntimePath: null, + modelId: "test/native-model", + expectedDimensions: DIM, + idleUnloadMs: 60_000, + }); }); afterEach(async () => { @@ -85,6 +98,30 @@ describe("native-embedding facade (worker-backed)", () => { worker.emit({ type: "embed_result", id: req?.type === "embed" ? req.id : -1, vector: vec() }); const result = await p; expect(result).toHaveLength(DIM); + expect(init?.modelId).toBe("test/native-model"); + expect(init?.expectedDimensions).toBe(DIM); + }); + + it("unloads the worker after the configured idle interval", async () => { + configureNativeEmbeddingAssets({ + embeddingWorkerPath: null, + wasmDir: null, + transformersRuntimePath: null, + idleUnloadMs: 10, + }); + const pending = nativeEmbed("idle"); + await flush(); + worker.emit({ type: "ready" }); + await flush(); + const request = worker.posted.find((message) => message.type === "embed"); + worker.emit({ + type: "embed_result", + id: request?.type === "embed" ? request.id : -1, + vector: vec(), + }); + await pending; + await Bun.sleep(30); + expect(worker.posted.some((message) => message.type === "shutdown")).toBe(true); }); it("checkNativeProvider resolves with available:false on init failure (does not reject)", async () => { @@ -176,6 +213,7 @@ describe("native-embedding facade (worker-backed)", () => { type: "check_result", id: checkReq?.type === "checkAvailable" ? checkReq.id : -1, available: true, + error: null, }); await checkP; await flush(); diff --git a/platform/daemon/src/native-embedding.ts b/platform/daemon/src/native-embedding.ts index e7bf97115..734624cf5 100644 --- a/platform/daemon/src/native-embedding.ts +++ b/platform/daemon/src/native-embedding.ts @@ -16,6 +16,11 @@ * paths never await the worker. */ +import { + DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + DEFAULT_NATIVE_EMBEDDING_IDLE_UNLOAD_MS, + DEFAULT_NATIVE_EMBEDDING_MODEL, +} from "@signet/core"; import { type EmbeddingProviderSnapshot, type EmbeddingProviderStatus, @@ -34,6 +39,7 @@ export type NativeProviderSnapshot = EmbeddingProviderSnapshot; let handlePromise: Promise | null = null; let resolvedHandle: EmbeddingWorkerHandle | null = null; let workerFactoryOverride: EmbeddingWorkerFactory | null = null; +let idleTimer: ReturnType | null = null; /** * Pre-resolved native asset paths, set by configureNativeEmbeddingAssets() @@ -48,6 +54,12 @@ let assetPathsOverride: { readonly wasmAssetDir: string | null; readonly transformersRuntimeAssetPath: string | null; } | null = null; +let providerConfig = { + modelId: DEFAULT_NATIVE_EMBEDDING_MODEL, + expectedDimensions: DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + idleUnloadMs: DEFAULT_NATIVE_EMBEDDING_IDLE_UNLOAD_MS, + isolateProcess: false, +}; /** * Configure pre-resolved native asset paths for the embedding worker. Called @@ -62,8 +74,18 @@ export function configureNativeEmbeddingAssets(paths: { readonly embeddingWorkerPath: string | null; readonly wasmAssetDir: string | null; readonly transformersRuntimeAssetPath: string | null; + readonly modelId?: string; + readonly expectedDimensions?: number; + readonly idleUnloadMs?: number; + readonly isolateProcess?: boolean; }): void { assetPathsOverride = paths; + providerConfig = { + modelId: paths.modelId ?? providerConfig.modelId, + expectedDimensions: paths.expectedDimensions ?? providerConfig.expectedDimensions, + idleUnloadMs: paths.idleUnloadMs ?? providerConfig.idleUnloadMs, + isolateProcess: paths.isolateProcess ?? providerConfig.isolateProcess, + }; } // Tracks the most recent init/warm-up attempt. When the daemon's startup @@ -75,19 +97,28 @@ export function configureNativeEmbeddingAssets(paths: { let initPromise: Promise | null = null; function getHandle(): Promise { + if (idleTimer) { + clearTimeout(idleTimer); + idleTimer = null; + } if (!handlePromise) { // SIGNET_EMBEDDING_REMOTE_HOST: test/debug seam that redirects the // transformers model fetch (env.remoteHost). The event-loop isolation // test points it at a local blackhole so first-run download "hangs" // hermetically, without real network. const remoteHostOverride = process.env.SIGNET_EMBEDDING_REMOTE_HOST?.trim() || undefined; - handlePromise = createEmbeddingWorkerHandle( - workerFactoryOverride - ? { workerFactory: workerFactoryOverride } - : remoteHostOverride - ? { remoteHostOverride, ...(assetPathsOverride ?? {}) } - : { ...(assetPathsOverride ?? {}) }, - ).then((h) => { + handlePromise = createEmbeddingWorkerHandle({ + ...providerConfig, + ...(workerFactoryOverride ? { workerFactory: workerFactoryOverride } : {}), + ...(remoteHostOverride ? { remoteHostOverride } : {}), + ...(assetPathsOverride + ? { + embeddingWorkerPath: assetPathsOverride.embeddingWorkerPath, + wasmAssetDir: assetPathsOverride.wasmAssetDir, + transformersRuntimeAssetPath: assetPathsOverride.transformersRuntimeAssetPath, + } + : {}), + }).then((h) => { resolvedHandle = h; return h; }); @@ -95,6 +126,16 @@ function getHandle(): Promise { return handlePromise; } +function scheduleIdleShutdown(): void { + if (providerConfig.idleUnloadMs <= 0) return; + if (idleTimer) clearTimeout(idleTimer); + idleTimer = setTimeout(() => { + idleTimer = null; + void shutdownNativeProvider(); + }, providerConfig.idleUnloadMs); + idleTimer.unref?.(); +} + // --------------------------------------------------------------------------- // Public API (unchanged signatures) // --------------------------------------------------------------------------- @@ -110,7 +151,11 @@ export async function nativeEmbed(text: string): Promise { await initPromise.catch(() => {}); initPromise = null; } - return handle.embed(text); + try { + return await handle.embed(text); + } finally { + scheduleIdleShutdown(); + } } export async function checkNativeProvider(): Promise { @@ -121,7 +166,11 @@ export async function checkNativeProvider(): Promise { p.finally(() => { if (initPromise === p) initPromise = null; }); - return p; + try { + return await p; + } finally { + scheduleIdleShutdown(); + } } export function getNativeProviderStatus(): NativeProviderSnapshot { @@ -132,6 +181,10 @@ export function getNativeProviderStatus(): NativeProviderSnapshot { } export async function shutdownNativeProvider(): Promise { + if (idleTimer) { + clearTimeout(idleTimer); + idleTimer = null; + } const pending = handlePromise; handlePromise = null; initPromise = null; @@ -165,4 +218,10 @@ export async function __resetEmbeddingProviderForTests(): Promise { await shutdownNativeProvider(); workerFactoryOverride = null; assetPathsOverride = null; + providerConfig = { + modelId: DEFAULT_NATIVE_EMBEDDING_MODEL, + expectedDimensions: DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + idleUnloadMs: DEFAULT_NATIVE_EMBEDDING_IDLE_UNLOAD_MS, + isolateProcess: false, + }; } diff --git a/platform/daemon/src/pipeline/skill-reconciler.ts b/platform/daemon/src/pipeline/skill-reconciler.ts index 4b5b24f8a..93079278b 100644 --- a/platform/daemon/src/pipeline/skill-reconciler.ts +++ b/platform/daemon/src/pipeline/skill-reconciler.ts @@ -12,7 +12,6 @@ import { existsSync, readFileSync, readdirSync } from "node:fs"; import { basename, dirname, join } from "node:path"; -import { watch } from "chokidar"; import type { DbAccessor } from "../db-accessor.js"; import { logger } from "../logger.js"; import type { EmbeddingConfig, PipelineV2Config } from "../memory-config.js"; @@ -31,6 +30,7 @@ export interface ReconcilerDeps { readonly fetchEmbedding: (text: string, cfg: EmbeddingConfig) => Promise; readonly getProvider: () => LlmProvider | null; readonly agentsDir: string; + readonly watchFiles?: boolean; } export interface ReconcilerHandle { @@ -231,41 +231,50 @@ export function startReconciler(deps: ReconcilerDeps): ReconcilerHandle { }, intervalMs); // File watcher for low-latency reconciliation - let watcher: ReturnType | null = null; - - if (existsSync(dir)) { - watcher = watch(join(dir, "*", "SKILL.md"), { - ignoreInitial: true, - awaitWriteFinish: { stabilityThreshold: 500 }, - }); - - watcher.on("add", (filePath) => { - const skillName = basename(dirname(filePath)); - logger.info("reconciler", "SKILL.md added", { skill: skillName }); - reconcileSkill(skillName, filePath, deps); - }); - - watcher.on("change", (filePath) => { - const skillName = basename(dirname(filePath)); - logger.info("reconciler", "SKILL.md changed", { skill: skillName }); - reconcileSkill(skillName, filePath, deps); - }); - - watcher.on("unlink", (filePath) => { - const skillName = basename(dirname(filePath)); - logger.info("reconciler", "SKILL.md removed", { skill: skillName }); - uninstallSkillNode({ skillName }, deps.accessor); - }); + let watcher: { close(): Promise | void } | null = null; + let stopped = false; + + if (deps.watchFiles !== false && existsSync(dir)) { + void import("chokidar") + .then(({ watch }) => { + if (stopped) return; + const active = watch(join(dir, "*", "SKILL.md"), { + ignoreInitial: true, + awaitWriteFinish: { stabilityThreshold: 500 }, + }); + active.on("add", (filePath) => { + const skillName = basename(dirname(filePath)); + logger.info("reconciler", "SKILL.md added", { skill: skillName }); + reconcileSkill(skillName, filePath, deps); + }); + active.on("change", (filePath) => { + const skillName = basename(dirname(filePath)); + logger.info("reconciler", "SKILL.md changed", { skill: skillName }); + reconcileSkill(skillName, filePath, deps); + }); + active.on("unlink", (filePath) => { + const skillName = basename(dirname(filePath)); + logger.info("reconciler", "SKILL.md removed", { skill: skillName }); + uninstallSkillNode({ skillName }, deps.accessor); + }); + watcher = active; + }) + .catch((error) => { + logger.error("reconciler", "Failed to start skill watcher", error instanceof Error ? error : undefined, { + error: String(error), + }); + }); } logger.info("reconciler", "Skill reconciler started", { intervalMs, skillsDir: dir, - watcherActive: watcher !== null, + watcherRequested: deps.watchFiles !== false && existsSync(dir), }); return { stop() { + stopped = true; clearInterval(timer); if (watcher) { watcher.close(); diff --git a/platform/daemon/src/pipeline/tokenizer.ts b/platform/daemon/src/pipeline/tokenizer.ts index 72a47155b..22e04f44c 100644 --- a/platform/daemon/src/pipeline/tokenizer.ts +++ b/platform/daemon/src/pipeline/tokenizer.ts @@ -5,19 +5,31 @@ * approximation for all major hosted LLM APIs and far more accurate than * the length/4 character heuristic. * - * The Tiktoken instance is initialised once at module load and shared across - * all callers (dreaming, summary-worker, memory-head) so we only pay the - * vocabulary-load cost once per daemon process. + * The Tiktoken instance is initialised on first use and shared across all + * callers (dreaming, summary-worker, memory-head). The cl100k vocabulary is + * several megabytes on disk and substantially larger once parsed, so loading + * it during module evaluation would dominate an otherwise idle edge daemon. */ +import { createRequire } from "node:module"; import { Tiktoken } from "js-tiktoken/lite"; -import cl100k_base from "js-tiktoken/ranks/cl100k_base"; -const tok = new Tiktoken(cl100k_base); +const require = createRequire(import.meta.url); +let tokenizer: Tiktoken | null = null; + +function getTokenizer(): Tiktoken { + if (tokenizer) return tokenizer; + const rankModule = require("js-tiktoken/ranks/cl100k_base") as { + readonly default?: ConstructorParameters[0]; + }; + const ranks = rankModule.default ?? (rankModule as unknown as ConstructorParameters[0]); + tokenizer = new Tiktoken(ranks); + return tokenizer; +} /** Count the BPE tokens in `text`. */ export function countTokens(text: string): number { - return tok.encode(text).length; + return getTokenizer().encode(text).length; } /** @@ -27,6 +39,7 @@ export function countTokens(text: string): number { */ export function truncateToTokens(text: string, limit: number): string { if (limit < 1) return ""; + const tok = getTokenizer(); const tokens = tok.encode(text); if (tokens.length <= limit) return text; return tok.decode(tokens.slice(0, limit)).trimEnd(); diff --git a/platform/daemon/src/polling-file-watcher.test.ts b/platform/daemon/src/polling-file-watcher.test.ts new file mode 100644 index 000000000..ee17667d9 --- /dev/null +++ b/platform/daemon/src/polling-file-watcher.test.ts @@ -0,0 +1,42 @@ +import { afterEach, describe, expect, it } from "bun:test"; +import { mkdtempSync, rmSync, unlinkSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { type PollingFileWatcher, startPollingFileWatcher } from "./polling-file-watcher"; + +const dirs: string[] = []; +const watchers: PollingFileWatcher[] = []; + +afterEach(() => { + for (const watcher of watchers.splice(0)) watcher.close(); + for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); +}); + +describe("polling file watcher", () => { + it("reports bounded add, change, and unlink events without recursive watches", async () => { + const dir = mkdtempSync(join(tmpdir(), "signet-poll-watch-")); + dirs.push(dir); + const path = join(dir, "agent.yaml"); + const events: string[] = []; + const watcher = await startPollingFileWatcher({ + paths: [path], + intervalMs: 10, + onError(error) { + throw error; + }, + onEvent(event) { + events.push(event); + }, + }); + watchers.push(watcher); + + writeFileSync(path, "version: 1\n"); + await Bun.sleep(30); + writeFileSync(path, "version: 22\n"); + await Bun.sleep(30); + unlinkSync(path); + await Bun.sleep(30); + + expect(events).toEqual(["add", "change", "unlink"]); + }); +}); diff --git a/platform/daemon/src/polling-file-watcher.ts b/platform/daemon/src/polling-file-watcher.ts new file mode 100644 index 000000000..7e055c76c --- /dev/null +++ b/platform/daemon/src/polling-file-watcher.ts @@ -0,0 +1,68 @@ +import { stat } from "node:fs/promises"; + +export type FileWatchEvent = "add" | "change" | "unlink"; + +interface FileFingerprint { + readonly mtimeMs: number; + readonly size: number; +} + +export interface PollingFileWatcher { + close(): void; +} + +async function fingerprint(path: string): Promise { + try { + const value = await stat(path); + return value.isFile() ? { mtimeMs: value.mtimeMs, size: value.size } : null; + } catch (error) { + if ( + typeof error === "object" && + error !== null && + "code" in error && + (error as { readonly code?: unknown }).code === "ENOENT" + ) { + return null; + } + throw error; + } +} + +export async function startPollingFileWatcher(options: { + readonly paths: readonly string[]; + readonly intervalMs: number; + readonly onError: (error: unknown) => void; + readonly onEvent: (event: FileWatchEvent, path: string) => void; +}): Promise { + const state = new Map(); + for (const path of options.paths) state.set(path, await fingerprint(path)); + + let scanning = false; + const timer = setInterval(() => { + if (scanning) return; + scanning = true; + void (async () => { + for (const path of options.paths) { + const before = state.get(path) ?? null; + const after = await fingerprint(path); + state.set(path, after); + if (!before && after) options.onEvent("add", path); + else if (before && !after) options.onEvent("unlink", path); + else if (before && after && (before.mtimeMs !== after.mtimeMs || before.size !== after.size)) { + options.onEvent("change", path); + } + } + })() + .catch(options.onError) + .finally(() => { + scanning = false; + }); + }, options.intervalMs); + timer.unref?.(); + + return { + close() { + clearInterval(timer); + }, + }; +} diff --git a/platform/daemon/src/routes/inference.ts b/platform/daemon/src/routes/inference.ts index ef83da92f..7f27963fe 100644 --- a/platform/daemon/src/routes/inference.ts +++ b/platform/daemon/src/routes/inference.ts @@ -1,4 +1,3 @@ -import { getModels, getProviders } from "@earendil-works/pi-ai"; import { ROUTING_COST_TIERS, ROUTING_OPERATION_KINDS, @@ -830,6 +829,7 @@ export function mountInferenceRoutes(app: Hono, opts: InferenceRouteOptions = {} // encrypted credential presence; it never returns credential values or makes // provider network calls. app.get("/api/inference/catalog", async (c) => { + const { getModels, getProviders } = await import("@earendil-works/pi-ai"); const providers = getProviders(); const models: Record< string, @@ -855,7 +855,7 @@ export function mountInferenceRoutes(app: Hono, opts: InferenceRouteOptions = {} modelErrors[provider] = error instanceof Error ? error.message : "model catalog resolution failed"; } } - const oauthProviders = await Promise.all(listOAuthProviderMetadata().map(oauthProviderStatus)); + const oauthProviders = await Promise.all((await listOAuthProviderMetadata()).map(oauthProviderStatus)); return c.json({ providers, models, @@ -868,13 +868,15 @@ export function mountInferenceRoutes(app: Hono, opts: InferenceRouteOptions = {} }); app.get("/api/inference/oauth/providers", async (c) => { - const providers = await Promise.all(listOAuthProviderMetadata().map(oauthProviderStatus)); + const providers = await Promise.all((await listOAuthProviderMetadata()).map(oauthProviderStatus)); return c.json({ providers }); }); - app.post("/api/inference/oauth/login/:id", (c) => { + app.post("/api/inference/oauth/login/:id", async (c) => { try { - const login = startOAuthLogin(c.req.param("id"), () => getInferenceRouterOrNull()?.invalidateCredentialState()); + const login = await startOAuthLogin(c.req.param("id"), () => + getInferenceRouterOrNull()?.invalidateCredentialState(), + ); return new Response(login.stream, { headers: { "Content-Type": "text/event-stream", diff --git a/platform/daemon/src/routes/memory-routes.ts b/platform/daemon/src/routes/memory-routes.ts index af6b1773a..d3e18e2f4 100644 --- a/platform/daemon/src/routes/memory-routes.ts +++ b/platform/daemon/src/routes/memory-routes.ts @@ -26,7 +26,6 @@ import { parseFeedback, recordAgentFeedback } from "../session-memories"; import { upsertSessionTranscript } from "../session-transcripts"; import { createTemporalEdgeId, normalizeTemporalTimestamp, validateTemporalTimeOptions } from "../temporal-recall"; import { txForgetMemory, txIngestEnvelope, txModifyMemory, txRecoverMemory, txSupersedeMemory } from "../transactions"; -import { cacheProjection, computeProjection, computeProjectionForQuery, getCachedProjection } from "../umap-projection"; import { AGENTS_DIR, INTERNAL_SELF_HOST, @@ -3261,6 +3260,9 @@ export function registerMemoryRoutes(app: Hono, deps: MemoryRoutesDeps = {}): vo // GET /api/embeddings/projection // ========================================================================= app.get("/api/embeddings/projection", async (c) => { + const { cacheProjection, computeProjection, computeProjectionForQuery, getCachedProjection } = await import( + "../umap-projection" + ); const dimParam = c.req.query("dimensions"); const nComponents: 2 | 3 = dimParam === "3" ? 3 : 2; const limit = parseOptionalBoundedInt(c.req.query("limit"), 1, 5000); diff --git a/platform/daemon/src/routes/repair-routes.ts b/platform/daemon/src/routes/repair-routes.ts index 3603b472b..0bcd584c5 100644 --- a/platform/daemon/src/routes/repair-routes.ts +++ b/platform/daemon/src/routes/repair-routes.ts @@ -6,7 +6,6 @@ import { fetchEmbedding } from "../embedding-fetch.js"; import { linkMemoryToEntities, previewMemoryEntityLinks } from "../inline-entity-linker.js"; import { getLlmProvider } from "../llm.js"; import { loadMemoryConfig } from "../memory-config.js"; -import { clusterEntities } from "../pipeline/community-detection.js"; import { DEFAULT_RETENTION, runRetentionSweepOnce } from "../pipeline/retention-worker.js"; import { type RepairContext, @@ -407,7 +406,8 @@ export function registerRepairRoutes( ); }); - app.post("/api/repair/cluster-entities", (c) => { + app.post("/api/repair/cluster-entities", async (c) => { + const { clusterEntities } = await import("../pipeline/community-detection.js"); const agentId = resolveRepairAgentId(c); const result = getDbAccessor().withWriteTx((db) => clusterEntities(db, agentId)); return c.json(result); diff --git a/platform/daemon/src/runtime-profile.test.ts b/platform/daemon/src/runtime-profile.test.ts new file mode 100644 index 000000000..e192078fb --- /dev/null +++ b/platform/daemon/src/runtime-profile.test.ts @@ -0,0 +1,46 @@ +import { afterEach, describe, expect, it } from "bun:test"; +import { mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; +import { loadRuntimeProfile } from "./runtime-profile"; + +const dirs: string[] = []; + +afterEach(() => { + for (const dir of dirs.splice(0)) rmSync(dir, { recursive: true, force: true }); +}); + +function workspace(contents = ""): string { + const dir = mkdtempSync(join(tmpdir(), "signet-runtime-profile-")); + dirs.push(dir); + if (contents) writeFileSync(join(dir, "agent.yaml"), contents); + return dir; +} + +describe("runtime profile", () => { + it("uses the standard profile by default", () => { + expect(loadRuntimeProfile(workspace(), {}).profile).toBe("standard"); + }); + + it("edge profile uses polling and keeps native embedding lazy", () => { + const config = loadRuntimeProfile(workspace("runtime:\n profile: edge\n"), {}); + expect(config).toMatchObject({ + embeddingIsolation: "process", + profile: "edge", + probeEmbeddingAtStartup: false, + watcher: "poll", + }); + expect(config.embeddingIdleUnloadMs).toBe(30_000); + }); + + it("a valid environment override wins over the file", () => { + const config = loadRuntimeProfile(workspace("runtime:\n profile: standard\n"), { + SIGNET_RUNTIME_PROFILE: " edge ", + }); + expect(config.profile).toBe("edge"); + }); + + it("invalid values fail safely to standard", () => { + expect(loadRuntimeProfile(workspace("runtime:\n profile: enormous\n"), {}).profile).toBe("standard"); + }); +}); diff --git a/platform/daemon/src/runtime-profile.ts b/platform/daemon/src/runtime-profile.ts new file mode 100644 index 000000000..bee67b43d --- /dev/null +++ b/platform/daemon/src/runtime-profile.ts @@ -0,0 +1,79 @@ +import { existsSync, readFileSync } from "node:fs"; +import { join } from "node:path"; +import { + DEFAULT_NATIVE_EMBEDDING_IDLE_UNLOAD_MS, + EDGE_NATIVE_EMBEDDING_IDLE_UNLOAD_MS, + parseSimpleYaml, +} from "@signet/core"; +import { logger } from "./logger"; + +export type RuntimeProfile = "standard" | "edge"; + +export interface RuntimeProfileConfig { + readonly profile: RuntimeProfile; + readonly embeddingIdleUnloadMs: number; + readonly embeddingIsolation: "thread" | "process"; + readonly probeEmbeddingAtStartup: boolean; + readonly watcher: "chokidar" | "poll"; +} + +const PROFILES: Readonly> = { + standard: { + profile: "standard", + embeddingIdleUnloadMs: DEFAULT_NATIVE_EMBEDDING_IDLE_UNLOAD_MS, + embeddingIsolation: "thread", + probeEmbeddingAtStartup: true, + watcher: "chokidar", + }, + edge: { + profile: "edge", + embeddingIdleUnloadMs: EDGE_NATIVE_EMBEDDING_IDLE_UNLOAD_MS, + embeddingIsolation: "process", + probeEmbeddingAtStartup: false, + watcher: "poll", + }, +}; + +function parseProfile(value: unknown): RuntimeProfile | null { + if (typeof value !== "string") return null; + const normalized = value.trim().toLowerCase(); + return normalized === "standard" || normalized === "edge" ? normalized : null; +} + +export function loadRuntimeProfile(agentsDir: string, env: NodeJS.ProcessEnv = process.env): RuntimeProfileConfig { + const envValue = env.SIGNET_RUNTIME_PROFILE; + if (envValue !== undefined) { + const profile = parseProfile(envValue); + if (profile) return PROFILES[profile]; + logger.warn("config", "Ignoring invalid SIGNET_RUNTIME_PROFILE", { + value: envValue, + allowed: ["standard", "edge"], + }); + } + + for (const name of ["agent.yaml", "AGENT.yaml", "config.yaml"]) { + const path = join(agentsDir, name); + if (!existsSync(path)) continue; + try { + const yaml = parseSimpleYaml(readFileSync(path, "utf8")); + const runtime = yaml.runtime as Record | undefined; + const raw = runtime?.profile; + if (raw === undefined) continue; + const profile = parseProfile(raw); + if (profile) return PROFILES[profile]; + logger.warn("config", "Ignoring invalid runtime.profile", { + file: path, + value: raw, + allowed: ["standard", "edge"], + }); + return PROFILES.standard; + } catch (error) { + logger.warn("config", "Unable to read runtime profile", { + file: path, + error: error instanceof Error ? error.message : String(error), + }); + } + } + + return PROFILES.standard; +} diff --git a/scripts/check-edge-runtime.ts b/scripts/check-edge-runtime.ts new file mode 100644 index 000000000..f427bb0ac --- /dev/null +++ b/scripts/check-edge-runtime.ts @@ -0,0 +1,82 @@ +#!/usr/bin/env bun + +import { existsSync, readFileSync } from "node:fs"; +import { homedir } from "node:os"; +import { join, resolve } from "node:path"; + +const RSS_LIMIT_KIB = 100 * 1024; +const EMBEDDING_LIMIT_MS = 5_000; +const EDGE_EMBEDDING_MODEL = "Xenova/all-MiniLM-L6-v2"; + +function argument(name: string): string | null { + const index = process.argv.indexOf(name); + return index >= 0 ? (process.argv[index + 1] ?? null) : null; +} + +function readLinuxRssKib(pid: number): number { + const statusPath = `/proc/${pid}/status`; + if (!existsSync(statusPath)) throw new Error(`Linux process status not found: ${statusPath}`); + const match = /^VmRSS:\s+(\d+)\s+kB$/m.exec(readFileSync(statusPath, "utf8")); + if (!match) throw new Error(`VmRSS missing from ${statusPath}`); + return Number(match[1]); +} + +const workspace = resolve(argument("--workspace") ?? process.env.SIGNET_PATH ?? join(homedir(), ".agents")); +const pidPath = join(workspace, ".daemon", "pid"); +const pid = Number(readFileSync(pidPath, "utf8").trim()); +if (!Number.isSafeInteger(pid) || pid <= 0) throw new Error(`Invalid daemon PID in ${pidPath}`); + +const idleRssKib = readLinuxRssKib(pid); +const result: Record = { + architecture: process.arch, + idleRssKib, + idleRssPass: idleRssKib < RSS_LIMIT_KIB, + pid, + platform: process.platform, + rssLimitKib: RSS_LIMIT_KIB, + workspace, +}; + +if (process.argv.includes("--with-embedding")) { + const port = Number(argument("--port") ?? process.env.SIGNET_PORT ?? "3850"); + const statusResponse = await fetch(`http://127.0.0.1:${port}/api/embeddings/status`, { + signal: AbortSignal.timeout(90_000), + }); + const status = (await statusResponse.json()) as Record; + if (!statusResponse.ok || status.available !== true) { + throw new Error(`Native embedding warm-up failed: ${JSON.stringify(status)}`); + } + + const startedAt = performance.now(); + const response = await fetch(`http://127.0.0.1:${port}/api/memory/recall`, { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ query: "edge runtime latency probe", limit: 1 }), + signal: AbortSignal.timeout(90_000), + }); + const requestElapsedMs = Math.round(performance.now() - startedAt); + const body = (await response.json()) as Record; + const meta = body.meta as Record | undefined; + const timings = meta?.timings as Record | undefined; + const stages = Array.isArray(timings?.stages) ? timings.stages : []; + const embeddingStage = stages.find( + (stage): stage is Record => + typeof stage === "object" && stage !== null && (stage as Record).name === "query_embedding_wait", + ); + const elapsedMs = typeof embeddingStage?.durationMs === "number" ? embeddingStage.durationMs : null; + const modelPass = status.model === EDGE_EMBEDDING_MODEL; + const latencyPass = elapsedMs !== null && elapsedMs < EMBEDDING_LIMIT_MS; + result.embedding = { + available: status.available === true, + elapsedMs, + latencyPass, + limitMs: EMBEDDING_LIMIT_MS, + model: status.model, + modelPass, + requestElapsedMs, + }; + if (!response.ok || !modelPass || !latencyPass) process.exitCode = 1; +} + +console.log(JSON.stringify(result, null, 2)); +if (idleRssKib >= RSS_LIMIT_KIB) process.exitCode = 1; diff --git a/surfaces/cli/src/cli.ts b/surfaces/cli/src/cli.ts index 6bf205c21..4b78b4cb7 100644 --- a/surfaces/cli/src/cli.ts +++ b/surfaces/cli/src/cli.ts @@ -31,6 +31,7 @@ import { OpenClawConnector } from "@signet/connector-openclaw"; import { OpenCodeConnector } from "@signet/connector-opencode"; import { PiConnector } from "@signet/connector-pi"; import { + DEFAULT_NATIVE_EMBEDDING_MODEL, IDENTITY_FILES, type ImportResult, type MigrationResult, @@ -614,6 +615,8 @@ async function syncNativeEmbeddingModel(basePath: string): Promise<{ }; } else { const active = typeof body.provider === "string" ? body.provider : "unknown"; + const activeModel = + typeof body.model === "string" && body.model.trim() ? body.model : DEFAULT_NATIVE_EMBEDDING_MODEL; const available = body.available === true; const err = typeof body.error === "string" ? body.error : null; const reported = body.modelCached === true; @@ -643,9 +646,7 @@ async function syncNativeEmbeddingModel(basePath: string): Promise<{ } else { result = { status: !hadCache && hasCache ? "updated" : "current", - message: hasCache - ? "nomic-ai/nomic-embed-text-v1.5" - : "nomic-ai/nomic-embed-text-v1.5 (runtime cache)", + message: hasCache ? activeModel : `${activeModel} (runtime cache)`, }; } } diff --git a/surfaces/cli/src/features/configure.ts b/surfaces/cli/src/features/configure.ts index c30d158e9..853415f4e 100644 --- a/surfaces/cli/src/features/configure.ts +++ b/surfaces/cli/src/features/configure.ts @@ -1,7 +1,12 @@ import { existsSync, readFileSync, writeFileSync } from "node:fs"; import { join } from "node:path"; import { checkbox, confirm, input, select } from "@inquirer/prompts"; -import { parseSimpleYaml, readNetworkMode } from "@signet/core"; +import { + DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + DEFAULT_NATIVE_EMBEDDING_MODEL, + parseSimpleYaml, + readNetworkMode, +} from "@signet/core"; import chalk from "chalk"; import { chooseWorkspaceCandidate, @@ -303,8 +308,9 @@ async function configureEmbedding(yaml: string): Promise { let dims = 768; if (provider === "native") { - model = "nomic-embed-text-v1.5"; - console.log(" Embedding model: nomic-embed-text-v1.5 (768d)"); + model = DEFAULT_NATIVE_EMBEDDING_MODEL; + dims = DEFAULT_NATIVE_EMBEDDING_DIMENSIONS; + console.log(` Embedding model: ${DEFAULT_NATIVE_EMBEDDING_MODEL} (${DEFAULT_NATIVE_EMBEDDING_DIMENSIONS}d)`); } else if (provider === "llama-cpp") { const selected = await select({ message: "Model:", diff --git a/surfaces/cli/src/features/setup-providers.ts b/surfaces/cli/src/features/setup-providers.ts index 2a53b509c..befe8741d 100644 --- a/surfaces/cli/src/features/setup-providers.ts +++ b/surfaces/cli/src/features/setup-providers.ts @@ -1,6 +1,7 @@ import { spawn, spawnSync } from "node:child_process"; import { platform } from "node:os"; import { confirm, select } from "@inquirer/prompts"; +import { DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, DEFAULT_NATIVE_EMBEDDING_MODEL } from "@signet/core"; import chalk from "chalk"; import ora from "ora"; import { getEmbeddingDimensions, readErr } from "./setup-shared.js"; @@ -78,7 +79,11 @@ export async function preflightOllamaEmbedding(model: string): Promise<{ const fallback = await promptOllamaFailureFallback(); if (fallback === "retry") continue; if (fallback === "native") { - return { provider: "native", model: "nomic-embed-text-v1.5", dimensions: 768 }; + return { + provider: "native", + model: DEFAULT_NATIVE_EMBEDDING_MODEL, + dimensions: DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + }; } if (fallback === "openai") { return promptOpenAIEmbeddingModel(); @@ -96,7 +101,11 @@ export async function preflightOllamaEmbedding(model: string): Promise<{ const fallback = await promptOllamaFailureFallback(); if (fallback === "retry") continue; if (fallback === "native") { - return { provider: "native", model: "nomic-embed-text-v1.5", dimensions: 768 }; + return { + provider: "native", + model: DEFAULT_NATIVE_EMBEDDING_MODEL, + dimensions: DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + }; } if (fallback === "openai") { return promptOpenAIEmbeddingModel(); @@ -121,7 +130,11 @@ export async function preflightOllamaEmbedding(model: string): Promise<{ const fallback = await promptOllamaFailureFallback(); if (fallback === "retry") continue; if (fallback === "native") { - return { provider: "native", model: "nomic-embed-text-v1.5", dimensions: 768 }; + return { + provider: "native", + model: DEFAULT_NATIVE_EMBEDDING_MODEL, + dimensions: DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + }; } if (fallback === "openai") { return promptOpenAIEmbeddingModel(); diff --git a/surfaces/cli/src/features/setup-shared.ts b/surfaces/cli/src/features/setup-shared.ts index 1c27c7685..6be67d7ff 100644 --- a/surfaces/cli/src/features/setup-shared.ts +++ b/surfaces/cli/src/features/setup-shared.ts @@ -1,5 +1,10 @@ import { OpenClawConnector } from "@signet/connector-openclaw"; -import type { SetupDetection, WorkspaceSourceRepoSyncResult } from "@signet/core"; +import { + DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + DEFAULT_NATIVE_EMBEDDING_MODEL, + type SetupDetection, + type WorkspaceSourceRepoSyncResult, +} from "@signet/core"; import chalk from "chalk"; export type HarnessChoice = @@ -46,7 +51,13 @@ export const SETUP_HARNESS_CHOICES: readonly HarnessChoice[] = [ "hermes-agent", "gemini", ]; -export const EMBEDDING_PROVIDER_CHOICES: readonly EmbeddingProviderChoice[] = ["native", "llama-cpp", "ollama", "openai", "none"]; +export const EMBEDDING_PROVIDER_CHOICES: readonly EmbeddingProviderChoice[] = [ + "native", + "llama-cpp", + "ollama", + "openai", + "none", +]; export const EXTRACTION_PROVIDER_CHOICES: readonly ExtractionProviderChoice[] = [ "acpx", "claude-code", @@ -60,7 +71,12 @@ export const EXTRACTION_PROVIDER_CHOICES: readonly ExtractionProviderChoice[] = ]; export const OPENCLAW_RUNTIME_CHOICES: readonly OpenClawRuntimeChoice[] = ["plugin", "legacy"]; export const DEPLOYMENT_TYPE_CHOICES: readonly DeploymentTypeChoice[] = ["local", "vps", "server"]; -const VPS_NON_LOCAL_EXTRACTION_PROVIDERS: readonly ExtractionProviderChoice[] = ["acpx", "claude-code", "codex", "opencode"]; +const VPS_NON_LOCAL_EXTRACTION_PROVIDERS: readonly ExtractionProviderChoice[] = [ + "acpx", + "claude-code", + "codex", + "opencode", +]; const DETECTED_EXTRACTION_PROVIDER_ORDER: readonly ExtractionProviderChoice[] = [ "acpx", "llama-cpp", @@ -192,8 +208,9 @@ export function failNonInteractiveSetup(message: string): never { export function getEmbeddingDimensions(model: string): number { switch (model) { + case DEFAULT_NATIVE_EMBEDDING_MODEL: case "all-minilm": - return 384; + return DEFAULT_NATIVE_EMBEDDING_DIMENSIONS; case "mxbai-embed-large": return 1024; case "text-embedding-3-large": diff --git a/surfaces/cli/src/features/setup.ts b/surfaces/cli/src/features/setup.ts index fc1e3c1f3..a65550371 100644 --- a/surfaces/cli/src/features/setup.ts +++ b/surfaces/cli/src/features/setup.ts @@ -3,6 +3,8 @@ import { join } from "node:path"; import { checkbox, confirm, input, select } from "@inquirer/prompts"; import { OpenClawConnector } from "@signet/connector-openclaw"; import { + DEFAULT_NATIVE_EMBEDDING_DIMENSIONS, + DEFAULT_NATIVE_EMBEDDING_MODEL, IDENTITY_MODES, IDENTITY_PRESETS, type IdentityContextFileEntry, @@ -836,8 +838,8 @@ export async function setupWizard(options: SetupWizardOptions, deps: SetupDeps): let embeddingDimensions = 768; if (embeddingProvider === "native") { - embeddingModel = "nomic-embed-text-v1.5"; - embeddingDimensions = 768; + embeddingModel = DEFAULT_NATIVE_EMBEDDING_MODEL; + embeddingDimensions = DEFAULT_NATIVE_EMBEDDING_DIMENSIONS; } else if (embeddingProvider === "llama-cpp") { if (nonInteractive) { const configuredModel = @@ -877,8 +879,8 @@ export async function setupWizard(options: SetupWizardOptions, deps: SetupDeps): console.log(chalk.yellow(` ⚠ ${ollamaCheck.error ?? "Ollama embedding model not available"}`)); console.log(chalk.yellow(" Downgrading embedding provider to 'native' (built-in ONNX).")); embeddingProvider = "native"; - embeddingModel = "nomic-embed-text-v1.5"; - embeddingDimensions = 768; + embeddingModel = DEFAULT_NATIVE_EMBEDDING_MODEL; + embeddingDimensions = DEFAULT_NATIVE_EMBEDDING_DIMENSIONS; } } else { console.log(); diff --git a/surfaces/cli/templates/agent.yaml.template b/surfaces/cli/templates/agent.yaml.template index 41357b350..498231b6e 100644 --- a/surfaces/cli/templates/agent.yaml.template +++ b/surfaces/cli/templates/agent.yaml.template @@ -31,21 +31,27 @@ harnesses: - opencode - openclaw +# Runtime profile: "standard" or "edge". +# Edge uses fixed-file polling and lazy native embeddings on constrained ARM. +runtime: + profile: standard + # ============================================================================ # Embedding Configuration # ============================================================================ embedding: - # Provider: "ollama" (local) or "openai" - provider: ollama + # Provider: "native", "llama-cpp", "ollama", "openai", or "none" + provider: native # Model name + # Native: Xenova/all-MiniLM-L6-v2 # Ollama: nomic-embed-text, all-minilm, mxbai-embed-large # OpenAI: text-embedding-3-small, text-embedding-3-large - model: nomic-embed-text + model: Xenova/all-MiniLM-L6-v2 # Vector dimensions (must match model output) - dimensions: 768 + dimensions: 384 # API endpoint (optional, defaults to standard URLs) # base_url: http://localhost:11434