Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions .env.example
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,21 @@

# EMBEDDING_PROVIDER=local # local | openai | voyage | cohere | gemini | openrouter

# LOCAL_EMBEDDING_MODEL=Xenova/all-MiniLM-L6-v2
# LOCAL_EMBEDDING_DIMENSIONS=384 # Set explicitly for a different model width
# LOCAL_EMBEDDING_POOLING=mean # mean | cls | first_token | last_token | eos
# LOCAL_EMBEDDING_DTYPE=q8 # Transformers.js dtype, e.g. q8 or fp32
# Optional exact prefixes. For E5-style inputs, quote env-file values to retain spaces:
# LOCAL_EMBEDDING_QUERY_PREFIX="query: "
# LOCAL_EMBEDDING_DOCUMENT_PREFIX="passage: "
# LOCAL_EMBEDDING_CACHE_DIR= # Optional pipeline cache directory
# LOCAL_EMBEDDING_BATCH_SIZE= # Optional positive batch limit
# Model, dimensions, pooling, dtype, document-prefix or cache-path changes require new vectors.
# Unidentified older local indexes require an explicit rebuild too: back up the
# store, set AGENTMEMORY_DROP_STALE_INDEX=true for one restart, then unset it.
# This removes only derived vectors. Set AGENTMEMORY_VECTOR_BACKFILL=all
# to run the full backfill; otherwise it waits for explicit opt-in.

# VOYAGE_API_KEY=pa-... # Optimised for code embeddings

# COHERE_API_KEY=... # General-purpose embeddings
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/release-gate.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ jobs:
engine_pin=$(node --import tsx --input-type=module -e 'import { III_PINNED_VERSION } from "./src/version.ts"; console.log(III_PINNED_VERSION)')
engine_bin="$HOME/.cache/agentmemory-release-gate/iii-${engine_pin}-linux-$(node -p process.arch)"
test -x "$engine_bin"
AGENTMEMORY_TEST_III="$engine_bin" AGENTMEMORY_REQUIRE_LIVE_TESTS=1 npx vitest run test/compose-supervision-integration.test.ts test/codex-live-daemon.test.ts test/compose-c1-integration.test.ts
AGENTMEMORY_TEST_III="$engine_bin" AGENTMEMORY_REQUIRE_LIVE_TESTS=1 npx vitest run test/compose-supervision-integration.test.ts test/codex-live-daemon.test.ts test/compose-cache-integration.test.ts test/compose-c1-integration.test.ts
- run: npm run test:startup:packaged -- --skip-build --out "${{ runner.temp }}/startup-gate"
- name: Test packaged vector recovery
shell: bash
Expand Down
13 changes: 13 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1114,6 +1114,19 @@ Reasoning-class models (`o1`-style with `<think>` blocks) can return empty `cont

Local embeddings ship as an optional dependency but are not enabled by default. Set `EMBEDDING_PROVIDER=local` to opt into `Xenova/all-MiniLM-L6-v2` (384-dim). The first embedding request downloads the model; inference is on-device afterward. Without that setting or a remote embedding key, vectors stay disabled, `mem::search` uses BM25, and `smart-search` can still add existing graph matches.

Local Transformers.js models can be selected with `LOCAL_EMBEDDING_MODEL`, `LOCAL_EMBEDDING_DIMENSIONS`, `LOCAL_EMBEDDING_POOLING` and `LOCAL_EMBEDDING_DTYPE` (defaults: the model above, `384`, `mean`, `q8`). Optional `LOCAL_EMBEDDING_QUERY_PREFIX` and `LOCAL_EMBEDDING_DOCUMENT_PREFIX` preserve exact prefixes, including trailing spaces. `LOCAL_EMBEDDING_CACHE_DIR` selects the pipeline cache and `LOCAL_EMBEDDING_BATCH_SIZE` bounds extraction batches. Use settings and a Transformers.js export compatible with the selected model. The native CLI forwards process environment values literally, including `$` and `${...}`; relative cache paths resolve from the CLI directory.

In `~/.agentmemory/.env`, quote prefixes that require trailing spaces. For E5-style inputs, for example:

```dotenv
LOCAL_EMBEDDING_QUERY_PREFIX="query: "
LOCAL_EMBEDDING_DOCUMENT_PREFIX="passage: "
```

Unquoted env-file values are trimmed before the provider reads them.

Startup records the local document-embedding configuration before generating vectors. Changing the model, dimensions, pooling, dtype, document prefix or configured cache directory refuses restoration of both checkpoints and pending vectors, even when dimensions match. Older local indexes without this identity also require an explicit rebuild: back up the store, set `AGENTMEMORY_DROP_STALE_INDEX=true` for one restart, then unset it. This removes derived vectors while retaining memories and observations. Set `AGENTMEMORY_VECTOR_BACKFILL=all` to run the full backfill; otherwise it waits for explicit opt-in. Query-prefix and batch-size changes do not require document vectors to be rebuilt. A cache-directory change conservatively requires rebuilding because it may select different artifacts under the same model ID. This configuration identity does not fingerprint weight files or pin mutable upstream model revisions. For Docker, use a cache path inside the container, such as `/data/models`, and mount or persist that location.

### Cost-aware model selection

Start keyless for capture and keyword recall. Enable `EMBEDDING_PROVIDER=local` for on-device semantic search after the initial model download. LLM-written compression requires a provider and `AGENTMEMORY_AUTO_COMPRESS=true`; it can make a request for each captured observation, so check your provider's current pricing before enabling it.
Expand Down
8 changes: 8 additions & 0 deletions deploy/coolify/docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,14 @@ services:
VIEWER_ALLOWED_HOSTS: "${VIEWER_ALLOWED_HOSTS:-localhost:3113,127.0.0.1:3113,[::1]:3113}"
VIEWER_ALLOWED_ORIGINS: "${VIEWER_ALLOWED_ORIGINS:-http://localhost:3113,http://127.0.0.1:3113}"
EMBEDDING_PROVIDER: "${EMBEDDING_PROVIDER:-}"
LOCAL_EMBEDDING_MODEL: "${LOCAL_EMBEDDING_MODEL:-}"
LOCAL_EMBEDDING_DIMENSIONS: "${LOCAL_EMBEDDING_DIMENSIONS:-}"
LOCAL_EMBEDDING_POOLING: "${LOCAL_EMBEDDING_POOLING:-}"
LOCAL_EMBEDDING_DTYPE: "${LOCAL_EMBEDDING_DTYPE:-}"
LOCAL_EMBEDDING_QUERY_PREFIX: "${LOCAL_EMBEDDING_QUERY_PREFIX:-}"
LOCAL_EMBEDDING_DOCUMENT_PREFIX: "${LOCAL_EMBEDDING_DOCUMENT_PREFIX:-}"
LOCAL_EMBEDDING_CACHE_DIR: "${LOCAL_EMBEDDING_CACHE_DIR:-}"
LOCAL_EMBEDDING_BATCH_SIZE: "${LOCAL_EMBEDDING_BATCH_SIZE:-}"
AGENTMEMORY_LLM_PROVIDER: "${AGENTMEMORY_LLM_PROVIDER:-}"
OPENAI_API_KEY: "${OPENAI_API_KEY:-}"
OPENAI_API_KEY_FOR_LLM: "${OPENAI_API_KEY_FOR_LLM:-}"
Expand Down
8 changes: 8 additions & 0 deletions deploy/runtime/worker-compose.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,14 @@ containers:
VIEWER_ALLOWED_HOSTS: "${VIEWER_ALLOWED_HOSTS:-localhost:3113,127.0.0.1:3113,[::1]:3113}"
VIEWER_ALLOWED_ORIGINS: "${VIEWER_ALLOWED_ORIGINS:-http://localhost:3113,http://127.0.0.1:3113}"
EMBEDDING_PROVIDER: "${EMBEDDING_PROVIDER:-}"
LOCAL_EMBEDDING_MODEL: "${LOCAL_EMBEDDING_MODEL:-}"
LOCAL_EMBEDDING_DIMENSIONS: "${LOCAL_EMBEDDING_DIMENSIONS:-}"
LOCAL_EMBEDDING_POOLING: "${LOCAL_EMBEDDING_POOLING:-}"
LOCAL_EMBEDDING_DTYPE: "${LOCAL_EMBEDDING_DTYPE:-}"
LOCAL_EMBEDDING_QUERY_PREFIX: "${LOCAL_EMBEDDING_QUERY_PREFIX:-}"
LOCAL_EMBEDDING_DOCUMENT_PREFIX: "${LOCAL_EMBEDDING_DOCUMENT_PREFIX:-}"
LOCAL_EMBEDDING_CACHE_DIR: "${LOCAL_EMBEDDING_CACHE_DIR:-}"
LOCAL_EMBEDDING_BATCH_SIZE: "${LOCAL_EMBEDDING_BATCH_SIZE:-}"
AGENTMEMORY_LLM_PROVIDER: "${AGENTMEMORY_LLM_PROVIDER:-}"
OPENAI_API_KEY: "${OPENAI_API_KEY:-}"
OPENAI_API_KEY_FOR_LLM: "${OPENAI_API_KEY_FOR_LLM:-}"
Expand Down
8 changes: 8 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,14 @@ services:
VIEWER_ALLOWED_HOSTS: "${VIEWER_ALLOWED_HOSTS:-localhost:${AGENTMEMORY_VIEWER_PORT:-3113},127.0.0.1:${AGENTMEMORY_VIEWER_PORT:-3113},[::1]:${AGENTMEMORY_VIEWER_PORT:-3113}}"
VIEWER_ALLOWED_ORIGINS: "${VIEWER_ALLOWED_ORIGINS:-http://localhost:${AGENTMEMORY_VIEWER_PORT:-3113},http://127.0.0.1:${AGENTMEMORY_VIEWER_PORT:-3113}}"
EMBEDDING_PROVIDER: "${EMBEDDING_PROVIDER:-}"
LOCAL_EMBEDDING_MODEL: "${LOCAL_EMBEDDING_MODEL:-}"
LOCAL_EMBEDDING_DIMENSIONS: "${LOCAL_EMBEDDING_DIMENSIONS:-}"
LOCAL_EMBEDDING_POOLING: "${LOCAL_EMBEDDING_POOLING:-}"
LOCAL_EMBEDDING_DTYPE: "${LOCAL_EMBEDDING_DTYPE:-}"
LOCAL_EMBEDDING_QUERY_PREFIX: "${LOCAL_EMBEDDING_QUERY_PREFIX:-}"
LOCAL_EMBEDDING_DOCUMENT_PREFIX: "${LOCAL_EMBEDDING_DOCUMENT_PREFIX:-}"
LOCAL_EMBEDDING_CACHE_DIR: "${LOCAL_EMBEDDING_CACHE_DIR:-}"
LOCAL_EMBEDDING_BATCH_SIZE: "${LOCAL_EMBEDDING_BATCH_SIZE:-}"
AGENTMEMORY_LLM_PROVIDER: "${AGENTMEMORY_LLM_PROVIDER:-}"
OPENAI_API_KEY: "${OPENAI_API_KEY:-}"
OPENAI_API_KEY_FOR_LLM: "${OPENAI_API_KEY_FOR_LLM:-}"
Expand Down
2 changes: 1 addition & 1 deletion scripts/docs-sync.state.json
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
"323"
],
"sourceFiles": [
"262"
"263"
],
"loc": [
"62000"
Expand Down
11 changes: 10 additions & 1 deletion scripts/release-gate/startup.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -414,8 +414,17 @@ async function main() {
const binary = join(i.runtime, "bin/iii"); const launchPid = join(i.dir, "launched-compose.pid");
writeFileSync(binary, `#!${process.execPath}\nconst {writeFileSync}=require('node:fs');\nif(process.argv.includes('--version')) { console.log('iii ${installed.dependencies["iii-sdk"]}'); process.exit(0); }\nwriteFileSync(${JSON.stringify(launchPid)},String(process.pid));\nsetInterval(()=>{},1000);\n`); chmodSync(binary, 0o755);
Object.assign(d, await expectFailure(i, [], { AGENTMEMORY_ENGINE_READY_TIMEOUT_MS: "300" }, { message: "The engine process started but the REST API never responded." }));
const pid = Number(readFileSync(launchPid, "utf8")); i.ownedPids.add(pid); await poll("owned Compose engine timeout cleanup", () => !alive(pid), 5_000);
const composePid = Number(i.cli.text.match(/Compose daemon started \(pid (\d+)\)/)?.[1]);
assert(Number.isSafeInteger(composePid) && composePid > 0, `No owned Compose daemon was started: ${i.cli.text}`);
i.ownedPids.add(composePid);
// The 300ms readiness budget can expire before the daemon launches iii on
// a busy runner. That is still a valid timeout; always check the daemon,
// and check its engine child too when the fixture reached its entrypoint.
const pid = existsSync(launchPid) ? Number(readFileSync(launchPid, "utf8")) : null;
if (pid !== null) { assert(Number.isSafeInteger(pid) && pid > 0); i.ownedPids.add(pid); }
await poll("owned Compose engine timeout cleanup", () => !alive(composePid) && (pid === null || !alive(pid)), 5_000);
assert(!existsSync(join(i.runtime, "engine-state.json")));
d.engineEntrypointReached = pid !== null;
d.transport = "version-compatible iii fixture accepting compose but never opening the engine port; actual installed CLI";
});
await scenario("cancel-owned-startup", async (i, d) => {
Expand Down
20 changes: 18 additions & 2 deletions src/cli/compose-project.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { chmodSync, existsSync, readFileSync, mkdirSync, renameSync, writeFileSync } from "node:fs";
import { dirname, isAbsolute, join, resolve, win32 } from "node:path";
import { parse, stringify } from "yaml";
import { parse as parseDotenv } from "dotenv";
import { MAX_STARTUP_TIMEOUT_MS } from "./readiness.js";
import { getStateSaveIntervalMs } from "../config.js";

Expand Down Expand Up @@ -52,6 +53,9 @@ export function composeProjectPaths(dataDir: string): {
}

const AGENTMEMORY_WORKER_ENV_ALLOWLIST = new Set([
"LOCAL_EMBEDDING_MODEL", "LOCAL_EMBEDDING_DIMENSIONS", "LOCAL_EMBEDDING_POOLING",
"LOCAL_EMBEDDING_QUERY_PREFIX", "LOCAL_EMBEDDING_DOCUMENT_PREFIX",
"LOCAL_EMBEDDING_CACHE_DIR", "LOCAL_EMBEDDING_BATCH_SIZE", "LOCAL_EMBEDDING_DTYPE",
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
"EMBEDDING_PROVIDER", "AGENTMEMORY_LLM_PROVIDER", "OPENAI_API_KEY", "OPENAI_BASE_URL",
"OPENAI_MODEL", "OPENAI_EMBEDDING_API_KEY", "OPENAI_EMBEDDING_BASE_URL",
"OPENAI_EMBEDDING_MODEL", "OPENAI_EMBEDDING_DIMENSIONS", "ANTHROPIC_API_KEY",
Expand Down Expand Up @@ -365,6 +369,13 @@ export function renderWorkerCompose(options: ComposeProjectOptions): string {
if (filteredEnvFile) lines.push(` env_file: [${yamlString(filteredEnvFile)}]`);
const explicitWorkerEnvironment = Object.entries(options.workerEnvironment ?? {})
.filter(([key, value]) => value !== undefined && AGENTMEMORY_WORKER_ENV_ALLOWLIST.has(key) && !COMPOSE_RESERVED_ENV.has(key))
.map(([key, value]) => {
const path = windows ? win32 : { isAbsolute, resolve };
const forwardedValue = key === "LOCAL_EMBEDDING_CACHE_DIR" && value && !path.isAbsolute(value)
? path.resolve(options.cwd ?? process.cwd(), value)
: value;
return [key, forwardedValue?.replaceAll("$", () => "$$")] as const;
})
.sort(([left], [right]) => left.localeCompare(right));
lines.push(
" environment:",
Expand Down Expand Up @@ -412,10 +423,15 @@ export function writeWorkerCompose(options: ComposeProjectOptions): { file: stri
chmodSync(join(projectDir, "iii.worker.yaml"), 0o600);
}
}
const text = renderWorkerCompose(options);
const envFileContents = options.envFile ? readFileSync(options.envFile, "utf8") : "";
const cacheDir = options.workerEnvironment?.LOCAL_EMBEDDING_CACHE_DIR ??
(options.envFile ? parseDotenv(envFileContents).LOCAL_EMBEDDING_CACHE_DIR : undefined);
const text = renderWorkerCompose(cacheDir === undefined ? options : {
...options, workerEnvironment: { ...options.workerEnvironment, LOCAL_EMBEDDING_CACHE_DIR: cacheDir },
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
});
if (options.envFile) {
const filteredPath = join(paths.directory, ".agentmemory-filtered.env");
const entries = readFileSync(options.envFile, "utf8").split(/\r?\n/).filter((line) => {
const entries = envFileContents.split(/\r?\n/).filter((line) => {
const match = line.match(/^\s*(?:export\s+)?([A-Za-z_][A-Za-z0-9_]*)\s*=/);
return !match || !COMPOSE_RESERVED_ENV.has(match[1]!);
});
Expand Down
2 changes: 1 addition & 1 deletion src/functions/search.ts
Original file line number Diff line number Diff line change
Expand Up @@ -124,7 +124,7 @@ export async function rankMemoryIds(
let semantic: Array<{ obsId: string }> = []
if (vectorIndex && vectorIndex.size > 0 && currentEmbeddingProvider) {
try {
const embedding = await currentEmbeddingProvider.embed(query)
const embedding = await (currentEmbeddingProvider.embedQuery?.(query) ?? currentEmbeddingProvider.embed(query))
semantic = vectorIndex
.search(embedding, fetchLimit, "memory", (id) => getSearchIndex().layerOf(id))
} catch (err) {
Expand Down
2 changes: 2 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@ import { VectorIndex } from "./state/vector-index.js";
import type { SearchLayer } from "./state/search-layer.js";
import { HybridSearch } from "./state/hybrid-search.js";
import { IndexPersistence } from "./state/index-persistence.js";
import { ensureEmbeddingIdentity } from "./state/embedding-identity.js";
import { SHUTDOWN_FLUSH_TIMEOUT_MS, SHUTDOWN_HARD_EXIT_MS, runStartupCleanup, settleWithin } from "./shutdown.js";
import { registerPrivacyFunction, withWriteScrubbing } from "./functions/privacy.js";
import { withRegistrationDefaults } from "./runtime/registration.js";
Expand Down Expand Up @@ -274,6 +275,7 @@ async function main() {
bootWarn(`Waiting for the state worker to register state::get and state::list (${error})`),
});
assertStateReady(stateReadiness);
await ensureEmbeddingIdentity(kv, embeddingProvider, isDropStaleIndexEnabled());
if (stateReadiness.attempts > 1) {
bootLog(
`State worker ready after ${stateReadiness.waitedMs}ms (${stateReadiness.attempts} attempts)`,
Expand Down
3 changes: 3 additions & 0 deletions src/providers/embedding/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,9 @@ export function withDimensionGuard(provider: EmbeddingProvider): EmbeddingProvid
out.forEach((v, i) => check(v, `embedBatch[${i}]`));
return out;
};
if (provider.embedQuery) {
wrapped.embedQuery = async (t) => check(await provider.embedQuery!(t), "embedQuery");
}
if (provider.embedImage) {
wrapped.embedImage = async (s: string) =>
check(await provider.embedImage!(s), "embedImage");
Expand Down
Loading
Loading