Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ jobs:
# src/functions/obsidian-export.ts needs os.tmpdir() + path.join
# rework before Windows can be added back. Tracked as follow-up.
os: [ubuntu-latest, macos-latest]
node-version: [20, 22]
node-version: [20, 22, 24, 26]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail
jq -r '.scripts.test // "<missing test script>"' package.json
rg -n 'RUN_HF_SMOKE' package.json .github/workflows/ci.yml

Repository: rohitg00/agentmemory

Length of output: 204


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

echo 'package test script:'
jq -r '.scripts.test // "<missing test script>"' package.json

echo
echo 'Workflow snippets around test invocation and env variables:'
sed -n '1,120p' .github/workflows/ci.yml

echo
echo 'RUN_HF_SMOKE occurrences:'
rg -n 'RUN_HF_SMOKE|HF|smoke|local-embedding' package.json .github/workflows/ci.yml test -S || true

Repository: rohitg00/agentmemory

Length of output: 2787


Enable the local embedding smoke test in CI.

The CI test step runs npm test, package scripts.test is vitest run --exclude test/integration.test.ts, and only test/local-embedding-smoke.test.ts gates on RUN_HF_SMOKE=1. Add RUN_HF_SMOKE to this step if the real-model embedding load should be part of CI.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/ci.yml at line 55, Update the CI test step that runs npm
test to set RUN_HF_SMOKE=1, enabling test/local-embedding-smoke.test.ts while
preserving the existing test command and exclusions.

steps:
- uses: actions/checkout@v6
with:
Expand Down
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -933,7 +933,7 @@ BM25 tokenizes Greek, Cyrillic, Hebrew, Arabic, and accented Latin out of the bo
agentmemory auto-detects your provider. For best results, install local embeddings (free):

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| Provider | Model | Cost | Notes |
Expand Down Expand Up @@ -1264,7 +1264,7 @@ OPENAI_MODEL=qwen2.5-coder-7b-instruct # match the model name from LM St

Reasoning-class models (`o1`-style with `<think>` blocks) can return empty `content` with a `reasoning` field your local server may not surface. If extractions come back blank, switch to a non-reasoning model first. The `OPENAI_REASONING_EFFORT=none` env can also disable thinking on Ollama Cloud thinking models that mirror the OpenAI reasoning schema.

Local embeddings ship out of the box via `@xenova/transformers` — `EMBEDDING_PROVIDER=local` (default) gives you BGE-small entirely on-device. No extra config needed.
Local embeddings ship out of the box via `@huggingface/transformers` — `EMBEDDING_PROVIDER=local` (default) gives you `Xenova/all-MiniLM-L6-v2` (384-dim) entirely on-device. No extra config needed.

### Cost-aware model selection

Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.de-DE.md
Original file line number Diff line number Diff line change
Expand Up @@ -809,7 +809,7 @@ BM25 tokenisiert Griechisch, Kyrillisch, Hebräisch, Arabisch und akzentuiertes
agentmemory erkennt Ihren Provider automatisch. Für die besten Ergebnisse installieren Sie lokale Embeddings (kostenlos):

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| Provider | Modell | Kosten | Hinweise |
Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.es-ES.md
Original file line number Diff line number Diff line change
Expand Up @@ -802,7 +802,7 @@ BM25 tokeniza griego, cirílico, hebreo, árabe y latín con tildes de serie. Pa
agentmemory autodetecta tu proveedor. Para mejores resultados, instala embeddings locales (gratis):

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| Proveedor | Modelo | Coste | Notas |
Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.fr-FR.md
Original file line number Diff line number Diff line change
Expand Up @@ -809,7 +809,7 @@ BM25 tokenise nativement le grec, le cyrillique, l'hébreu, l'arabe et le latin
agentmemory détecte automatiquement votre fournisseur. Pour de meilleurs résultats, installez les embeddings locaux (gratuits) :

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| Fournisseur | Modèle | Coût | Notes |
Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.hi-IN.md
Original file line number Diff line number Diff line change
Expand Up @@ -812,7 +812,7 @@ BM25 box से बाहर ही Greek, Cyrillic, Hebrew, Arabic, और acce
agentmemory आपके provider को auto-detect करता है। सर्वोत्तम परिणामों के लिए, local embeddings install करें (free):

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| Provider | Model | Cost | नोट्स |
Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.ja-JP.md
Original file line number Diff line number Diff line change
Expand Up @@ -812,7 +812,7 @@ BM25 は箱から出してすぐにギリシャ文字、キリル文字、ヘブ
agentmemory はプロバイダーを自動検出します。最良の結果を得るには、ローカル埋め込み(無料)をインストール:

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| プロバイダー | モデル | コスト | 備考 |
Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.ko-KR.md
Original file line number Diff line number Diff line change
Expand Up @@ -793,7 +793,7 @@ BM25는 기본적으로 그리스어, 키릴 문자, 히브리어, 아랍어,
agentmemory는 프로바이더를 자동 감지합니다. 최상의 결과를 위해 로컬 임베딩을 설치하십시오 (무료):

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| 프로바이더 | 모델 | 비용 | 비고 |
Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.pt-BR.md
Original file line number Diff line number Diff line change
Expand Up @@ -802,7 +802,7 @@ BM25 tokeniza grego, cirílico, hebraico, árabe e latim acentuado de fábrica.
agentmemory autodetecta seu provider. Para melhores resultados, instale embeddings locais (gratuito):

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| Provider | Modelo | Custo | Notas |
Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.ru-RU.md
Original file line number Diff line number Diff line change
Expand Up @@ -809,7 +809,7 @@ BM25 «из коробки» токенизирует греческий, кир
agentmemory автоматически определяет вашего провайдера. Для лучших результатов поставьте локальные эмбеддинги (бесплатно):

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| Провайдер | Модель | Стоимость | Заметки |
Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.tr-TR.md
Original file line number Diff line number Diff line change
Expand Up @@ -813,7 +813,7 @@ BM25, Yunanca, Kiril, İbranice, Arapça ve aksanlı Latin'i kutudan çıkar ç
agentmemory sağlayıcınızı otomatik algılar. En iyi sonuçlar için yerel embedding'leri kurun (ücretsiz):

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| Sağlayıcı | Model | Maliyet | Notlar |
Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -810,7 +810,7 @@ BM25 开箱即用支持希腊语、西里尔语、希伯来语、阿拉伯语和
agentmemory 自动检测你的提供者。为获得最佳效果,安装本地嵌入(免费):

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| 提供者 | 模型 | 成本 | 备注 |
Expand Down
2 changes: 1 addition & 1 deletion READMEs/README.zh-TW.md
Original file line number Diff line number Diff line change
Expand Up @@ -810,7 +810,7 @@ BM25 開箱即用支援希臘文、西里爾文、希伯來文、阿拉伯文和
agentmemory 自動偵測你的提供者。為獲得最佳效果,安裝本地嵌入(免費):

```bash
npm install @xenova/transformers
npm install @huggingface/transformers
```

| 提供者 | 模型 | 成本 | 備註 |
Expand Down
2 changes: 1 addition & 1 deletion SECURITY.md
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,7 @@ Supply-chain monitoring we already do:

- Dependabot opens PRs for every minor/patch bump on the production dep list (visible in the open PRs).
- Every PR runs the full test suite on ubuntu-latest + macos-latest, Node 20 + 22, before any merge.
- `optionalDependencies` (`@xenova/transformers`, `onnxruntime-node`, etc.) are guarded by `try { await import("...") } catch` so a missing or compromised optional dep cannot break the core runtime path.
- `optionalDependencies` (`@huggingface/transformers`, transitively `onnxruntime-node`, etc.) are guarded by `try { await import("...") } catch` so a missing or compromised optional dep cannot break the core runtime path.

If you find a malicious package in our dep tree, file via the GHSA flow at the top of this document — that's the fastest path to a fixed release on npm.

Expand Down
4 changes: 4 additions & 0 deletions __mocks__/@huggingface/transformers.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
throw Object.assign(
new Error("Cannot find package '@huggingface/transformers'"),
{ code: "ERR_MODULE_NOT_FOUND" },
);
2 changes: 1 addition & 1 deletion benchmark/LONGMEMEVAL.md
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,6 @@ hf_hub_download(repo_id='xiaowu0162/longmemeval-cleaned', filename='longmemeval_
# Run BM25-only
npx tsx benchmark/longmemeval-bench.ts bm25

# Run BM25+Vector hybrid (requires @xenova/transformers)
# Run BM25+Vector hybrid (requires @huggingface/transformers)
npx tsx benchmark/longmemeval-bench.ts hybrid
```
2 changes: 1 addition & 1 deletion benchmark/REAL-EMBEDDINGS.md
Original file line number Diff line number Diff line change
Expand Up @@ -59,7 +59,7 @@ Embedding is a one-time cost at ingestion. Search is sub-millisecond after index

## Recommendation

Enable local embeddings by default (`EMBEDDING_PROVIDER=local` or install `@xenova/transformers`).
Enable local embeddings by default (`EMBEDDING_PROVIDER=local` or install `@huggingface/transformers`).
This gives agentmemory genuine semantic search that built-in agent memories cannot match —
understanding that "database performance optimization" relates to "N+1 query fix" and "eager loading".

Expand Down
4 changes: 2 additions & 2 deletions benchmark/real-embeddings-eval.ts
Original file line number Diff line number Diff line change
Expand Up @@ -335,7 +335,7 @@ function generateReport(systems: SystemResult[], obsCount: number): string {
w("");
w("## Recommendation");
w("");
w("Enable local embeddings by default (`EMBEDDING_PROVIDER=local` or install `@xenova/transformers`).");
w("Enable local embeddings by default (`EMBEDDING_PROVIDER=local` or install `@huggingface/transformers`).");
w("This gives agentmemory genuine semantic search that built-in agent memories cannot match —");
w("understanding that \"database performance optimization\" relates to \"N+1 query fix\" and \"eager loading\".");
w("");
Expand All @@ -357,7 +357,7 @@ async function main() {
console.log(`Model loaded. Dimensions: ${testEmbed.length}\n`);
} catch (err) {
console.error("Failed to load Xenova model:", err);
console.error("Install with: npm install @xenova/transformers");
console.error("Install with: npm install @huggingface/transformers");
process.exit(1);
}

Expand Down
4 changes: 1 addition & 3 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -69,10 +69,8 @@
"zod": "^4.0.0"
},
"optionalDependencies": {
"@huggingface/transformers": "^4.2.0",
"@node-rs/jieba": "^2.0.1",
"@xenova/transformers": "^2.17.2",
"onnxruntime-node": "^1.14.0",
"onnxruntime-web": "^1.14.0",
"tiny-segmenter": "^0.2.0"
},
"devDependencies": {
Expand Down
63 changes: 25 additions & 38 deletions src/providers/embedding/clip.ts
Original file line number Diff line number Diff line change
@@ -1,32 +1,20 @@
import { readFile } from "node:fs/promises";
import type { RawImage } from "@huggingface/transformers";
import type { EmbeddingProvider } from "../../types.js";

type TransformersModule = {
pipeline: (
task: string,
model: string,
) => Promise<ClipPipeline>;
RawImage: {
fromBlob: (blob: Blob) => Promise<RawImageInstance>;
};
};

type RawImageInstance = unknown;

type TransformersModule = typeof import("@huggingface/transformers");
type ClipPipeline = (
input: string[] | RawImageInstance | RawImageInstance[],
input: string[] | RawImage | RawImage[],
options?: { pooling?: string; normalize?: boolean },
) => Promise<{ tolist: () => number[][]; data: Float32Array }>;

const DEFAULT_MODEL = "Xenova/clip-vit-base-patch32";
const DIMENSIONS = 512;

export class ClipEmbeddingProvider implements EmbeddingProvider {
readonly name = "clip";
readonly dimensions = DIMENSIONS;
readonly dimensions = 512;
private textExtractor: ClipPipeline | null = null;
private imageExtractor: ClipPipeline | null = null;
private transformers: TransformersModule | null = null;
private readonly modelId: string;

constructor(modelId: string = DEFAULT_MODEL) {
Expand All @@ -45,55 +33,54 @@ export class ClipEmbeddingProvider implements EmbeddingProvider {
}

async embedImage(src: string): Promise<Float32Array> {
const t = await this.getTransformers();
const t = await loadTransformers();
const image = await loadImage(t, src);
const extractor = await this.getImageExtractor();
const output = await extractor(image);
const vec = output.data ?? new Float32Array(output.tolist()[0] || []);
return normalize(vec);
}

private async getTransformers(): Promise<TransformersModule> {
if (this.transformers) return this.transformers;
try {
this.transformers = (await import("@xenova/transformers")) as unknown as TransformersModule;
} catch {
throw new Error(
"Install @xenova/transformers for CLIP image embeddings: npm install @xenova/transformers",
);
}
return this.transformers;
}

private async getTextExtractor(): Promise<ClipPipeline> {
if (this.textExtractor) return this.textExtractor;
const t = await this.getTransformers();
this.textExtractor = await t.pipeline("feature-extraction", this.modelId);
const t = await loadTransformers();
this.textExtractor = (await t.pipeline("feature-extraction", this.modelId, { dtype: "q8" })) as ClipPipeline;
return this.textExtractor;
}

private async getImageExtractor(): Promise<ClipPipeline> {
if (this.imageExtractor) return this.imageExtractor;
const t = await this.getTransformers();
this.imageExtractor = await t.pipeline("image-feature-extraction", this.modelId);
const t = await loadTransformers();
this.imageExtractor = (await t.pipeline("image-feature-extraction", this.modelId, { dtype: "q8" })) as ClipPipeline;
return this.imageExtractor;
}
}

async function loadTransformers(): Promise<TransformersModule> {
try {
return await import("@huggingface/transformers");
} catch (err) {
if ((err as NodeJS.ErrnoException).code === "ERR_MODULE_NOT_FOUND") {
throw new Error(
"Install @huggingface/transformers for CLIP embeddings: npm install @huggingface/transformers",
);
}
throw err;
}
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved

async function loadImage(
t: TransformersModule,
src: string,
): Promise<RawImageInstance> {
): Promise<RawImage> {
if (src.startsWith("data:")) {
const comma = src.indexOf(",");
const b64 = comma >= 0 ? src.slice(comma + 1) : src;
const buf = Buffer.from(b64, "base64");
const blob = new Blob([buf]);
return t.RawImage.fromBlob(blob);
return t.RawImage.fromBlob(new Blob([buf]));
}
const data = await readFile(src);
const blob = new Blob([data]);
return t.RawImage.fromBlob(blob);
return t.RawImage.fromBlob(new Blob([data]));
}

function normalize(vec: Float32Array): Float32Array {
Expand Down
41 changes: 18 additions & 23 deletions src/providers/embedding/local.ts
Original file line number Diff line number Diff line change
@@ -1,19 +1,14 @@
import type { EmbeddingProvider } from "../../types.js";

type Pipeline = (
task: string,
model: string,
) => Promise<
(
texts: string[],
options: { pooling: string; normalize: boolean },
) => Promise<{ tolist: () => number[][] }>
>;
type FeatureExtractor = (
texts: string[],
options: { pooling: string; normalize: boolean },
) => Promise<{ tolist: () => number[][] }>;

export class LocalEmbeddingProvider implements EmbeddingProvider {
readonly name = "local";
readonly dimensions = 384;
private extractor: Awaited<ReturnType<Pipeline>> | null = null;
private extractor: FeatureExtractor | null = null;

async embed(text: string): Promise<Float32Array> {
const [result] = await this.embedBatch([text]);
Expand All @@ -26,27 +21,27 @@ export class LocalEmbeddingProvider implements EmbeddingProvider {
pooling: "mean",
normalize: true,
});
const vectors = output.tolist();
return vectors.map((v: number[]) => new Float32Array(v));
return output.tolist().map((v) => new Float32Array(v));
}

private async getExtractor() {
if (this.extractor) return this.extractor;

let transformers: { pipeline: Pipeline };
let transformers: typeof import("@huggingface/transformers");
try {
// @ts-ignore - optional peer dependency
transformers = await import("@xenova/transformers");
} catch {
throw new Error(
"Install @xenova/transformers for local embeddings: npm install @xenova/transformers",
);
transformers = await import("@huggingface/transformers");
} catch (err) {
if ((err as NodeJS.ErrnoException).code === "ERR_MODULE_NOT_FOUND") {
throw new Error(
"Install @huggingface/transformers for local embeddings: npm install @huggingface/transformers",
);
}
throw err;
}

this.extractor = await transformers.pipeline(
this.extractor = (await transformers.pipeline(
"feature-extraction",
"Xenova/all-MiniLM-L6-v2",
);
{ dtype: "q8" },
)) as FeatureExtractor;
return this.extractor;
}
}
4 changes: 2 additions & 2 deletions src/state/reranker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -12,12 +12,12 @@ async function loadPipeline(): Promise<any> {
pipelineLoading = (async () => {
try {
const { pipeline: createPipeline } = await import(
"@xenova/transformers"
"@huggingface/transformers"
);
pipeline = await createPipeline(
"text-classification",
"Xenova/ms-marco-MiniLM-L-6-v2",
{ quantized: true },
{ dtype: "q8" },
);
return pipeline;
} catch {
Expand Down
3 changes: 0 additions & 3 deletions src/xenova.d.ts

This file was deleted.

Loading