Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@ serde = { version = "1.0", features = ["derive"], optional = true }
serde_json = { version = "1.0", optional = true, features = ["float_roundtrip"] }
thiserror = "1.0"
num-traits = "0.2"
# Optional unsupervised input/codebook clustering, shared with RuVector.
ruvector-e-means = { version = "0.1.0", optional = true }

# Parallel processing
rayon = { version = "1.8", optional = true }
Expand Down Expand Up @@ -102,6 +104,7 @@ simd = []
binary = ["dep:bincode"]
compression = ["dep:flate2"]
io = ["binary", "compression", "serde"]
clustering = ["dep:ruvector-e-means", "std"]

# no_std support
no_std = []
Expand Down
55 changes: 55 additions & 0 deletions docs/adr/ADR-013-e-means-clustering.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# ADR-013: Shared e-means input clustering with RuVector

Status: Implemented source architecture; release requires publishing the shared dependencies first.
Date: 2026-11-11.
Reference: Bessa et al., [e-means: outperforming k-means (in one-tenth the epochs), arXiv:2611.09501v1](https://arxiv.org/abs/2611.09501v1).

## Context

ruv-FANN applications need to group training inputs and train representative codebooks. RuVector needs the same operation for vector quantization. Maintaining separate clustering algorithms would allow numerical behavior, validation and defaults to diverge between the two projects and their JavaScript interfaces.

The paper describes mini-batch centroid updates using exponentially weighted counts and sums, cosine scheduling, donor splitting to revive inactive clusters, and last-epoch mean finalization. Its batch-size and peak-step scaling laws provide defaults without application-specific tuning. The published performance measurements use different hardware and datasets from this implementation; they do not establish local CPU or WASM speedups.

## Decision

Use the independent `ruvector-e-means` Rust crate, version `0.1.0`, owned by the RuVector repository. Enable it through the optional `clustering` Cargo feature. Export its `fit`, `Config`, `Finalization`, `Model` and `Error` interfaces from `ruv_fann::clustering`.

Add `TrainingData<f32>::cluster_inputs(&Config)` as the bridge from nested training input rows to the shared flat row-major API. Validate that rows are nonempty and rectangular before flattening. Delegate configuration, finite-coordinate and clustering validation to the shared crate. Return a distinct adapter error for invalid input shapes. Outputs do not participate in this unsupervised operation; the method borrows data without changing inputs, outputs or network weights.

Add the explicit `ruv-fann/clustering` npm subpath, backed by an exact `@ruvector/e-means@0.1.0` dependency. Its JavaScript and TypeScript modules re-export the shared package directly, including asynchronous `init` and synchronous `fit`/`predict` after initialization. The npm package runs the same Rust algorithm compiled to WASM. The root neural inference and training entrypoint does not eagerly initialize clustering.

The initial scope includes exact squared-Euclidean assignment, seeded reproducibility, automatic scaling-law parameters, EMA updates, cosine annealing, dead-cluster revival and last-epoch mean finalization. The alternative `ema` finalization exposes the terminal EMA centroids. Slow-EMA finalization, approximate assignment and GPU kernels remain outside this first implementation.

## Alternatives

Embedding a second Rust implementation in ruv-FANN would duplicate maintenance and numerical testing. A JavaScript rewrite would also introduce a second algorithm and different random-number behavior. Both lose the shared implementation boundary.

Replacing backpropagation, RProp or AdamW with clustering would change the supervised learning contract. The new feature instead supports input analysis and codebook training through a separate API. Applications choose explicitly how to use the resulting assignments and centroids.

## Dependencies and release order

Cargo uses a version-only dependency and npm uses an exact registry version. There are no committed paths to a sibling checkout or unpublished Git revision. Publish `ruvector-e-means@0.1.0` and `@ruvector/e-means@0.1.0`, verify fresh registry installs, then release the consuming ruv-FANN artifacts. Until then, independent dependency resolution requires a local override; passing local checks does not prove registry availability.

For development with sibling checkouts:

```sh
cargo test --features clustering --test clustering_tests \
--config 'patch.crates-io.ruvector-e-means.path="../ruvector/crates/ruvector-e-means"'
```

For npm validation, build and pack the shared package, install that exact archive in an isolated consumer, and import the packed ruv-FANN `clustering` subpath. An archive override is for qualification only; published dependency metadata retains the registry version.

Rollback removes the optional feature usage and npm subpath usage. Existing supervised inference and training callers have no API migration.

## Acceptance and evidence

| Requirement | Executable evidence |
| --- | --- |
| Use the shared Rust dependency | `tests/clustering_tests.rs` compares adapter and direct shared-crate centroids/assignments |
| Provide useful input clustering | Separated-cluster prediction and one-cluster mean numerical oracles |
| Preserve supervised inputs and targets | Adapter test compares borrowed data before and after clustering |
| Reject malformed boundaries | Empty/ragged rows and non-finite-coordinate regressions |
| Expose the same dependency to npm | `packages/core/test/clustering.test.mjs` imports `ruv-fann/clustering` and executes WASM training/prediction |
| Remain reproducible | Same seed/data/config yields identical centroids |

Focused validation must run against actual crate and WASM bytes. The shared crate owns algorithm-level tests and benchmarks; ruv-FANN owns the adapter and package-boundary checks. Broader Cargo and npm regressions remain required when preparing a release. No accuracy guarantee for arbitrary datasets or reproduction of the paper's GPU throughput is implied.
10 changes: 8 additions & 2 deletions integrations/ruv-fann/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,12 @@
".": { "types": "./src/index.d.ts", "import": "./src/index.mjs" },
"./ruvector": { "types": "./src/ruvector.d.ts", "import": "./src/ruvector.mjs" },
"./metaharness": { "types": "./src/metaharness.d.ts", "import": "./src/metaharness.mjs" },
"./node": { "types": "./src/node.d.ts", "import": "./src/node.mjs" }
"./node": { "types": "./src/node.d.ts", "import": "./src/node.mjs" },
"./clustering": { "types": "./src/clustering.d.ts", "import": "./src/clustering.mjs" },
"./clustering/quantization": { "types": "./src/clustering-quantization.d.ts", "import": "./src/clustering-quantization.mjs" },
"./clustering/ivf": { "types": "./src/clustering-ivf.d.ts", "import": "./src/clustering-ivf.mjs" },
"./clustering/metaharness": { "types": "./src/clustering-metaharness.d.ts", "import": "./src/clustering-metaharness.mjs" },
"./clustering/cloudflare": { "types": "./src/clustering-cloudflare.d.ts", "import": "./src/clustering-cloudflare.mjs" }
},
"files": ["src/", "README.md", "LICENSE-MIT", "LICENSE-APACHE"],
"engines": { "node": ">=22.13.0" },
Expand All @@ -25,7 +30,8 @@
},
"dependencies": {
"@ruvector/wasm": "0.2.1",
"@metaharness/darwin": "0.13.0"
"@metaharness/darwin": "0.13.0",
"@ruvector/e-means": "0.1.0"
},
"devDependencies": { "typescript": "5.9.3" }
}
2 changes: 2 additions & 0 deletions integrations/ruv-fann/src/clustering-cloudflare.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering capability; initialize the numerical runtime explicitly. */
export * from '@ruvector/e-means/cloudflare';
2 changes: 2 additions & 0 deletions integrations/ruv-fann/src/clustering-cloudflare.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering capability; initialize the numerical runtime explicitly. */
export * from '@ruvector/e-means/cloudflare';
2 changes: 2 additions & 0 deletions integrations/ruv-fann/src/clustering-ivf.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering capability; initialize the numerical runtime explicitly. */
export * from '@ruvector/e-means/ivf';
2 changes: 2 additions & 0 deletions integrations/ruv-fann/src/clustering-ivf.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering capability; initialize the numerical runtime explicitly. */
export * from '@ruvector/e-means/ivf';
2 changes: 2 additions & 0 deletions integrations/ruv-fann/src/clustering-metaharness.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering capability; initialize the numerical runtime explicitly. */
export * from '@ruvector/e-means/metaharness';
2 changes: 2 additions & 0 deletions integrations/ruv-fann/src/clustering-metaharness.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering capability; initialize the numerical runtime explicitly. */
export * from '@ruvector/e-means/metaharness';
2 changes: 2 additions & 0 deletions integrations/ruv-fann/src/clustering-quantization.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering capability; initialize the numerical runtime explicitly. */
export * from '@ruvector/e-means/quantization';
2 changes: 2 additions & 0 deletions integrations/ruv-fann/src/clustering-quantization.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering capability; initialize the numerical runtime explicitly. */
export * from '@ruvector/e-means/quantization';
2 changes: 2 additions & 0 deletions integrations/ruv-fann/src/clustering.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering capability; initialize the numerical runtime explicitly. */
export * from '@ruvector/e-means';
2 changes: 2 additions & 0 deletions integrations/ruv-fann/src/clustering.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering capability; initialize the numerical runtime explicitly. */
export * from '@ruvector/e-means';
19 changes: 19 additions & 0 deletions integrations/ruv-fann/test/clustering-types.test.mts
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
import { init, type EMeansOptions } from '@ruv-fann/integrations/clustering';
import { trainProductQuantizer, encodeProductQuantizer, decodeProductQuantizer, type ProductQuantizer } from '@ruv-fann/integrations/clustering/quantization';
import { partition, postingList, type IVFPartition } from '@ruv-fann/integrations/clustering/ivf';
import { createClusteringEvaluator } from '@ruv-fann/integrations/clustering/metaharness';
import { createClusteringWorker, type WorkerOptions } from '@ruv-fann/integrations/clustering/cloudflare';

const data = new Float32Array([1, 2, 3, 4]);
const options: EMeansOptions = { clusters: 1 };
await init();
const model: ProductQuantizer = trainProductQuantizer(data, 2, 2, options);
const decoded: Float32Array = decodeProductQuantizer(model, encodeProductQuantizer(model, data));
const ivf: IVFPartition = partition(decoded, 2, options);
const list: Uint32Array = postingList(ivf, 0);
const evaluator: typeof createClusteringEvaluator = createClusteringEvaluator;
declare const workerOptions: WorkerOptions;
const worker = createClusteringWorker(workerOptions);
void [list, evaluator, worker];
// @ts-expect-error float32 vectors are required
trainProductQuantizer([1, 2], 2, 1, options);
42 changes: 42 additions & 0 deletions integrations/ruv-fann/test/clustering.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises';
import { init } from '@ruv-fann/integrations/clustering';
import { trainProductQuantizer, encodeProductQuantizer, decodeProductQuantizer } from '@ruv-fann/integrations/clustering/quantization';
import { partition, postingList } from '@ruv-fann/integrations/clustering/ivf';
import { createClusteringEvaluator } from '@ruv-fann/integrations/clustering/metaharness';
import { createClusteringWorker } from '@ruv-fann/integrations/clustering/cloudflare';

await init();

test('optional integrations expose shared Rust/WASM clustering without a prepared network', () => {
const data = new Float32Array([1, 3, 5, 7]);
const model = trainProductQuantizer(data, 2, 2, { clusters: 1 });
assert.deepEqual(Array.from(model.centroids), [3, 5]);
assert.deepEqual(Array.from(decodeProductQuantizer(model, encodeProductQuantizer(model, data))), [3, 5, 3, 5]);
const result = partition(data, 2, { clusters: 1 });
assert.deepEqual(Array.from(postingList(result, 0)), [0, 1]);
});

test('optional MetaHarness clustering alias evaluates held-out data', async () => {
const evaluator = createClusteringEvaluator({
training: new Float32Array([1, 3, 5, 7]), holdout: new Float32Array([3, 5, 7, 9]), dimensions: 2,
baseline: { clusters: 1, epochs: 2, batchSize: 2, peakStep: 0.5, seed: 42 },
domain: { peakStep: { min: 0.25, max: 1, type: 'float', scale: 'linear' } },
budget: { maxEvaluations: 1, maxDistanceCoordinates: 10000 },
});
const score = await evaluator.evaluate(evaluator.baselineGenome, 'fann-integrations-baseline');
assert.equal(score.raw.holdoutMse, 16);
assert.equal(evaluator.receipt().sourceApplied, false);
});

test('optional Cloudflare clustering alias executes a local Worker fit', async () => {
const wasmModule = await WebAssembly.compile(await readFile(new URL(import.meta.resolve('@ruvector/e-means/wasm'))));
const worker = createClusteringWorker({ wasmModule, allowUnauthenticated: true, maxWork: 10000 });
const response = await worker.fetch(new Request('https://local.test/fit', {
method: 'POST', headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ data: [1, 3, 5, 7], dimensions: 2, options: { clusters: 1 } }),
}));
assert.equal(response.status, 200);
assert.deepEqual((await response.json()).centroids, [3, 5]);
});
2 changes: 1 addition & 1 deletion integrations/ruv-fann/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,5 +9,5 @@
"skipLibCheck": false,
"types": []
},
"include": ["src/**/*.d.ts", "test/types.test.mts"]
"include": ["src/**/*.d.ts", "test/types.test.mts", "test/clustering-types.test.mts"]
}
23 changes: 22 additions & 1 deletion packages/core/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,26 @@
"./agenticow": {
"types": "./src/capabilities/agenticow.d.ts",
"import": "./src/capabilities/agenticow.js"
},
"./clustering": {
"types": "./src/capabilities/clustering.d.ts",
"import": "./src/capabilities/clustering.js"
},
"./clustering/quantization": {
"types": "./src/capabilities/clustering-quantization.d.ts",
"import": "./src/capabilities/clustering-quantization.js"
},
"./clustering/ivf": {
"types": "./src/capabilities/clustering-ivf.d.ts",
"import": "./src/capabilities/clustering-ivf.js"
},
"./clustering/metaharness": {
"types": "./src/capabilities/clustering-metaharness.d.ts",
"import": "./src/capabilities/clustering-metaharness.js"
},
"./clustering/cloudflare": {
"types": "./src/capabilities/clustering-cloudflare.d.ts",
"import": "./src/capabilities/clustering-cloudflare.js"
}
},
"files": [
Expand All @@ -93,7 +113,8 @@
"ruvnet-constellation": "0.1.0",
"ruvnet-constellation-contracts": "0.1.0",
"ruvnet-constellation-telemetry": "0.1.0",
"agenticow": "0.3.0"
"agenticow": "0.3.0",
"@ruvector/e-means": "0.1.0"
},
"scripts": {
"test": "node --test test/*.test.mjs",
Expand Down
2 changes: 2 additions & 0 deletions packages/core/src/capabilities/clustering-cloudflare.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering cloudflare capability. */
export * from '@ruvector/e-means/cloudflare';
2 changes: 2 additions & 0 deletions packages/core/src/capabilities/clustering-cloudflare.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering cloudflare capability. */
export * from '@ruvector/e-means/cloudflare';
2 changes: 2 additions & 0 deletions packages/core/src/capabilities/clustering-ivf.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering ivf capability. */
export * from '@ruvector/e-means/ivf';
2 changes: 2 additions & 0 deletions packages/core/src/capabilities/clustering-ivf.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering ivf capability. */
export * from '@ruvector/e-means/ivf';
2 changes: 2 additions & 0 deletions packages/core/src/capabilities/clustering-metaharness.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering metaharness capability. */
export * from '@ruvector/e-means/metaharness';
2 changes: 2 additions & 0 deletions packages/core/src/capabilities/clustering-metaharness.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering metaharness capability. */
export * from '@ruvector/e-means/metaharness';
2 changes: 2 additions & 0 deletions packages/core/src/capabilities/clustering-quantization.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering quantization capability. */
export * from '@ruvector/e-means/quantization';
2 changes: 2 additions & 0 deletions packages/core/src/capabilities/clustering-quantization.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Explicit shared clustering quantization capability. */
export * from '@ruvector/e-means/quantization';
2 changes: 2 additions & 0 deletions packages/core/src/capabilities/clustering.d.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Shared e-means clustering; call init() before synchronous fit/predict. */
export * from '@ruvector/e-means';
2 changes: 2 additions & 0 deletions packages/core/src/capabilities/clustering.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
/** Shared e-means clustering; call init() before synchronous fit/predict. */
export * from '@ruvector/e-means';
46 changes: 46 additions & 0 deletions packages/core/test/clustering-submodules.test.mjs
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
import test from 'node:test';
import assert from 'node:assert/strict';
import { readFile } from 'node:fs/promises';
import { init } from 'ruv-fann/clustering';
import { trainProductQuantizer, encodeProductQuantizer, decodeProductQuantizer } from 'ruv-fann/clustering/quantization';
import { partition, postingList } from 'ruv-fann/clustering/ivf';
import { createClusteringEvaluator } from 'ruv-fann/clustering/metaharness';
import { createClusteringWorker } from 'ruv-fann/clustering/cloudflare';

await init();

test('clustering submodules expose real shared codebook codecs and IVF posting lists', () => {
const data = new Float32Array([0, 2, 4, 6, 8, 10]);
const model = trainProductQuantizer(data, 2, 2, { clusters: 1, batchSize: 2 });
const codes = encodeProductQuantizer(model, data);
assert.deepEqual(codes, new Uint8Array(6));
assert.deepEqual(Array.from(decodeProductQuantizer(model, codes)), [4, 6, 4, 6, 4, 6]);
const result = partition(data, 2, { clusters: 1, batchSize: 2 });
assert.deepEqual(Array.from(result.model.centroids), [4, 6]);
assert.deepEqual(Array.from(postingList(result, 0)), [0, 1, 2]);
});

test('clustering MetaHarness alias measures a real held-out numerical objective', async () => {
const evaluator = createClusteringEvaluator({
training: new Float32Array([1, 3, 5, 7]), holdout: new Float32Array([3, 5, 7, 9]), dimensions: 2,
baseline: { clusters: 1, epochs: 2, batchSize: 2, peakStep: 0.5, seed: 42 },
domain: { peakStep: { min: 0.25, max: 1, type: 'float', scale: 'linear' } },
budget: { maxEvaluations: 1, maxDistanceCoordinates: 10000 },
});
const score = await evaluator.evaluate(evaluator.baselineGenome, 'fann-core-baseline');
assert.equal(score.raw.trainMse, 8);
assert.equal(score.raw.holdoutMse, 16);
assert.equal(evaluator.receipt().trials.length, 1);
assert.equal(evaluator.receipt().sourceApplied, false);
});

test('clustering Cloudflare alias executes a local Worker request with real WASM', async () => {
const wasmModule = await WebAssembly.compile(await readFile(new URL(import.meta.resolve('@ruvector/e-means/wasm'))));
const worker = createClusteringWorker({ wasmModule, allowUnauthenticated: true, maxWork: 10000 });
const response = await worker.fetch(new Request('https://local.test/fit', {
method: 'POST', headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ data: [1, 3, 5, 7], dimensions: 2, options: { clusters: 1 } }),
}));
assert.equal(response.status, 200);
assert.deepEqual((await response.json()).centroids, [3, 5]);
});
20 changes: 20 additions & 0 deletions packages/core/test/clustering-submodules.types.mts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import { init, type EMeansOptions } from 'ruv-fann/clustering';
import { trainProductQuantizer, encodeProductQuantizer, decodeProductQuantizer, type ProductQuantizer } from 'ruv-fann/clustering/quantization';
import { partition, postingList, type IVFPartition } from 'ruv-fann/clustering/ivf';
import { createClusteringEvaluator, type ClusteringEvaluator } from 'ruv-fann/clustering/metaharness';
import { createClusteringWorker, type WorkerOptions } from 'ruv-fann/clustering/cloudflare';

const data = new Float32Array([1, 2, 3, 4]);
const options: EMeansOptions = { clusters: 1 };
await init();
const model: ProductQuantizer = trainProductQuantizer(data, 2, 2, options);
const reconstructed: Float32Array = decodeProductQuantizer(model, encodeProductQuantizer(model, data));
const ivf: IVFPartition = partition(reconstructed, 2, options);
const list: Uint32Array = postingList(ivf, 0);
const evaluator: typeof createClusteringEvaluator = createClusteringEvaluator;
declare const instance: ClusteringEvaluator;
declare const workerOptions: WorkerOptions;
const worker: { fetch(request: Request): Promise<Response> } = createClusteringWorker(workerOptions);
void [list, evaluator, instance, worker];
// @ts-expect-error byte codes are required
decodeProductQuantizer(model, new Uint32Array());
Loading
Loading