What problem does this solve?
RAG at the edge could benefit from Rust, as this enables higher performance and an easy path to WASM. A pure Rust implementation would also give us an easier path to eclectic platforms such as Windows etc #14
Proposed solution
https://github.com/starpit/LEANN/tree/feat/rust/leann-rs
This is a 100% Rust port, including of the HNSW algorithm,. This includes a Rust CLI. There are a few remaining TODOs (e.g. DiskAnn), but I figured this was a good starting point. There is a leann-python crate, which has python bindings, with tests to cover the API compatibility. We could offer wasm bindings, etc. Lots of possibilities.
Compliance
Performance
I have done some performance benchmarking: Benchmark Results. In summary:
- Geometric mean speedup: 6.07x
- Rust better: 40/44
- Python better: 0/44
- Statistical tie: 4/44
- Best Rust speedup: distance/l2/128 (206.0x)
Example usage
What problem does this solve?
RAG at the edge could benefit from Rust, as this enables higher performance and an easy path to WASM. A pure Rust implementation would also give us an easier path to eclectic platforms such as Windows etc #14
Proposed solution
https://github.com/starpit/LEANN/tree/feat/rust/leann-rs
This is a 100% Rust port, including of the HNSW algorithm,. This includes a Rust CLI. There are a few remaining TODOs (e.g. DiskAnn), but I figured this was a good starting point. There is a leann-python crate, which has python bindings, with tests to cover the API compatibility. We could offer wasm bindings, etc. Lots of possibilities.
Compliance
Performance
I have done some performance benchmarking: Benchmark Results. In summary:
Example usage