Skip to content

Add vector embedding support and hybrid similarity search to the optimizer ToolStore #3732

@jerm-dro

Description

@jerm-dro

Depends on: #3731

Context

Issue #3731 introduces a ToolStore interface backed by SQLite + FTS5 for basic text similarity search. This issue adds vector embedding support to enable semantic search — finding tools by meaning rather than keyword overlap. A hybrid approach (FTS5 + vector similarity) improves recall for natural-language queries.

In this iteration, we use a fake/stub embedding client to keep the change small and focused on the storage and search mechanics. A real embedding provider is wired in a subsequent issue.

Prototype reference: #3516 (see pkg/vmcp/optimizer/internal/embeddings/ and the hybrid search implementation in internal/db/)

Requirements

  1. Extend the ToolStore interface (from Back the dummyOptimizer with a shared SQLite instance and basic similarity search #3731) to support storing and querying vector embeddings alongside tool metadata.
  2. Introduce an embedding client interface that the store uses to generate embeddings for tool descriptions at ingestion time and for queries at search time. This iteration uses a fake/stub implementation that returns deterministic vectors.
  3. Implement hybrid search: combine FTS5 BM25 scores with vector cosine similarity to produce a blended ranking. The weighting between the two should be configurable.
  4. Include Go benchmark tests (go test -bench) demonstrating embedding search latency over ~1000 tools with a production-grade number of embedding dimensions (e.g., 768 or 1536). These benchmarks establish a baseline for performance regression detection.

High-Level Implementation

  • Define an embedding client interface (e.g., EmbeddingClient) in pkg/vmcp/optimizer/ with a method to generate embeddings from text
  • Create a fake/stub implementation that returns deterministic fixed-dimension vectors (useful for testing and benchmarking)
  • Extend the SQLite ToolStore to store embedding vectors alongside tool records
  • Implement vector cosine similarity search (consider using an in-process vector library or pure-Go implementation)
  • Add a hybrid search method that blends FTS5 rank and cosine similarity with a configurable ratio
  • Update the dummyOptimizer to use hybrid search when an EmbeddingClient is available, falling back to FTS5-only when it is not

Acceptance Tests

  • Embedding client interface tests: Fake client returns vectors of the expected dimensionality and is deterministic
  • Vector storage: Tools ingested with embeddings can be retrieved; embeddings are persisted correctly in SQLite
  • Cosine similarity search: Querying with a vector returns tools ordered by semantic similarity
  • Hybrid search blending: Results reflect both keyword relevance (FTS5) and semantic relevance (vector), controlled by the configurable ratio
  • Fallback behavior: When no EmbeddingClient is provided, search falls back to FTS5-only (no regression from Back the dummyOptimizer with a shared SQLite instance and basic similarity search #3731)
  • Benchmark: 1000 tools @ production dimensions: go test -bench over ~1000 tools with 768+ dimension embeddings, reporting p50/p99 search latency. This benchmark should be checked into the repo for ongoing regression detection.
  • Concurrent embedding search: Multiple goroutines performing hybrid search simultaneously without data races

Metadata

Metadata

Labels

enhancementNew feature or requestgoPull requests that update go codevmcpVirtual MCP Server related issues

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions