-
Notifications
You must be signed in to change notification settings - Fork 192
Closed
Labels
enhancementNew feature or requestNew feature or requestgoPull requests that update go codePull requests that update go codevmcpVirtual MCP Server related issuesVirtual MCP Server related issues
Description
Depends on: #3731
Context
Issue #3731 introduces a ToolStore interface backed by SQLite + FTS5 for basic text similarity search. This issue adds vector embedding support to enable semantic search — finding tools by meaning rather than keyword overlap. A hybrid approach (FTS5 + vector similarity) improves recall for natural-language queries.
In this iteration, we use a fake/stub embedding client to keep the change small and focused on the storage and search mechanics. A real embedding provider is wired in a subsequent issue.
Prototype reference: #3516 (see pkg/vmcp/optimizer/internal/embeddings/ and the hybrid search implementation in internal/db/)
Requirements
- Extend the
ToolStoreinterface (from Back the dummyOptimizer with a shared SQLite instance and basic similarity search #3731) to support storing and querying vector embeddings alongside tool metadata. - Introduce an embedding client interface that the store uses to generate embeddings for tool descriptions at ingestion time and for queries at search time. This iteration uses a fake/stub implementation that returns deterministic vectors.
- Implement hybrid search: combine FTS5 BM25 scores with vector cosine similarity to produce a blended ranking. The weighting between the two should be configurable.
- Include Go benchmark tests (
go test -bench) demonstrating embedding search latency over ~1000 tools with a production-grade number of embedding dimensions (e.g., 768 or 1536). These benchmarks establish a baseline for performance regression detection.
High-Level Implementation
- Define an embedding client interface (e.g.,
EmbeddingClient) inpkg/vmcp/optimizer/with a method to generate embeddings from text - Create a fake/stub implementation that returns deterministic fixed-dimension vectors (useful for testing and benchmarking)
- Extend the SQLite
ToolStoreto store embedding vectors alongside tool records - Implement vector cosine similarity search (consider using an in-process vector library or pure-Go implementation)
- Add a hybrid search method that blends FTS5 rank and cosine similarity with a configurable ratio
- Update the
dummyOptimizerto use hybrid search when anEmbeddingClientis available, falling back to FTS5-only when it is not
Acceptance Tests
- Embedding client interface tests: Fake client returns vectors of the expected dimensionality and is deterministic
- Vector storage: Tools ingested with embeddings can be retrieved; embeddings are persisted correctly in SQLite
- Cosine similarity search: Querying with a vector returns tools ordered by semantic similarity
- Hybrid search blending: Results reflect both keyword relevance (FTS5) and semantic relevance (vector), controlled by the configurable ratio
- Fallback behavior: When no
EmbeddingClientis provided, search falls back to FTS5-only (no regression from Back the dummyOptimizer with a shared SQLite instance and basic similarity search #3731) - Benchmark: 1000 tools @ production dimensions:
go test -benchover ~1000 tools with 768+ dimension embeddings, reporting p50/p99 search latency. This benchmark should be checked into the repo for ongoing regression detection. - Concurrent embedding search: Multiple goroutines performing hybrid search simultaneously without data races
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
enhancementNew feature or requestNew feature or requestgoPull requests that update go codePull requests that update go codevmcpVirtual MCP Server related issuesVirtual MCP Server related issues