Skip to content

feat(routing): calibrated hybrid semantic search scoring, explicit search modes, and test bench simulator overhaul - #261

Merged
spelech merged 10 commits into
mainfrom
feat/semantic-search-hybrid-calibration
Oct 3, 2026
Merged

spelech merged 10 commits into
mainfrom
feat/semantic-search-hybrid-calibration

Conversation

@spelech

@spelech spelech commented Oct 3, 2026

Copy link
Copy Markdown
Owner

Summary

This PR aligns Model Context Gateway's semantic search and simulator with ContextCortex (PR #36), introducing calibrated hybrid scoring, explicit search modes, decomposed score diagnostics, and a redesigned Semantic Router Simulator.

1. Calibrated Hybrid Scoring & Explicit Search Modes (MCP-40, MCP-41, MCP-42)

  • Scoring Formula: Normalized linear combination model:
    374453\text{Score} = \alpha \cdot \text{DenseScore} + (1.0 - \alpha) \cdot \text{SparseScore}374453
  • Configurable Dense Weight: Dynamic dense weight $\alpha \in [0.0, 1.0]$ (default: 0.5 balanced).
  • Search Modes:
    • hybrid: Balanced or weighted combination of dense vector similarity and normalized lexical matching.
    • semantic: Pure dense SIMD vector similarity ($\alpha = 1.0$).
    • lexical: Pure sparse / token keyword matching ($\alpha = 0.0$, bypassing embedding generation).
  • Score Decomposition: ToolSearchResult exposes Score, DenseScore, SparseScore, DenseRank, and SparseRank.
  • Meta-Mode search_tools: MCP tool declaration and execution updated with optional mode and dense_weight parameters with 100% backward compatibility for existing callers.
  • REST API Endpoint: POST /api/test/semantic-search receives mode, denseWeight, and limit, returning an envelope with decomposed results.

2. Semantic Router Simulator Redesign (UI-142, UI-143)

  • 3-Way Search Mode Toggle: Segmented buttons for Hybrid Fusion, Semantic (Vector), and Lexical (Keyword).
  • Interactive Hybrid Split Slider & Presets: Conditionally rendered in hybrid mode with a live ratio indicator (Semantic X% / Keyword Y%) and 5 quick calibration presets:
    • Balanced (50/50)
    • Semantic Bias (70/30)
    • Keyword Bias (30/70)
    • Pure Semantic (100/0)
    • Pure Keyword (0/100)
  • Result Limit Selector: Dropdown for 5, 10, 15, or 25 tools.
  • Enhanced Hit Cards: Decomposed score badges (Score: XX.X%, Semantic: XX.X%, Keyword: XX.X%), server badges, and formatted markdown descriptions.
  • 1-Click "Test Tool" Action: Instantly transitions to the Tools tab and preselects the server and tool in the Tool Tester form.

3. Documentation & Live Real Screenshots

  • Captured live screenshot of the updated simulator: docs/assets/semantic_router_simulator.jpg.
  • Updated docs/user-guide/test-bench/semantic-search.md and ARCHITECTURE.md with scoring diagrams, formula, modes, and API guides.

4. Verification & Quality Gates

  • Backend .NET Tests: 909/909 tests passing (including unit & integration tests for calibrated hybrid search, modes, and endpoints).
  • Frontend Vitest: 47/47 test files passing (300/300 tests passing).
  • Frontend ESLint: 0 errors, 0 warnings.
  • Frontend Build: SPA production bundle compiled cleanly in wwwroot/.
  • Playwright E2E: Multi-viewport layout audits passing with 0 overflow across desktop and half-ultrawide viewports (1720x1440 and 1280x1080).
  • SRS Requirements Catalog: Verified zero drift (464 requirements across 1078 proofs).
  • Version Bump: Bumped to 5.21.0 across ModelContextGateway.csproj, frontend/package.json, useUserStore.ts, CHANGELOG.md, and README.md.

catch (Exception ex)
{
rrfScores[matched.Tool] = rrf;
logger?.LogWarning(ex, "Failed to generate query embedding for '{Query}'. Falling back to keyword ranking.", query);
@spelech
spelech merged commit f11110c into main Oct 3, 2026
9 checks passed
@spelech
spelech deleted the feat/semantic-search-hybrid-calibration branch October 3, 2026 04:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants