diff --git a/CHANGELOG.md b/CHANGELOG.md index e3d433f..995e0d5 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,16 @@ All notable changes to the `agent-root` CLI are documented in this file. Format ## [Unreleased] +### Added + +- `search` now falls back to the registry's hybrid semantic endpoint + (`/api/search`, BM25 + vector RRF) when keyword search returns no results. + Natural-language queries like "skill to pay in USDC" surface relevant records + under a "closest semantic matches" heading, each tagged by match type + (`{hybrid}`/`{vector}`/`{keyword}`). Add `--no-semantic` to opt out + (keyword-only). One request per failed search; `--all` unaffected. `--json` + gains a top-level `approximate` boolean. + ## [0.3.0] 2026-05-20 First standalone release. Cut from the independent diff --git a/README.md b/README.md index dc42fbe..3785790 100644 --- a/README.md +++ b/README.md @@ -145,6 +145,7 @@ OPTIONS --quiet, -q Suppress non-essential output (spinners + notes) --no-install Skip auto-install when resolving skill= records --no-color Disable ANSI color (also auto-off in non-TTY) + --no-semantic Disable the search semantic-search fallback (keyword-only) ``` Flag names accept kebab-case (`--manifest-url`) or camelCase (`--manifestUrl`); use `--key=value` or `--key value`. Pass `--` to stop option parsing. @@ -228,7 +229,14 @@ agent-root search doma --page 2 --limit 50 agent-root search doma --all --type skill ``` -For scripting, `--json` returns the full envelope (`results`, `total`, `page`, `pages`, `limit`): +When a keyword search finds nothing, the CLI automatically falls back to the registry's semantic search (hybrid keyword + vector) and shows the closest matches, each tagged `{hybrid}`, `{vector}`, or `{keyword}`. Pass `--no-semantic` to disable it and keep keyword-only behavior: + +```bash +agent-root search "skill to pay in USDC" # semantic match — no literal "usdc" needed +agent-root search "register domain" --no-semantic # keyword-only, no fallback +``` + +For scripting, `--json` returns the full envelope (`results`, `total`, `page`, `pages`, `limit`, `approximate`): ```bash agent-root search doma --json | jq -r '.results[] | select(.type=="skill") | .address' diff --git a/docs/TESTING.md b/docs/TESTING.md index a89edc6..0c24584 100644 --- a/docs/TESTING.md +++ b/docs/TESTING.md @@ -77,8 +77,8 @@ agent-root search payments --json ``` Expected: exit `0`. Valid JSON. Top-level keys are -`results`, `total`, `page`, `pages`, `limit`. This is the full envelope, -not a bare array. +`results`, `total`, `page`, `pages`, `limit`, `approximate`. This is the full +envelope, not a bare array. ```bash agent-root search doma --page 2 --limit 5 --json @@ -86,6 +86,16 @@ agent-root search doma --page 2 --limit 5 --json Expected: exit `0`. `page=2`, `limit=5`, up to 5 entries in `results`. +```bash +agent-root search "skill to pay in USDC" +agent-root search "skill to pay in USDC" --no-semantic +``` + +Expected: both exit `0`. The first may print a +`// no exact match — closest semantic matches:` header with records tagged +`{hybrid}`/`{vector}`/`{keyword}` (the semantic fallback fires only when the +keyword search returns nothing); the second never shows that section. + ### health ```bash @@ -292,12 +302,17 @@ MONO=../agentroot/packages/cli/dist/bin/agentroot.js # (cd ../agentroot && pnpm --filter agent-root build) diff <(node $CLI resolve agentroot.io) <(node $MONO resolve agentroot.io) -diff <(node $CLI search payments --json) <(node $MONO search payments --json) diff <(node $CLI list --json) <(node $MONO list --json) ``` -Expected: zero differences for `resolve` and `search`. `list` output should -also match (same on-disk store). +Expected: zero differences for `resolve`. `list` output should also match +(same on-disk store). + +> `search` is **intentionally no longer byte-for-byte** with the monorepo CLI: +> the standalone adds the semantic-search fallback (`/api/search`) plus an +> `approximate` field in the `--json` envelope, neither of which the monorepo +> `packages/cli` has. The standalone repo is the source of truth — do not diff +> `search` output against the monorepo. For `install --json` parity (paths embed `$HOME` and `$PWD`, so normalize those before diffing): diff --git a/src/cli/parse-args.ts b/src/cli/parse-args.ts index b8f9a87..fac0c9c 100644 --- a/src/cli/parse-args.ts +++ b/src/cli/parse-args.ts @@ -14,7 +14,7 @@ export interface ParsedArgs { */ const BOOLEAN_FLAGS: ReadonlySet = new Set([ 'all', 'project', 'help', 'json', 'force', 'yes', 'version', 'quiet', - 'noInstall', 'noColor', + 'noInstall', 'noColor', 'noSemantic', ]); /** @@ -42,7 +42,7 @@ const SHORT_ALIASES: Record = { * normalizes to `noInstall`). */ const POSITIVE_NO_FLAGS: ReadonlySet = new Set([ - 'noInstall', 'noColor', + 'noInstall', 'noColor', 'noSemantic', ]); /** diff --git a/src/commands/help.ts b/src/commands/help.ts index bab37b0..6190b5e 100644 --- a/src/commands/help.ts +++ b/src/commands/help.ts @@ -63,17 +63,25 @@ ${section('DESCRIPTION')} filters narrow by record kind. Pagination is server-side; use --page / --limit to walk results, or --all to collect every match (capped at 1000). + When a keyword search returns no results, the CLI falls back to the registry's + hybrid semantic search (/api/search, keyword + vector) and shows the closest + matches, each tagged {hybrid}, {vector}, or {keyword}. Pass --no-semantic to + disable this and keep keyword-only behavior. With --json, an 'approximate' + boolean indicates the results came from the semantic fallback. + ${section('OPTIONS')} --type Filter by record type: agent, mcp, skill, a2a, payment --page Page number (1-indexed, default 1) --limit Per-page limit (1..100, default 20) --all Fetch every page (hard-capped at 1000) + --no-semantic Disable the semantic-search fallback (keyword-only) --json, -j Output as JSON envelope --no-color Disable ANSI color --quiet, -q Suppress progress chatter ${section('EXAMPLES')} agentroot search billing + agentroot search "skill to pay in USDC" # semantic fallback when no keyword hit agentroot search "register domain" --type skill agentroot search doma --limit 5 --json | jq '.results[].address' diff --git a/src/commands/search.ts b/src/commands/search.ts index c1dfc34..8084713 100644 --- a/src/commands/search.ts +++ b/src/commands/search.ts @@ -30,6 +30,12 @@ const ALL_MODE_HARD_CAP = 1000; */ const BARE_KEYWORD_TLDS: readonly string[] = ['.io', '.com']; +/** + * `/api/search` (semantic) caps `limit` at 50 — it returns a ranked top-N, not + * a browsable page. We never paginate it; one request per failed keyword search. + */ +const SEMANTIC_MAX_LIMIT = 50; + export interface SearchResult { domain: string; type: string; @@ -44,6 +50,10 @@ export interface SearchResult { transport?: string | null; index?: string | null; capabilities?: string[] | null; + /** Match provenance — set only on hits from the semantic (/api/search) tier. */ + match?: 'hybrid' | 'vector' | 'keyword'; + /** True only for semantic-tier hits (keyword search returned nothing). */ + approximate?: boolean; } export interface SearchEnvelope { @@ -77,6 +87,30 @@ interface RecordsApiResponse { pages?: number; } +/** A single hit from the registry's hybrid /api/search endpoint (BM25 + vector RRF). */ +export interface SemanticSearchHit { + id: number; + record_id: string; + domain: string; + type: string; + name: string | null; + description: string | null; + manifest_url: string | null; + rrf_score: number; + bm25_rank: number | null; + vector_rank: number | null; +} + +/** Response envelope from GET /api/search. */ +export interface SemanticSearchResponse { + query: string; + type_filter: string | null; + results: SemanticSearchHit[]; + total: number; + /** True when the embedder is unavailable; results are BM25-only. */ + degraded: boolean; +} + type ManifestRecord = { record_id?: unknown; id?: unknown; @@ -139,6 +173,63 @@ export function recordToSearchResult(row: RecordsApiRow): SearchResult { }; } +/** + * Semantic fallback. Queries the registry's hybrid /api/search endpoint + * (BM25 + pgvector RRF) and maps the hits to approximate-tagged SearchResults. + * One request, no pagination; limit capped at SEMANTIC_MAX_LIMIT. + * + * Mirrors the silent-catch contract of fallbackFindSkills / fallbackManifestProbe: + * any failure resolves to [] so searchWithFallback continues down the chain. + * fetch() follows 3xx, so the apex->www redirect is handled transparently. + */ +export async function searchSemantic(query: string, typeFilter: string, flags: Record): Promise { + const limit = Math.min(clampLimit(flags['limit']), SEMANTIC_MAX_LIMIT); + const params = new URLSearchParams({ q: query, limit: String(limit) }); + if (typeFilter) params.set('type', typeFilter); + try { + const data = await fetchJSON(`${getApiBase()}/api/search?${params.toString()}`); + const hits = Array.isArray(data.results) ? data.results : []; + return hits.map(semanticHitToSearchResult); + } catch { + // 429 (fetchJSON throws "HTTP 429 …"), 5xx, timeout, or network error — + // treat exactly like "no results" so the caller falls through to the legacy + // find-skills / manifest-probe tiers instead of crashing the run. + return []; + } +} + +/** + * Map a /api/search hit onto the shared SearchResult shape, tagged `approximate` + * so the display + JSON layers can flag it. `verified` is true because the + * hybrid SQL only returns rows with status='active'. + */ +export function semanticHitToSearchResult(hit: SemanticSearchHit): SearchResult { + const recordId = String(hit.record_id ?? ''); + return { + domain: String(hit.domain ?? ''), + type: String(hit.type ?? 'skill'), + id: recordId, + record_id: recordId, + name: hit.name ?? recordId, + description: hit.description ?? '', + address: `${hit.domain}/${recordId}`, + verified: true, + match: matchKind(hit), + approximate: true, + }; +} + +/** + * Classify a hybrid-search hit by which signal(s) ranked it, for an intuitive + * match-type tag. When the embedder is offline the registry returns BM25-only + * hits (vector_rank null), which classify as 'keyword' — the degraded signal. + */ +export function matchKind(hit: SemanticSearchHit): 'hybrid' | 'vector' | 'keyword' { + if (hit.bm25_rank !== null && hit.vector_rank !== null) return 'hybrid'; + if (hit.vector_rank !== null) return 'vector'; + return 'keyword'; +} + interface FetchRecordsPageOpts { query: string; typeFilter: string; @@ -272,9 +363,14 @@ async function fallbackManifestProbe(query: string, typeFilter: string): Promise * returned nothing. */ export async function searchWithFallback(query: string, typeFilter: string, flags: Record): Promise { - void flags; const env = await searchRecords(query, typeFilter, 1, DEFAULT_LIMIT); if (env.results.length > 0) return env.results; + // Semantic tier: hybrid /api/search recall before the legacy fallbacks. + // Opt out with --no-semantic (parsed as flags.noSemantic). + if (!flags['noSemantic']) { + const semantic = await searchSemantic(query, typeFilter, flags); + if (semantic.length > 0) return semantic; + } // Only chain into find-skills if the filter is open or explicitly skill, // it only returns skills, so calling it for type=agent etc. is wasted I/O. if (!typeFilter || typeFilter === 'skill') { @@ -285,14 +381,22 @@ export async function searchWithFallback(query: string, typeFilter: string, flag } export function displayResults(results: SearchResult[]): void { + // Semantic-tier results are approximate; lead with a header so the user knows + // these are closest matches, not exact keyword hits. + if (results.some(r => r?.approximate)) { + console.log(` ${colors.dim('// no exact match — closest semantic matches:')}`); + console.log(); + } for (let i = 0; i < results.length; i++) { const r = results[i]; if (!r) continue; const typeLabel = RECORD_TYPES[r.type] ?? r.type; const addr = r.address ?? `${r.domain}/${r.id ?? r.record_id}`; const verified = r.verified ? colors.green(' ✓') : ''; + // Match-type tag ({hybrid}/{vector}/{keyword}) appears only on semantic hits. + const matchTag = r.match ? ` ${colors.dim(`{${r.match}}`)}` : ''; const num = colors.dim(`${i + 1}.`); - console.log(` ${num} ${colors.bold(r.name ?? r.id ?? '')} ${colors.dim(`[${typeLabel}]`)} ${colors.dim(`(${addr})`)}${verified}`); + console.log(` ${num} ${colors.bold(r.name ?? r.id ?? '')} ${colors.dim(`[${typeLabel}]`)} ${colors.dim(`(${addr})`)}${verified}${matchTag}`); if (r.description) { console.log(` ${colors.dim(r.description)}`); } @@ -431,7 +535,7 @@ export async function cmdSearch(positional: string[], flags: Record r.approximate === true); + console.log(JSON.stringify({ ...envelope, approximate }, null, 2)); return; } diff --git a/src/index.ts b/src/index.ts index d3c9c51..67ab283 100644 --- a/src/index.ts +++ b/src/index.ts @@ -107,6 +107,7 @@ ${colors.bold('OPTIONS')} --quiet, -q Suppress non-essential output (spinners + notes) --no-install Skip auto-install when resolving skill= records --no-color Disable ANSI color (also auto-off in non-TTY) + --no-semantic Disable the search semantic-search fallback (keyword-only) Flag names accept both kebab-case (--manifest-url) and camelCase (--manifestUrl). Use --key=value or --key value. Use -- to end option parsing. @@ -127,8 +128,9 @@ ${colors.bold('EXAMPLES')} npx agent-root resolve doma.xyz npx agent-root resolve doma.xyz/doma-protocol - ${colors.dim('# Search the public registry')} + ${colors.dim('# Search the public registry (semantic fallback on zero keyword hits)')} npx agent-root search doma + npx agent-root search "skill to pay in USDC" npx agent-root search "register domain" --type skill ${colors.dim('# Install a record')} diff --git a/tests/cli/parse-args-extra.test.ts b/tests/cli/parse-args-extra.test.ts index 9106f8d..75b2172 100644 --- a/tests/cli/parse-args-extra.test.ts +++ b/tests/cli/parse-args-extra.test.ts @@ -20,6 +20,11 @@ describe('parseArgs additional branches', () => { expect(r.flags['noColor']).toBe(true); }); + it('--no-semantic is treated as a positive boolean flag (noSemantic=true)', () => { + const r = parseArgs(['node', 'agentroot', 'search', 'usdc', '--no-semantic']); + expect(r.flags['noSemantic']).toBe(true); + }); + it('--no-json sets json=false (POSIX negation)', () => { const r = parseArgs(['node', 'agentroot', 'install', 'x.com', '--no-json']); expect(r.flags['json']).toBe(false); diff --git a/tests/commands/search.test.ts b/tests/commands/search.test.ts index ff0062d..fb8a503 100644 --- a/tests/commands/search.test.ts +++ b/tests/commands/search.test.ts @@ -1,5 +1,32 @@ -import { describe, expect, it } from 'vitest'; -import { clampLimit, clampPage, recordToSearchResult } from '../../src/commands/search'; +import { describe, expect, it, vi, beforeEach } from 'vitest'; +import { clampLimit, clampPage, recordToSearchResult, matchKind, semanticHitToSearchResult, searchSemantic, searchWithFallback, displayResults, cmdSearch, type SemanticSearchHit, type SemanticSearchResponse } from '../../src/commands/search'; + +const stripAnsi = (s: string): string => s.replace(/\[[0-9;]*m/g, ''); +import { fetchJSON } from '../../src/services/http/fetch'; + +// Mock the HTTP layer so searchSemantic tests don't hit the network. Preserve +// the module's other exports (fetch, postJSON) via importActual so unrelated +// importers don't break. +vi.mock('../../src/services/http/fetch', async (importActual) => { + const actual = await importActual(); + return { ...actual, fetchJSON: vi.fn() }; +}); + +const mockFetchJSON = vi.mocked(fetchJSON); + +const semHit = (over: Partial = {}): SemanticSearchHit => ({ + id: 1, + record_id: 'usdc-checkout', + domain: 'doma.xyz', + type: 'skill', + name: 'USDC Checkout', + description: 'Accept USDC payments via Coinbase Commerce', + manifest_url: null, + rrf_score: 0.0325, + bm25_rank: 1, + vector_rank: 2, + ...over, +}); describe('clampLimit', () => { it('returns the default for undefined input', () => { @@ -125,3 +152,158 @@ describe('recordToSearchResult', () => { expect(r.type).toBe('skill'); }); }); + +describe('matchKind', () => { + it('returns hybrid when both bm25 and vector ranks are present', () => { + expect(matchKind(semHit())).toBe('hybrid'); + }); + + it('returns vector when only vector_rank is present', () => { + expect(matchKind(semHit({ bm25_rank: null }))).toBe('vector'); + }); + + it('returns keyword when only bm25_rank is present (degraded / BM25-only)', () => { + expect(matchKind(semHit({ vector_rank: null }))).toBe('keyword'); + }); +}); + +describe('semanticHitToSearchResult', () => { + it('maps a hit to an approximate SearchResult with address + match tag', () => { + const r = semanticHitToSearchResult(semHit()); + expect(r.domain).toBe('doma.xyz'); + expect(r.type).toBe('skill'); + expect(r.id).toBe('usdc-checkout'); + expect(r.record_id).toBe('usdc-checkout'); + expect(r.name).toBe('USDC Checkout'); + expect(r.address).toBe('doma.xyz/usdc-checkout'); + expect(r.verified).toBe(true); + expect(r.approximate).toBe(true); + expect(r.match).toBe('hybrid'); + }); + + it('falls back to record_id when name is null', () => { + expect(semanticHitToSearchResult(semHit({ name: null })).name).toBe('usdc-checkout'); + }); +}); + +describe('searchSemantic', () => { + beforeEach(() => mockFetchJSON.mockReset()); + + const resp = (over: Partial = {}): SemanticSearchResponse => ({ + query: 'usdc', type_filter: null, total: 0, degraded: false, results: [], ...over, + }); + + it('maps a 200 response to approximate, match-tagged results', async () => { + mockFetchJSON.mockResolvedValueOnce(resp({ total: 1, results: [semHit()] })); + const out = await searchSemantic('usdc', '', {}); + expect(out).toHaveLength(1); + expect(out[0]?.approximate).toBe(true); + expect(out[0]?.match).toBe('hybrid'); + expect(out[0]?.address).toBe('doma.xyz/usdc-checkout'); + }); + + it('returns [] on a 429 (fetchJSON throws "HTTP 429 …")', async () => { + mockFetchJSON.mockRejectedValueOnce(new Error('HTTP 429 for https://www.agentroot.io/api/search?q=usdc')); + await expect(searchSemantic('usdc', '', {})).resolves.toEqual([]); + }); + + it('returns [] on network/5xx/timeout errors', async () => { + mockFetchJSON.mockRejectedValueOnce(new Error('Timeout after 30000ms fetching ...')); + await expect(searchSemantic('usdc', '', {})).resolves.toEqual([]); + }); + + it('returns [] when results is missing or not an array', async () => { + mockFetchJSON.mockResolvedValueOnce({ query: 'x', type_filter: null, total: 0, degraded: false } as unknown as SemanticSearchResponse); + await expect(searchSemantic('x', '', {})).resolves.toEqual([]); + }); + + it('forwards --type and caps limit at 50', async () => { + mockFetchJSON.mockResolvedValueOnce(resp({ type_filter: 'skill' })); + await searchSemantic('usdc', 'skill', { limit: '500' }); + const url = String(mockFetchJSON.mock.calls[0]?.[0]); + expect(url).toContain('/api/search?'); + expect(url).toContain('type=skill'); + expect(url).toContain('limit=50'); + }); +}); + +describe('searchWithFallback ordering', () => { + beforeEach(() => mockFetchJSON.mockReset()); + + function routeByUrl(map: { records?: unknown; search?: unknown; findSkills?: unknown }): void { + mockFetchJSON.mockImplementation(((url?: string) => { + const u = String(url ?? ''); + if (u.includes('/api/records')) return Promise.resolve(map.records ?? { records: [] }); + if (u.includes('/api/search')) return Promise.resolve(map.search ?? { results: [] }); + if (u.includes('/api/find-skills')) return Promise.resolve(map.findSkills ?? { skills: [] }); + // Manifest-probe + anything else: benign empty manifest (no records). + return Promise.resolve({ records: [] }); + }) as typeof fetchJSON); + } + + const calledSearch = (): boolean => + mockFetchJSON.mock.calls.some(c => String(c[0]).includes('/api/search')); + + it('does NOT call /api/search when /api/records has hits', async () => { + routeByUrl({ records: { records: [{ domain: 'a.io', record_id: 'r', type: 'mcp', name: 'R', status: 'active', manifest_status: 'active' }], total: 1, page: 1, pages: 1 } }); + const out = await searchWithFallback('r', '', {}); + expect(out).toHaveLength(1); + expect(out[0]?.approximate).toBeUndefined(); + expect(calledSearch()).toBe(false); + }); + + it('falls back to /api/search and returns approximate hits when records is empty', async () => { + routeByUrl({ records: { records: [], total: 0, page: 1, pages: 0 }, search: { query: 'usdc', type_filter: null, total: 1, degraded: false, results: [semHit()] } }); + const out = await searchWithFallback('usdc', '', {}); + expect(out).toHaveLength(1); + expect(out[0]?.approximate).toBe(true); + expect(out[0]?.match).toBe('hybrid'); + expect(calledSearch()).toBe(true); + }); + + it('skips the semantic tier when noSemantic is set', async () => { + routeByUrl({ records: { records: [], total: 0, page: 1, pages: 0 } }); + await searchWithFallback('usdc', '', { noSemantic: true }); + expect(calledSearch()).toBe(false); + }); +}); + +describe('displayResults approximate framing', () => { + it('prints a closest-semantic-matches header and a match tag for approximate results', () => { + const spy = vi.spyOn(console, 'log').mockImplementation(() => {}); + displayResults([semanticHitToSearchResult(semHit())]); + const out = stripAnsi(spy.mock.calls.map(c => c.join(' ')).join('\n')); + spy.mockRestore(); + expect(out).toContain('closest semantic matches'); + expect(out).toContain('{hybrid}'); + }); + + it('does NOT print the semantic header for normal keyword results', () => { + const spy = vi.spyOn(console, 'log').mockImplementation(() => {}); + displayResults([recordToSearchResult({ domain: 'a.io', record_id: 'r', type: 'mcp', name: 'R' })]); + const out = stripAnsi(spy.mock.calls.map(c => c.join(' ')).join('\n')); + spy.mockRestore(); + expect(out).not.toContain('closest semantic matches'); + expect(out).not.toContain('{'); + }); +}); + +describe('cmdSearch --json approximate flag', () => { + beforeEach(() => mockFetchJSON.mockReset()); + + it('includes approximate:true in JSON when results come from the semantic tier', async () => { + mockFetchJSON.mockImplementation(((url?: string) => { + const u = String(url ?? ''); + if (u.includes('/api/records')) return Promise.resolve({ records: [], total: 0, page: 1, pages: 0 }); + if (u.includes('/api/search')) return Promise.resolve({ query: 'usdc', type_filter: null, total: 1, degraded: false, results: [semHit()] }); + return Promise.resolve({ records: [] }); + }) as typeof fetchJSON); + const spy = vi.spyOn(console, 'log').mockImplementation(() => {}); + await cmdSearch(['usdc'], { json: true }); + const out = spy.mock.calls.map(c => String(c[0])).join('\n'); + spy.mockRestore(); + const parsed = JSON.parse(out); + expect(parsed.approximate).toBe(true); + expect(parsed.results[0].match).toBe('hybrid'); + }); +});