Skip to content

Verify: project-scoped smart-search returns a short page #459

Closed

Description

@possiblyneal

Part of #448

Question

Does fork issue #444 reproduce, where stronger hits from other projects crowd a scoped page below limit? Does a fix raise Recall on the evals? Upstream: rohitg00#1668, rohitg00#1669.

Activity

  1. self-assigned this
    on Nov 18, 2026
  2. possiblyneal commented on Nov 18, 2026

    @possiblyneal
    OwnerAuthor

    #459 / fork #444: project-scoped smart-search returns a short page

    Verdict: APPLY (partial premise: fork already over-fetches max(limit*10, 100); defect remains because the window is still fixed).

    Repro: test/smart-search.test.ts > "scoped page when other projects crowd the over-fetch (#444)" > "returns this project's matches when 150 stronger ones belong elsewhere" (limit 5 -> window 100). Fails 1/1 on base (returns []). At the default limit 20 the window is 200, so the issue's literal 150-others case at default limit passes on base; it breaks at limit <= 10 or with > window others. The second test (cap respected, nothing in scope) passes on base and after the fix (guard against unbounded widening).

    Fix (src/functions/smart-search.ts): when a project/agent scope is active, the scoped set is smaller than limit and the search returned a full window, re-run searchFn with 4x the window, up to MAX_OVER_FETCH 1600. Widening keys on the pre-floor in-scope count, so a page shortened only by the relevance floor does not trigger a second search. Bar (tmp/verify/444-verify.log, branch verify/444-scoped-short-page): typecheck 0 errors; npm test 271 files, 3,118 passed, 1 skipped; build ok; eval:gate and eval:gate:rules pass; eval:suite --tier fast against base-fdc3d459: all 15 metrics unchanged (Δ 0.000 on coding-life, PrecisionMemBench, LongMemEval).

    Live store (copy): 103,152 Observations over ~190 projects; the largest projects hold ~10% each (devex, homelab, agentmemory-sqlite...), while many hold <1% (e.g. merganser 0.5%, undine 0.4%). A scoped query from a small project competes with 90%+ foreign rows for a 100-200 candidate window, so the crowd-out is realistic there. Not measured per query (needs the index/daemon).

    Class 2: YES. Changes which hits smart-search returns on scoped queries (adds hits below the old window) -> eval:suite flat, as expected (each bench corpus is one project, so no foreign rows crowd the window; the burst/latency note in the AC applies: the extra search runs only on short, full-window pages, cost one more BM25+vector pass).

    Not measured: the gain on a small project's scoped query in the live store (no bench holds a many-project corpus). An earlier run of these checks collided with a duplicate run in the same worktree and timed out 1–4 unrelated tests; this clean rerun replaces it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions