Retrieval & ranking quality: smarter context, provenance-aware centrality, richer search

[zzet]

Summary

A focused quality pass on how Gortex retrieves and ranks code. It sharpens the rerank pipeline, routes smart_context through that pipeline for the first time, hardens query expansion and zero-result recovery, adds finer-grained context compression, recovers exact embedding cosine, and introduces a dependency-closure context tool plus provenance-aware graph centrality.

29 commits, one self-contained feature each. No schema migrations — new behavior is additive and backward-compatible, with the higher-risk levers gated behind config/env flags.

What changed

Rerank pipeline (internal/search/rerank)

• Generated-file down-ranking — a sixth path-penalty class demotes generated files (*.pb.go, mock_*.go, *_pb2.py, *.g.dart, …), but only when a real same-named hand-written implementation exists, so a generated file that is the sole definition is never demoted into oblivion.
• Source-over-test signal — when a query surfaces both an implementation and its test, the implementation is lifted above the test. Batch-relative: it contributes nothing unless the matching test co-occurs, so it sharpens the tie that matters without shifting the rest of the result set.
• Edge-provenance attenuation — LSP enrichment materialises a dense layer of interface-dispatch / framework-wiring edges that inflate the apparent centrality of utility code. A shared graph.ProvenanceWeight attenuates the abundant LSP tier (and the weak text-matched tier) relative to the unambiguous ast_resolved baseline, applied in HITS, PageRank, and a new rerank signal. Uniform-weight-safe: a graph with no provenance stamped ranks identically to before.
• Continuous α tuning — the discrete query-class → α lookup is replaced by a continuous query-shape score, so a half-identifier query lands between the symbol and natural-language blends instead of jumping a whole tier. Falls back to the discrete table when α is unset, so every existing caller is unchanged.

smart_context composer (internal/mcp)

• Ranks its working set through the full rerank pipeline — churn, HITS, community, co-change, frecency, feedback, path-penalty, source-bias and provenance now influence smart_context ordering (previously it used a feedback-only re-sort over raw BM25).
• Always-on blast-radius block — inline callers grouped by file plus exact covering tests, and a no covering tests found warning when a symbol is untested.
• Adaptive seeds + project-scaled budget — seed count and token budget scale with graph size (only when the caller didn't pin them explicitly).
• Source-over-test embed ordering and polymorphic-sibling skeletonization — large interchangeable symbol families (an interface with many implementers) ship one detailed representative and stub the rest.

Query expansion & recovery (internal/mcp, internal/search)

• Operator-free keyword-soup detection — a bare keyword list is now detected and split into per-disjunct fetches; genuine NL queries stay classified as concept.
• Zero-result decomposition fallback — a compound/dotted/CamelCase query that returns nothing is decomposed into leaf symbol-name terms and retried.
• Vocabulary-anchored expansion (opt-in) — constrains LLM-expander terms to the repo's own symbol vocabulary.
• Concept-relatedness thesaurus — a weighted concept→concept layer on top of the equivalence classes (kept separate from the union-find so distinct concepts don't merge).
• Real docs retrieval channel + prose-tuned reranking — corpus:docs now has its own fetch (not just a post-filter) and a prose weight profile that suppresses code-structural signals for documentation nodes.

Context compression (internal/elide, internal/embedding)

• Per-glob fidelity tiers — full | compress | omit per glob pattern on read_file / get_editing_context (with real ** support), generalizing the binary compress_bodies.
• AST sub-chunking extended to Ruby, PHP, Kotlin, and Swift (node kinds verified against the live grammars).

Embeddings (internal/query, internal/embedding, internal/graph)

• Post-rerank pure-cosine refinement — recovers the exact cosine that the rank-based semantic signal discards, blended into the top-N. Strict no-op when vectors are absent.
• Launch-time embedding-model variant selection — choose a named local model via config/env; zero-value preserves the current default.

Graph traversal & cache (internal/query, internal/analysis, internal/graph)

• context_closure tool — given seed files/symbols, walks the transitive import/dependency closure and packs it under a token budget, ranked by graph distance or by seeded random-walk proximity.
• Community-filtered walks — constrain walk_graph to a detected community.
• Precomputed CSR adjacency snapshot + personalized PageRank — built once per analysis pass; powers the closure tool's proximity ranking.
• Content-addressed package-scoped bundle cache — serves unchanged packages' search bundles from cache, invalidated by edge-aware per-package fingerprints (conservative: a package whose fingerprint was never reported is never served).

Text search & tokenization (internal/search, internal/mcp)

• search_text regexp mode — runs the trigram backbone through a compiled pattern; bad patterns return a tool error, results carry the enclosing symbol.
• Optional sparse sub-word n-gram tokenization + a learned boundary table mined from symbol names at index time. Ships off behind an env flag (precision-sensitive, reindex-required); index and query paths share one emission stage so postings always match.

Defaults & compatibility

• Provenance attenuation and cosine refinement are on but provably no-op on graphs/queries without the relevant data.
• Continuous α falls back to the legacy discrete table when unset; an explicit query_class pin and keyword-soup keep their discrete handling.
• Sparse n-gram tokenization and vocabulary anchoring are off by default.
• No response field was added without also updating its compact-wire encoder and pack-root etag, so caching and GCX callers stay coherent.