Skip to content

Search relevance: learned ranking for Lunar search (spec 0084) - #2731

Open
glennjacobs wants to merge 20 commits into
2.xfrom
feat/search-relevance
Open

glennjacobs wants to merge 20 commits into
2.xfrom
feat/search-relevance

Conversation

@glennjacobs

@glennjacobs glennjacobs commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements spec 0084 (specs/0084-search-relevance.md): a learned, engine-agnostic ranking layer for Lunar search, in two parts.

lunarphp/search hooks (Part 1)

  • lunar.search.pipelines.request / .results, run by every built-in engine through Illuminate\Pipeline\Pipeline with Lunar\Search\Pipelines\SearchRequest / SearchResponse passables.
  • AbstractEngine::page(), getPage(), getPerPage(), withParams() (merged into the engine request last; null removes a key), pipeRequest(), pipeResults().
  • SearchHit::$meta and SearchResults::$meta (defaults, non-breaking). Typesense fills meta['score'] from text_match, Meilisearch from _rankingScore (now requested).
  • Retrieval baseline fixes from the prototype: Typesense distance_threshold on the hybrid vector query (lunar.search.typesense.vector_distance_threshold, default 0.6), Meilisearch hybrid config (embedder, semantic_ratio, ranking_score_threshold), skus_normalised on the product index, ScoutIndexer::getExactMatchFields() and lunar:meilisearch:setup disabling typo tolerance on those fields.

lunarphp/search-relevance (Part 2, new package)

  • Logs every search (search_queries) and shopper events (search_events), scores them nightly into search_query_scores, and reranks results within buckets from the learned scores.
  • Pipeline stages PartNumberRetrieval, WidenRequest, RankResults; contracts QueryNormaliser, Signal, Ranker, ScoreAggregator bound from config so hosts can swap any of them.
  • Storefront tracking: a new npm package @lunarphp/search-relevance (framework-agnostic client plus a Vue useSearchTracking composable and v-lunar-search-hit directive) for the Inertia/Vue storefront starter kit; the Blade component <x-lunar-search-relevance::tracking /> inlines the same client's IIFE build, with lunar_search_attrs() for markup. POST /lunar/search/events for anything else. SearchHit/SearchResults meta are typed for the generated Lunar.Search types. Basket and purchase attribution through cart line meta.
  • Commands lunar:search-relevance:score (scheduled daily), :replay (proves uplift before switching on), :prune (weekly).
  • Modes off / shadow (default) / on, set in config or LUNAR_SEARCH_RELEVANCE_MODE like every other store setting. The panel settings page is read-only.
  • Abuse guards: event window and one-event-per-type dedupe (unique index on search_events), per-session dedupe and trusted-sessions-only in scoring, crawler user agents skipped, purchases forgotten on cancel/refund, and staff overrides (exclude a product from learning, reset a query) with panel controls. The bucketed ranker already caps the blast radius: nothing can enter the first bucket unless the engine put it there.
  • Panel add-on: Search relevance section (KPIs, uplift, top / zero-result / no-click queries, per-query explainability with exclude and reset overrides), read-only settings screen, dashboard widget, product edit slot, global search source. Compiled bundle committed under packages/search-relevance/build.

Docs PR: lunarphp/docs#52

Test plan

  • vendor/bin/pest --testsuite search --parallel
  • vendor/bin/pest --testsuite search-relevance --parallel
  • vendor/bin/pest --testsuite core --parallel
  • vendor/bin/pest --testsuite panel --parallel
  • vendor/bin/pest --group=cross-db on mysql / pgsql (CI)
  • vendor/bin/phpstan analyse
  • npm run build --workspace @lunarphp/search-relevance-addon
  • npm test && npm run type-check && npm run build --workspace @lunarphp/search-relevance

🤖 Generated with Claude Code

glennjacobs and others added 3 commits September 14, 2026 15:46
…al baseline fixes (spec 0084)

Adds lunar.search.pipelines.request / .results run by every built-in engine,
page() and withParams() on AbstractEngine, meta on SearchHit and
SearchResults, the Typesense vector distance threshold, Meilisearch hybrid
and ranking score threshold config, skus_normalised on the product index,
and typo tolerance disabled on exact-match fields by lunar:meilisearch:setup.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…stration

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…pec 0084)

Logs searches and shopper events, scores them nightly into per-query product
scores, and reranks results within buckets. Ships PartNumberRetrieval,
WidenRequest and RankResults pipeline stages, swappable QueryNormaliser,
Signal, Ranker and ScoreAggregator contracts, Blade and headless storefront
tracking, basket and purchase attribution, score/replay/prune commands, and
an Inertia panel add-on (section, settings, dashboard widget, product slot,
global search source) with its compiled bundle.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
glennjacobs and others added 17 commits September 14, 2026 16:27
…Search nav group last

The add-on relies on the panel's compiled CSS, which has no xl:grid-cols-5
or md:grid-cols-2, so the KPI cards never left two columns on wide screens.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…shared with the Blade component

Framework-agnostic sendSearchEvent/trackHit/attachSearchTracking plus a Vue
useSearchTracking composable and v-lunar-search-hit directive, built to ESM
and an IIFE the Blade tracking component inlines. SearchHit and
SearchResults meta now carry TypeScript shapes.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Event window and one-event-per-type dedupe on the recorder (unique index
on search_events), per-session dedupe and trusted-sessions-only in both
aggregators, crawler user agents neither logged nor ranked, purchase events
forgotten on order cancel or refund, and staff overrides (exclude a product
from learning, reset a query) with panel controls.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…a worker

RefreshDatabase migrates once per process with the first test case's
providers, so a panel test after a plain one (or after core's cross-db
tests) found its tables missing.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tabase's NOW()

A database server in a non-UTC zone aged every event by its offset, skewing
the decay. Also re-migrate when another suite migrated the worker first.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s called

Passing an explicit page to Scout bypassed its ?page= resolution, so a
storefront paginating through the query string was stuck on page one.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… page is read-only

Store behaviour is configured in code throughout Lunar. Drops the
search_relevance_settings table and Settings service; the panel page now
shows the configured mode with its env var, weights, scoring schedule and
last run, and retrieval versions.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
DataObjects, Jobs, Observers and Support folders as in core; models
registered with the ModelManifest; composer type library; the split-repo
close-pull-request workflow; injected config in the RecordEvent job.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s table

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… add-on page

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…-on page

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
DataTable draws its own card, so wrapping it in a SideCard produced a
double frame. Tables now sit under a Section-style heading block like the
panel's own edit pages.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Also gives the zero-result and no-click tables their own messages; an
empty no-click table is good news, not missing data.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r card

The product edit page's content slot becomes a sidebar card with the top
three queries and a link to a full report page under Search relevance.
Products without search activity get no card. Query page rows link to
the report.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…by Vue

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e width

DataTable neither scrolls nor floors its flexible column, so eight fixed
tracks collapsed the product name under the relative bar. Narrower numeric
columns, a minimum product width, and the last event folded under the name.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Status: Todo

Development

Successfully merging this pull request may close these issues.

2 participants