Search relevance: learned ranking for Lunar search (spec 0084) - #2731
Open
glennjacobs wants to merge 20 commits into
Open
glennjacobs wants to merge 20 commits into
glennjacobs wants to merge 20 commits into
Conversation
…al baseline fixes (spec 0084) Adds lunar.search.pipelines.request / .results run by every built-in engine, page() and withParams() on AbstractEngine, meta on SearchHit and SearchResults, the Typesense vector distance threshold, Meilisearch hybrid and ranking score threshold config, skus_normalised on the product index, and typo tolerance disabled on exact-match fields by lunar:meilisearch:setup. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…stration Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…pec 0084) Logs searches and shopper events, scores them nightly into per-query product scores, and reranks results within buckets. Ships PartNumberRetrieval, WidenRequest and RankResults pipeline stages, swappable QueryNormaliser, Signal, Ranker and ScoreAggregator contracts, Blade and headless storefront tracking, basket and purchase attribution, score/replay/prune commands, and an Inertia panel add-on (section, settings, dashboard widget, product slot, global search source) with its compiled bundle. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…Search nav group last The add-on relies on the panel's compiled CSS, which has no xl:grid-cols-5 or md:grid-cols-2, so the KPI cards never left two columns on wide screens. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…shared with the Blade component Framework-agnostic sendSearchEvent/trackHit/attachSearchTracking plus a Vue useSearchTracking composable and v-lunar-search-hit directive, built to ESM and an IIFE the Blade tracking component inlines. SearchHit and SearchResults meta now carry TypeScript shapes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Event window and one-event-per-type dedupe on the recorder (unique index on search_events), per-session dedupe and trusted-sessions-only in both aggregators, crawler user agents neither logged nor ranked, purchase events forgotten on order cancel or refund, and staff overrides (exclude a product from learning, reset a query) with panel controls. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…a worker RefreshDatabase migrates once per process with the first test case's providers, so a panel test after a plain one (or after core's cross-db tests) found its tables missing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…tabase's NOW() A database server in a non-UTC zone aged every event by its offset, skewing the decay. Also re-migrate when another suite migrated the worker first. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s called Passing an explicit page to Scout bypassed its ?page= resolution, so a storefront paginating through the query string was stuck on page one. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… page is read-only Store behaviour is configured in code throughout Lunar. Drops the search_relevance_settings table and Settings service; the panel page now shows the configured mode with its env var, weights, scoring schedule and last run, and retrieval versions. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
DataObjects, Jobs, Observers and Support folders as in core; models registered with the ModelManifest; composer type library; the split-repo close-pull-request workflow; injected config in the RecordEvent job. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…s table Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… add-on page Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…-on page Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
DataTable draws its own card, so wrapping it in a SideCard produced a double frame. Tables now sit under a Section-style heading block like the panel's own edit pages. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Also gives the zero-result and no-click tables their own messages; an empty no-click table is good news, not missing data. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r card The product edit page's content slot becomes a sidebar card with the top three queries and a link to a full report page under Search relevance. Products without search activity get no card. Query page rows link to the report. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…by Vue Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…e width DataTable neither scrolls nor floors its flexible column, so eight fixed tracks collapsed the product name under the relative bar. Narrower numeric columns, a minimum product width, and the last event folded under the name. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements spec 0084 (
specs/0084-search-relevance.md): a learned, engine-agnostic ranking layer for Lunar search, in two parts.lunarphp/searchhooks (Part 1)lunar.search.pipelines.request/.results, run by every built-in engine throughIlluminate\Pipeline\PipelinewithLunar\Search\Pipelines\SearchRequest/SearchResponsepassables.AbstractEngine::page(),getPage(),getPerPage(),withParams()(merged into the engine request last;nullremoves a key),pipeRequest(),pipeResults().SearchHit::$metaandSearchResults::$meta(defaults, non-breaking). Typesense fillsmeta['score']fromtext_match, Meilisearch from_rankingScore(now requested).distance_thresholdon the hybrid vector query (lunar.search.typesense.vector_distance_threshold, default 0.6), Meilisearch hybrid config (embedder,semantic_ratio,ranking_score_threshold),skus_normalisedon the product index,ScoutIndexer::getExactMatchFields()andlunar:meilisearch:setupdisabling typo tolerance on those fields.lunarphp/search-relevance(Part 2, new package)search_queries) and shopper events (search_events), scores them nightly intosearch_query_scores, and reranks results within buckets from the learned scores.PartNumberRetrieval,WidenRequest,RankResults; contractsQueryNormaliser,Signal,Ranker,ScoreAggregatorbound from config so hosts can swap any of them.@lunarphp/search-relevance(framework-agnostic client plus a VueuseSearchTrackingcomposable andv-lunar-search-hitdirective) for the Inertia/Vue storefront starter kit; the Blade component<x-lunar-search-relevance::tracking />inlines the same client's IIFE build, withlunar_search_attrs()for markup.POST /lunar/search/eventsfor anything else.SearchHit/SearchResultsmetaare typed for the generatedLunar.Searchtypes. Basket and purchase attribution through cart linemeta.lunar:search-relevance:score(scheduled daily),:replay(proves uplift before switching on),:prune(weekly).off/shadow(default) /on, set in config orLUNAR_SEARCH_RELEVANCE_MODElike every other store setting. The panel settings page is read-only.search_events), per-session dedupe and trusted-sessions-only in scoring, crawler user agents skipped, purchases forgotten on cancel/refund, and staff overrides (exclude a product from learning, reset a query) with panel controls. The bucketed ranker already caps the blast radius: nothing can enter the first bucket unless the engine put it there.packages/search-relevance/build.Docs PR: lunarphp/docs#52
Test plan
vendor/bin/pest --testsuite search --parallelvendor/bin/pest --testsuite search-relevance --parallelvendor/bin/pest --testsuite core --parallelvendor/bin/pest --testsuite panel --parallelvendor/bin/pest --group=cross-dbon mysql / pgsql (CI)vendor/bin/phpstan analysenpm run build --workspace @lunarphp/search-relevance-addonnpm test && npm run type-check && npm run build --workspace @lunarphp/search-relevance🤖 Generated with Claude Code