Skip to content

fix(search): show similarity, not fused rank, as match - #171

Merged
Paul-Kyle merged 2 commits into
phasespace-labs:mainfrom
chiruu12:fix/150-rank-not-confidence
Aug 31, 2026
Merged

fix(search): show similarity, not fused rank, as match#171
Paul-Kyle merged 2 commits into
phasespace-labs:mainfrom
chiruu12:fix/150-rank-not-confidence

Conversation

@chiruu12

Copy link
Copy Markdown
Contributor

Part of #150, taking the two Python surfaces. The OpenClaw and shared-core TypeScript renderers and the hook pair are not in this PR.

Presentation only, as you asked. score and raw_score on the REST payload are untouched.

The three cases are decided in one place, palinode/core/scoring.py, rather than once per surface. You flagged the duplicated jq as a wart you did not want extended; two Python renderers making the same judgement separately is the same wart in a different language, so MCP and the CLI now both call describe_match.

Result Before After
cosine 0.421, fused 1.0 100% match 42% match
raw_score: null (BM25-only) 100% match keyword match, rank 1.00
raw_score absent (pre-0.12) 100% match rank 1.00

I kept the null and absent cases apart, as you asked. A null means the ranker told us this hit had no vector arm, so the surface says which arm did fire. An absent field means the server never sent one and the arm is unknown, so the rank is all it says. Neither claims a similarity.

On the naming: rank, used everywhere. For the BM25-only case I went with a word rather than silence. keyword match says what actually matched, and dropping the number entirely loses the ordering a reader is using the flag to see. It reads as a match without ever reading as a similarity, which was the property you wanted.

Validation

  • pytest -q: 3263 passed, 10 skipped, 5 xfailed
  • ruff check: clean
  • The three surface tests fail on unfixed renderers and pass on the fix. Reverting just mcp.py and search.py gives [1.00] notes/a.md from the CLI and (100% match) from MCP.

The tests use the measured numbers from the issue: 0.421 as the median of the 32 searches, and each of the three points across the 0.402 to 0.459 range asserted to render under half.

Copilot AI lite review requested due to automatic review settings August 31, 2026 17:54

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@Paul-Kyle Paul-Kyle left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Merging. The shared helper is the right shape here: MCP and CLI make the same three-way decision once, numeric raw cosine is the only value presented as match percentage, present-null is labelled as keyword-only plus rank, and an absent legacy field is rank-only. I re-read the refreshed four-file diff at 5d591f3 after the branch update and all 9 required checks are green. This remains correctly partial, so #150 stays open for the TypeScript and hook renderers.

@Paul-Kyle
Paul-Kyle merged commit a96c5ef into phasespace-labs:main Aug 31, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants