fix(chat): show normalized confidence in source citations, not raw score

[RaghavChamadiya]

The bug

The sources list under a chat answer rendered percentages far above 100%:

1 File: packages/server/...  1808%    2 auth.py  1475%    3 File: ...  1374%
4 File: ...  1348%           5 client.py  1279%           6 ...  1187%

Root cause

source-citations.tsx multiplied relevance_score by 100 as if it were always a 0-1 fraction:

{(source.score * 100).toFixed(0)}%

It is not. relevance_score carries the raw backend score, and its scale depends on which retriever answered. _retrieve_with_method tries semantic search and falls back to FTS: embedding hits are cosine (0-1), but the BM25 fallback is unbounded. A score of 18.08 rendered as "1808%".

The two chat surfaces already disagreed about this field, which is the tell: artifacts.tsx:270 renders the same relevance_score raw as score 18.08, while source-citations.tsx:181 treated it as a fraction.

The fix

search_codebase already derives confidence_score alongside it:

# tool_search.py, _assign_confidence
item[target_key] = round(raw / max_score, 2) if max_score > 0 else 0.0

That is bounded 0-1 whichever retriever produced the hits, which is exactly what a percentage badge needs. This reads confidence_score instead, and renames the field score -> confidence so a value that is not a raw score stops being called one.

The ?? r.score fallback is dropped rather than kept: that field is unbounded too, so it would reintroduce the bug whenever confidence_score is absent. The badge now hides in that case.

Only the search_codebase branch of extractSources ever set this field, so the blast radius is one file. Nothing outside source-citations.tsx reads it.

Verified

Ranking is unaffected: results are sorted server-side and confidence is monotonic in the raw score. Against a live index, the same ten sources now read:

100%  82%  76%  75%  71%  66%  65%  63%  55%  53%

Each matches raw / max_score exactly (1475/1808 = 82%, 949/1808 = 53%), and all ten badges still render, so nothing dropped.

Tests

Adds __tests__/chat/source-citations.test.tsx. The component had no coverage, which is how this shipped; the existing source-citation.test.tsx covers the unrelated singular component. The new test pins a BM25-scale payload (relevance_score: 18.08) and asserts the badge stays within 0-100%.

[repowise-bot]

✅ Health: 7.6 (unchanged)

📋 At a glance
1 hotspot touched · 1 new finding introduced. Scoped to packages.

🔎 More signals (1)

🔥 Hotspot touched (1)

• .../chat/source-citations.tsx — 4 commits/90d, 3 dependents · primary owner: Raghav Chamadiya (92%)

---

📊 Full report · ⭐ Star Repowise · 📥 Install bot · Last updated 2026-07-15 08:07 UTC
_{Silence on a single PR with [skip repowise] in the title · Per-repo toggle on repowise.dev/settings?tab=bot}