Skip to content

Record where each shown claim sat; score ratings against that slot - #83

Merged
vu1n merged 1 commit into
mainfrom
claude/project-thread-nsa4xg
Nov 7, 2026
Merged

vu1n merged 1 commit into
mainfrom
claude/project-thread-nsa4xg

Conversation

@vu1n

@vu1n vu1n commented Nov 7, 2026

Copy link
Copy Markdown
Owner

Requested by Vu · project thread

A used rating on the top claim of every briefing means less than one on position 9 of a recall. This change records the slot when a claim is shown and scores ratings against it (the "score centering" idea from Applied Compute's production-RL post).

  • recordUsage now takes the reply's sections in render order. Every shown row stores metadata: {position, section, rank}, with no migration because the column already exists. The local record_usage does the same.
  • scripts/usefulness.py is read-only, one SELECT. It reports the used rate per slot and each claim's lift, meaning its used rate minus the rate its slots get anyway.
  • Ranking and promotion are unchanged. Feedback started today, so there is nothing to center on yet.

Verification: worker npm test passes 376 tests, including a new one for slot metadata across recall, briefing and expand. tsc --noEmit is clean. Python unittest discover -s tests passes 351 tests. kypp.verify and the kypp.store self-test pass, and so does featuremap.py check. The hosted-worker map now lists the script and its test.


Generated by Claude Code

Every shown claim_usages row now keeps {position, section, rank} in the order the reply renders,
so a used rating can be read against what that slot gets anyway. scripts/usefulness.py reports
each claim's lift over its slots' used rate (read-only, one SELECT on D1).
@vu1n vu1n self-assigned this Nov 7, 2026
@vu1n
vu1n merged commit e031df7 into main Nov 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant