Repository navigation
Score sentences by likelihood and connect sentences within a note - #4
Merged
Merged
Conversation
Previously, generateSentence scored candidates only by how close their length was to the target. Now pick() also returns the log probability of the token it chose, these are summed over the walk, and the score adds the per-token average log probability (normalized by sentence length so it does not just penalize longer sentences). This is best-of-N reranking by model likelihood, applied to the existing multi-attempt generation loop rather than changing what can be generated: Li & Bertsekas (2024) frame this as a rollout policy over an n-gram's distribution.
generateNote used to generate every sentence after the first with no connection to what came before it, even though reply generation already seeds from a noun in the other person's note to stay on topic. Expose the token list alongside the generated text (generateSentenceWithTokens), and use the same noun-picking logic to seed sentence N+1 from a noun in sentence N, so multi-sentence notes read as one topic instead of unrelated sentences stitched together. Falls back to an unseeded sentence when no suitable noun is found, so this only adds a better default and does not remove any existing generation path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two follow-ups from #1, building on #2's Markov chain smoothing work.
generateSentencepreviously scored candidate sentences only by closeness to the target length.pick()now also returns the log probability of the token it chose; these are summed over the walk and folded into the score as a length-normalized average, so among several sampled attempts the one the model finds more probable is preferred, not just the one closest to the target length. This is best-of-N reranking by model likelihood applied to the existing multi-attempt loop (Li & Bertsekas, 2024, rollout algorithms for n-gram sequence generation).generateNoteused to generate every sentence after the first independently (seed: undefined), with no connection to the sentence(s) before it, even though reply generation already seeds from a noun in the other person's note to stay on topic. It now seeds sentence N+1 from a noun in sentence N the same way, so a multi-sentence note reads as one topic. Falls back to an unseeded sentence when no suitable noun is found.Neither change adds a new rejection rule, so the range of generatable sentences is not narrowed.
Test plan
npm run typecheck,npm run lint,npm run format:check,npm testall pass at every commit on this branchHI_MAX_SENTENCES=3) and confirmed a noun from the first sentence reappears in the next sentence, where previously sentences were unrelatedCloses #1