Repository navigation
Front-load findings in the README, and rewrite the scope line as prose - #35
Merged
Merged
Conversation
The README opened with status and corpus composition, so a reader met the method before any result. It now opens with one line on what the benchmark measures, then a Findings section carrying the four numbers that matter, each with what it means for a system acting on the user's behalf, then the live link, then status and scope. Everything that followed keeps its existing order. Findings, all dev split at IoU 0.5 and all reproduced from pnpm eval before writing: F1 50.5% hosted-large against 25.7% local, with recall falling 49.5% to 16.8% while precision holds at 51.5% and 54.5% deadline phrase located 72.7%, resolved to a date 8.6% superseded reported open 2 of 14 matched pairs, 14.3% evidence grounding 103/103 and 33/33, both 100% The corpus composition tables move into a Corpus composition disclosure block. Every number in them is kept. The separability and limitations sections stay full and visible: they are what makes the rest credible, and collapsing them would hide the parts a reader should weigh most. One figure was wrong and is corrected: 25 threads carry three or four loops each, not three to five. The loops-per-thread histogram tops out at four. The scope line is now four plain sentences with no semicolons and no dashes, replacing the semicolon-chained version. It lives in viewer-results.json, so the single edit covers all three viewer pages, and the README carries the same wording. pnpm validate passes: 200 threads, 273 loops, 510 spans all resolving. pnpm check passes with 227 tests.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two edits: the scope line becomes plain sentences everywhere it appears, and the README is reordered so a reader meets the results before the method.
Scope line
The semicolon-chained version is replaced with four plain sentences, no semicolons and no dashes:
All three viewer pages read
results.scopefromresults/viewer-results.json, so one edit covers/,/corpusand/failures. Confirmed in the built output for all three. The README carries the same wording.README reordering
New order: one line on what the benchmark measures, then Findings, then the live link, then status and scope, then everything that was already there in its existing order.
The four findings, each stated as a number with what it means for a system acting on the user's behalf:
Corpus composition tables move into a
<details>block titled "Corpus composition". Every number is kept, just behind a disclosure.The separability and limitations sections are untouched and stay fully visible. They are 82 and 66 lines respectively, and they are what makes the rest of the document credible. Collapsing or trimming them would hide exactly the parts a reader should weigh most heavily.
Verification
Every figure was re-derived from live
pnpm evalandpnpm statsoutput before writing, not copied from the previous README.One figure was wrong and is corrected: the README claimed 25 threads carry "three to five" loops each. The count of 25 is right, but the loops-per-thread histogram is
{0: 40, 1: 77, 2: 58, 3: 20, 4: 5}, so the maximum is four. It now reads "three or four loops each".pnpm validate: 200 threads, 273 loops, 510 spans, all resolve. PASSpnpm check: 227 tests passing.🤖 Generated with Claude Code