fix(core): pin the parser's line-ending and blank-line boundaries - #40
Merged
Conversation
parseTasks and the Kotlin TaskParser are one rule written in two languages, and each left two boundary decisions to its runtime, where the two disagree. Java's regex `$` also matches before a trailing line terminator while JavaScript's does not, so a tasks.md ending a line in a bare CR reported a different total in IntelliJ than in Web and VS Code (#33). Separately, `isBlank()` and `trim() === ""` cover different whitespace, so an NBSP-only line moved the continuation boundary on one side only. Measured against the reference renderer, neither side was right. It counts a lone CR as a line ending, which both parsers dropped, and counts only spaces and tabs as blank, where each runtime errs in one direction. So both boundaries are now stated rather than inherited: all three CommonMark line endings are normalised before the split, and an explicit [ \t] predicate does the blank test and the trims on both sides. Kotlin's CHECKBOX_RE and SECTION_RE move to \z, which also closes U+2028/U+2029. U+0085 stays divergent, pinned by a test on each side rather than fixed: closing it needs an identical negated class in both engines' `(.+)`, which is not worth the cost in the parser's two most load-bearing patterns for a character with no natural source in a task list. Case-mirrored tests cannot catch this class of drift by construction, so the spelling is the control. A shared-fixture corpus that could catch it structurally is left as follow-up. Closes #33 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01HjHibWXs9MTLcVmewMXDrL
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #33.
The problem
parseTasksand the KotlinTaskParserare one rule written in two languages, and each left twoboundary decisions to its runtime — where the two runtimes disagree, and case-mirrored unit tests
cannot notice, because both sides spell the rule the same way.
Line endings. Java's regex
$also matches before a trailing line terminator; JavaScript's doesnot. A
tasks.mdleaving a bare\rat the end of a line therefore reported a differenttotalinIntelliJ than in Web and VS Code.
Blank lines.
isBlank()andtrim() === ""cover different whitespace, so an NBSP-only linemoved the continuation boundary on one side only. This one was found while verifying the first.
Neither side was right
Measured against the reference renderer (
react-markdown+remark-gfm, the same reference theexisting folding tests compare to):
- [x] a\r- [x] a\r\r\n- [x] b- [x] a\n- [x] b\r- [x] a\r- [x] bCommonMark counts a lone
\ras a line ending and counts only spaces and tabs as blank. AligningKotlin down to TypeScript's answer — the fix the issue suggests — would have settled both on the
wrong result and left the last row of the first table broken on both sides.
What changed
\nbefore the split, on both sides.[ \t]predicate does the blank test and the trims on both sides, replacingtrim() === ""/isBlank()at four sites each.CHECKBOX_REandSECTION_REmove to\z, matching whatBLOCK_OPENER_REalready does.Not cosmetic: Java's
$also admits U+2028 / U+2029, which\zcloses.characters.
U+0085 stays divergent and is pinned by a test on each side rather than fixed — it is an ordinary
character to JS's
.and a terminator to Java's, and closing it needs an identical negated class inboth engines'
(.+), which is not worth the cost for a character with no natural source in a tasklist.
Verification
Each new test was checked against a temporarily reverted implementation, since a test that passes on
the old code proves nothing: 5 turn red on the TypeScript side, 4 on the Kotlin side. The two sets
differ because each guards the direction the other runtime got wrong; the rest are regression
guards and the documented exception.
Gates:
type-check,lint,npm test(207 + 24 + 60),./gradlew test(40) — all green.Note for whoever cuts the release
@spekjs/corechanges observable output for files with CR line endings or exotic whitespace (nothingin this repo — verified by scanning every
.mdfor those characters). That warrants a minor bump,not a patch.
🤖 Generated with Claude Code
https://claude.ai/code/session_01HjHibWXs9MTLcVmewMXDrL