Skip to content

nbsp: do not insert before the semicolon that ends a character reference - #3

Merged
rogulia merged 1 commit into
mainfrom
fix/character-reference-guard
Sep 18, 2026
Merged

rogulia merged 1 commit into
mainfrom
fix/character-reference-guard

Conversation

@rogulia

@rogulia rogulia commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Implements spec 1.3.0's nbsp.md §3.3 step 4 (polytypo/polytypo@aaa1b90, canonical issue #22). The same change landed in polytypo-js#12.

text mode has no markup concept, so fr — whose narrowBeforePunctuation lists ; — inserted U+202F before the ; that ends a character reference, and the reference stopped being one:

In Before After
Bonjour : oui Bonjour&#160 + U+202F + : oui unchanged
Tom & Jerry Tom &amp + U+202F + Jerry unchanged
Oui ; non Oui + U+202F + ; non same — the guard is not a blanket refusal of ;

The guard

Walk left from the ; over ASCII alphanumerics, optionally one #, and decline if an & closes the walk. Shape, not the HTML named-reference table: that table is thousands of entries which five runtimes would have to carry identically, and declining on &notaname; costs nothing. The walk is bounded at 32 code points, one past the longest named reference.

Verification

  • Five tests added, two of them pinning what the guard does not do.
  • 1126 tests pass locally. The markdown-mode cases need tree_sitter_markdown and the packaging tests need a build toolchain, neither of which this machine has — those are left to CI.
  • Every canonical fixture that runs here (1046 of 1073, the rest being the markdown ones above) passes with spec 1.3.0 locale data generated in, each one idempotent.

The vendored spec copy is untouched; it refreshes in the 1.3.0 release round.

polytypo/polytypo#22, spec 1.3.0 (nbsp.md 3.3 step 4). text mode has no markup
concept, so fr -- whose narrowBeforePunctuation lists ";" -- inserted U+202F
before the ";" that ends a reference: 'Bonjour : oui' and 'Tom & Jerry'
stopped being what they were. Only locales that put a space before ";" could
reach it, so the defect was invisible in English.

N1/N2 now walk left from a ";" over ASCII alphanumerics, optionally one "#",
and decline if an "&" closes the walk. Shape, not the named-reference table:
declining on '&notaname;' costs nothing, and no runtime carries thousands of
entries for it. The walk is bounded at 32, one past the longest named reference.

Five tests, two pinning what the guard does not do. Locally 1126 tests pass; the
markdown-mode cases and the packaging tests need dependencies this machine does
not have and are left to CI. All 1073 canonical fixtures that run here pass with
spec 1.3.0 locale data generated in, each idempotent.
@rogulia
rogulia merged commit 341f274 into main Sep 18, 2026
4 checks passed
@rogulia
rogulia deleted the fix/character-reference-guard branch September 18, 2026 11:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant