Skip to content

fix CJK text width estimation#19

Open
reallav0 wants to merge 4 commits into
tt-a1i:mainfrom
reallav0:agent/fix-cjk-text-width
Open

fix CJK text width estimation#19
reallav0 wants to merge 4 commits into
tt-a1i:mainfrom
reallav0:agent/fix-cjk-text-width

Conversation

@reallav0

Copy link
Copy Markdown

Summary

  • replace the incomplete hard-coded width ranges with Unicode script-aware CJK detection
  • cover supplementary Kana, Tangut, Nushu, Khitan, and vertical CJK punctuation
  • preserve single-width handling for halfwidth Katakana and Hangul forms
  • add pure CJK, mixed ASCII/CJK, punctuation, and supplementary-plane regression tests
  • rebuild and smoke-test the distributed archify.zip

Root cause

The previous classifier covered common BMP Han ranges and supplementary unified ideographs, but skipped several supplementary East Asian scripts and U+FE10–U+FE19 vertical punctuation. Those glyphs were counted as one unit, which could underestimate label and collision-box widths.

Validation

  • npm run check:validators
  • node --test test/*.test.mjs — 111 tests passed
  • packaged archify --help, archify doctor, and architecture validation smoke checks

Closes #14

@tt-a1i

tt-a1i commented Jul 13, 2026

Copy link
Copy Markdown
Owner

Thanks for the PR — looked at this carefully against #14.

For the Chinese labels called out in that issue (写入, 审批通过, 注入提示词, mixed ASCII/CJK), textUnits already returns the same widths on main as with this change. So this doesn't appear to fix the reported overlap symptom.

What it does change is broader Unicode coverage (vertical punctuation, Tangut/Nushu/Khitan, etc.). That also shifts width for ~tens of thousands of code points, including some that shouldn't be double-width in our monospace stack (e.g. middle dot · U+00B7 goes 1 → 2).

So I'm going to hold off on merging this as a #14 fix. If you want to continue:

  1. Please drop Closes #14 (or retarget) — this shouldn't close that issue.
  2. If the goal is still [P2] CJK text width estimation may be inaccurate #14, a repro diagram where labels overflow under current textUnits would help a lot more than expanding script coverage.
  3. If the goal is supplementary-script / vertical-punctuation coverage on its own, please keep the diff narrowly scoped and avoid over-counting non-fullwidth characters.

Happy to take another look with either a repro or a tightened version.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[P2] CJK text width estimation may be inaccurate

2 participants