From 25cc198d7299c7765ec9a1063921aceae93d66c9 Mon Sep 17 00:00:00 2001 From: Iurii Rogulia Date: Fri, 18 Sep 2026 19:39:04 +0300 Subject: [PATCH] analyze: prove document offsets past a non-ASCII character MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Every html/markdown offset assertion added with .analyze was ASCII-only, so none of them discriminated a code-point document offset from a byte one through the span adapter — the place a byte-oriented runtime actually gets it wrong. Ruby's String#index is character-based, so this runtime was never at risk, but the five ports share one proof set. "café" in the first span puts the UTF-8 byte offset one ahead of the code-point offset, and the example now asserts both numbers. Test-only. 19 examples pass. Co-Authored-By: Claude Opus 5 --- spec/analyze_spec.rb | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/spec/analyze_spec.rb b/spec/analyze_spec.rb index eea896e..6c36815 100644 --- a/spec/analyze_spec.rb +++ b/spec/analyze_spec.rb @@ -133,13 +133,16 @@ def rule_ids(changes) end it "reports document offsets in a later span too" do - # Three spans, and the change is in the third: the two markers before it are the only code - # points in the joined array with no origin, so a doubled or dropped one shifts this offset - # and nothing in a one- or two-span document would notice. - input = "

one

two

Wait... three

" + # Three spans, a non-ASCII character before the change, and the change in the third span: + # the two markers are the only code points in the joined array with no origin, so a doubled + # or dropped one shifts this offset and nothing in a one- or two-span document would + # notice, and "café" puts the UTF-8 byte offset one ahead of the code-point one, so a byte + # offset leaking out of the span adapter cannot pass either. + input = "

café

two

Wait... three

" first = Polytypo.analyze(input, locale: "en-US", mode: "html").first expect(first.rule_id).to eq("ellipsis") expect(first.start).to eq(input.index("...")) + expect(input[0...input.index("...")].bytesize).to eq(first.start + 1) end it "reports document offsets in markdown mode too" do