Developed v2.4.0 - #10
Merged
Merged
Conversation
:HexPairDiffShow read the cursor with s:PagedByteOffset(), which knows a dump: three columns per byte, one line of bytes per line. The windowed text view has one column per byte and lines as long as the bytes between two 0x0a make them, so in that view the offset came out short by however much of the page was above the cursor - and the command then reported a real byte of the file, with a real comparison against the other file, from the wrong place. Confidently wrong is worse than an error here: nothing on screen says the answer is about a byte 400 back. s:Here() is the same question the markings, the search and the diff jumps already ask, and it dispatches on the view. The test asks it from the text view and pins the byte; without the fix it answers about byte 1062 instead of 1501, which is what a plausible wrong answer looks like. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The modified-byte marking answers "which bytes did I change" and covers the NEW byte, which means the old one is exactly what the screen no longer shows. There was no way to ask for it: :HexPairDiffShow does this against ANOTHER file and deliberately reports the page as it was read, leaving unwritten edits out of the comparison, so it cannot be turned on the view's own file. :HexPairModifiedShow is that question the other way round - the buffer as it stands now against the page as it was read from disk - in the same shape, down to the two aligned rows and the 32-byte cap, so a reader who knows one knows the other. <Leader>d, the lowercase of <Leader>D. Three things it has to get right that the shape alone does not give: - Bytes an INSERT added are past the end of the page as read and have nothing behind them. Dashes and a heading that says where the page as read stopped, never a plausible 00. - The live side of the comparison is new work, and the hex view's half of it is a whole-page scan; s:ModifiedRuns() was already doing that scan and throwing the bytes away. One s:LiveHex(), cached against b:changedtick, now serves both. - A page that no longer scans has no byte to report AND no way to number the one under the cursor - counting the bytes above it is that same scan, and s:PagedScan()'s error result has no 'bytes' key, so asking first is what turns an E716 into the message naming the character. The text view answers it too, and inherits that view's one blind spot: a Vim string holds no NUL, so a NUL and a line break read alike there. Said in the help rather than worked around, as the markings say it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
:HexPairUnhex refuses over unwritten edits to the page, and :HexPairUnhex! discards them - which is exactly the relation every other pair in hexpair.vimrc already spells lowercase/capital: j/J and k/K for a page turn, g/G for the page prompt, b/B for the byte prompt, mg/mG for a mark. Unhex was the one force variant with no key at all, and it was sitting on the capital while the lowercase went unused, so the file's own rule read backwards precisely where it mattered most - discarding a page's edits is the most destructive thing any of these keys do. <Leader>u is now the bare command and <Leader>U the bang, mapped directly, as the file's rule for a force variant with no <Plug> target says. Anyone with <Leader>U in their fingers gets the discarding one, so the changelog says so in as many words. The plugin is untouched: it defines no mappings, :HexPairUnhex[!] is what it always was, and a key already mapped is still left alone. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
README.md is 1300 lines, and the first thing in it after the badges was prose. Someone deciding whether this plugin is worth installing, and someone who installed it last month and wants the key for "go to byte", both need the same page and neither was going to find it: the mappings are quoted a third of the way down inside the installation section, the commands are spread over five more. A TL;DR: Quick Start now opens the file - the tarball, the two lines that source hexpair.vimrc and hexpair.bashrc, and then every key it defines plus every command that has no key, grouped and one per line. Commands that DO have a key are not repeated there; the key is the answer. Complete is the whole value of such a list, and a hand-kept copy of another file's contents is exactly the thing that rots. So it is checked: the suite extracts the keys from hexpair.vimrc and from the quick-start block and requires the two sets to be equal, in both directions, plus a count so that a block which stops being extracted at all cannot pass by being as empty as the other side. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two invariants and three bug-table rows, all found while adding :HexPairModifiedShow and none of them derivable from the code without walking into them: - s:Here(), never s:PagedByteOffset(), in anything both views can run. That is what the DiffShow fix was, and the next command to ask "which byte am I on" would have made the same choice for the same reason. - s:LiveHex() is the live side of every comparison and b:hexpair_page_hex the read side; an empty live answer means "cannot be told", not "no bytes", and it has to be asked BEFORE anything numbers a byte on the page - because numbering one runs the same scan that just failed. - And the E716 that comes of not doing so, which is still open in :HexPairPages and :HexPairInspect, with the reproduction, the two places it is NOT reachable and why, and why the fix is not one line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The quick-start list is for using, not for reading: :HPReplaceAllInPage is what anyone actually types, and spelling it :HexPairReplaceAllInPage there costs eight characters on the one screen meant to save them. The long names stay the documented ones - :help and the rest of the README are unchanged - and the block says so, along with the one setting that takes the short ones away. Which makes the list a hand-kept copy of a second thing, so it is checked the same way the keys are: a Vim with the plugin sourced is asked whether each name it types is a command, rather than the alias table being read out of the source. That is also what g:hexpair_short_commands decides, so asking the loaded plugin is asking the right question. A count with it, since an extraction that stops matching finds nothing undefined. (The generated script keeps one statement to a line: -u NONE starts in 'compatible', where a backslash continuation is a truncated statement - which is how this first came back with an empty output file and no error.) Also: `vimhex` and `vimhexdiff` with no arguments print their usage, so the shell block shows that before it shows a real call. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Make outline of the README more clear and structured. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Restructuring the README moved the Windows material to the end of Usage and renamed its three headings. Six pointers elsewhere still named the old ones, and one of them was a link that no longer resolved: - README's own note about xxd past 2 GiB linked #windows-and-the-2-gib-limit, which GitHub now silently scrolls nowhere. It is #2-gib-limit-on-windows. - vimhex.cmd, gvimhex.cmd, gvimhexdiff.cmd and vimhexdiff.cmd each tell the reader to look up "Windows: vimhex and vimhexdiff outside Vim". These ship in the tarball, so a stale name there is a stale name on a user's disk. (Edited byte-wise: they are CRLF, pinned by .gitattributes, and a batch file with LF endings breaks `goto`.) - doc/hexpair.txt quoted the same heading for the registry entries. - Two changelog entries name the section holding the table of what runs where past 2 GiB. The entries themselves are history and stay as they are; only the pointer into a living document is brought up to date. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Typos and half-sentences in the block that is now the first thing anyone reads: - "intall" in the tar line, and a comment claiming the .bashrc line loads a "*hexpair* shell function" - it loads two, and they are called vimhex and vimhexdiff, which is what the next lines then type. - "show different to the original state" and "show different to the file are are comparing to" are not sentences. Both are now what the key answers, in the fewest words that say it, which also brings the two longest lines back inside the block's width. - "... but previous one", twice, where "the" was meant. - The mark keys read as three comma-spliced clauses; a colon after the verb makes each one a phrase. And one line back, so that a screen full of :HP... names does not read as a second command set: they are the short names of the :HexPair... ones the rest of the page uses. The long explanation stays where it was, under the command table. Also two of my own from the commits before: ":HexPairDiffShow below asked of" needed a comma to parse, and a paragraph began "and" after a full stop. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
:HexPairPageGoto, :HexPairGoOffset and :HexPairOpen's [page] have taken $-N since the jumps were added - N back from the end, which is the one thing a bare -N cannot say, since that counts from wherever the view or the cursor happens to be. Every enumeration of the forms in README.md stopped at $, in five places: two command tables, the mapping list, the prompt description under the Pages table, and the quick start. The byte prompt was worse off than the page one: <Leader>b was shown as "1-based (0x... ok)" and nothing else, so neither the steps nor either $ form was anywhere near it. The help had it right for both commands and wrong in one place, the paragraph describing what the two prompts accept. Checked against the plugin rather than against the parser's comments: $, $-N and $-0xN resolve on a 5000-byte fixture through :HexPairOpen, :HexPairPageGoto and :HexPairGoOffset, on the Ex command line as well as through the function form. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Between the quick start and the features, a clickable outline of the file's own sections, nested as the headings nest. The quick start answers "how do I use this"; the outline answers "what else is in here", which until now took scrolling to find out. It is a hand-kept copy of the headings, so it is checked like the quick-start lists beside it: the headings and the outline's links must be the same set, in the same order, with a count to catch an extraction that stops matching. Both directions matter and for different reasons - a new section missing from the outline is invisible, while a link left behind by a rename is a link that scrolls nowhere, which is what the renamed Windows sections had just done to the note at the top of the file. The outline's own heading is excluded, being a link to the list you are already reading. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
README.md's Pages section and :help hexpair-paged both said that only a
write which SHORTENS a file and :w {file} need Vim 9.0.0795 - and then,
in the same breath, listed inserts among the things that run on the 8.0
floor. An insert with more than half the file after it does not: moving
that much of a tail costs more than rewriting the file, so it takes the
splice, and the splice is precisely what readblob()'s offset argument is
for. HexPairPagedGateMessage() has always named the boundary correctly
("an edit that keeps the page's length, or that inserts bytes with no
more than half the file after them, does not need it"), and the README's
own Requirements section has always said three.
The help had the tell in it: two sentences after naming two operations it
says "the checks for the three operations above". Both places now name
the same three the gate does.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hexpair.bashrc has defined four functions since the gVim pair was added - vimhex, vimhexdiff, and the two that are those again with VIMHEX_VIM defaulting to gvim - and said so in exactly one place: its own comments. The README named gvimhex only as gvimhex.cmd, under Windows; :help hexpair-vimhex stopped at vimhexdiff. So on Linux, macOS, the BSDs and in WSL the only way to learn you had them was to read the file you had already sourced and forgotten about. Both now list all four, with the one thing that is not obvious from the name - a VIMHEX_VIM you set yourself is left alone, so pointing it at a particular gVim still works - and the two pairs point at each other across the platform split. :help hexpair-vimhex-gui is the new section. And the thing that made the omission visible: `gvimhex` with no arguments answered "usage: vimhex FILE|- [PAGE|@byte]", naming a command the user had not typed, because the argument check lives in the function gvimhex delegates to. One helper reads FUNCNAME for the caller, so each of the four says its own name and the grammar stays defined once. Only our two delegating names are taken from it - a user's own function calling vimhex gets "vimhex", which is what they typed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A file-wide scan - :HexPairFind, :HexPairDiffNext - reads the file as flat hex, and flat means taking back out the line breaks xxd -p put in. The default wrap for -p is 30 bytes, so a megabyte of file comes back with some 35 000 breaks in it and the strip that removes them is a measurable part of every block read. -c 256 is the widest line xxd will portably agree to (s:bytesperlinemax already records why: xxd.c's `#define COLS 256`). Newer versions accept more in -p mode, and -c 0 for no wrapping at all; neither can be assumed of the xxd that ships with the Vim this plugin supports. Measured on a 256 MiB file, scanned end to end with the 1 MiB block: 11.4 s -> 10.3 s, and the strip itself over an 8 MiB block 76 ms -> 43 ms. Dropping the breaks entirely (-c 0) would take that to 3 ms, which is another 40 ms against a read that costs 170 ms - not worth a probe. The two callers shared the strip by copy; it is one function now, so the comment explaining why it is two substitute() passes rather than one lives in a single place. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
:HexPairFind and the comparison behind :HexPairDiffNext read the file a
block at a time, and the block was a fixed megabyte. That is one xxd
process - some 8 ms of starting it - for every megabyte of a file that
paging exists to let be terabytes, and on a large one it is a fifth of
what the scan costs.
g:hexpair_scan_block now says how big that block is, between 1 MiB and
1 GiB, and the default is 8 MiB rather than 1. Measured on a 256 MiB
file scanned end to end, block against wall time and peak RSS:
1 MiB 10.3 s 19 MB 16 MiB 8.1 s 142 MB
4 MiB 8.4 s 44 MB 64 MiB 7.9 s 536 MB
8 MiB 8.4 s 77 MB 128 MiB 8.2 s 1036 MB
8 MiB is the knee: it is where the per-process cost has essentially gone
and where the trade stops paying, since what a scan actually spends is
xxd turning bytes into hex (some 64 MB/s) and Vim reading, stripping and
matching two characters for every byte - none of which cares how the
file is cut up. 64 MiB is seven times the memory of 8 MiB for another
half a percent of the time, so it is a legal setting rather than a
recommended one, and 77 MB is still a number a plugin may spend on a
scan without asking.
The limits are cost boundaries and say so, unlike s:pagesizemax, which
is a correctness one: a block goes to xxd's -l exactly as a page does
and 1 GiB is nowhere near the 32-bit number that takes. Below 1 MiB a
scan spends its time starting processes; above 1 GiB a comparison, which
holds a block of each file as hex, would want some 16 GiB of Vim.
Read at the start of each scan rather than baked into the buffer the way
g:hexpair_page_size is, because nothing bakes it: it can be changed
between two presses of the same key and has to be believed when it is.
Once per scan, into a local, so a value changed mid-scan cannot move a
seam under the loop walking them; and s:FindFrom() now reports a bad one
the way s:DiffJump() already did.
The forward scan steps on by the block less the pattern's overlap, so a
block no longer than the pattern would step by nothing and read the same
bytes for ever. Neither end of that is reachable - the floor is a
megabyte and no pattern typed at a : prompt comes near it - but the
block is a setting now and the pattern is input, so the scan takes the
longer of the two rather than resting on both.
The suite gets the first fixture in it with a scan seam inside: at the
smallest block allowed, that is a megabyte in, with the needle lying
across it and a change six bytes past it. Both block sizes are asked the
same questions - the minimum, where the seam is real, and the default,
where the file is one block - and must answer alike.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four places described the scan as reading "a megabyte at a time", which was the whole of the truth while the number was fixed in the source and is now neither the default nor a fact about the reader. The README, the suite's own comment and CLAUDE.md name the setting instead. CLAUDE.md also gains what a future session would otherwise re-measure: the block-against-time-and-memory table, why 8 MiB is the knee, that its limits are cost boundaries rather than the correctness one s:pagesizemax is, why it is read once per scan, and why the forward scan takes the longer of the block and the pattern. And the -c 256 the two flat-hex readers now ask for, next to the strip it is there to make cheaper. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
:HexPairDiffNext walked both files as HEX: a block of each through xxd,
into a pipe, with its line breaks taken back out, and then two strings of
twice the data compared. Everything about that is the shape xxd's answer
comes in, and none of it is what the question needs - "are these two
ranges the same, and where do they part company" is a memcmp.
readblob() with an offset and a size makes the range readable without
xxd at all. That is patch 9.0.0795 with +num64, which this plugin
already requires for the splice, so the predicate exists; it gets a
second name here (HexPairPagedBlobRangeSupported) because a scan is not
a splice and the reason to want the patch is a different one.
Measured on a 256 MiB pair differing at byte 200000001:
:HexPairDiffNext 12.41 s -> 0.28 s RSS 113 MB -> 47 MB
Per 8 MiB block: two xxd reads and their strips cost 418 ms and the
comparison 53 ms, against 6 ms of readblob() and 3 ms of blob equality.
The four questions a comparison asks are answered twice over - the hex
primitives stay exactly as they were, and the Blob ones are new beside
them rather than one pair taught both forms. That is deliberate: the two
answer in different units, a hex index being a nibble of which half are
the wrong half, and merging them would put that trap back where this
whole area exists to keep it out. Instead four dispatchers pick the
family and normalise to BYTES, which is what every caller wanted anyway
- the `idx / 2` that used to sit at each call site is gone, along with
the -1 it would have turned into a 0.
A Vim without the patch keeps the xxd reader untouched. The suite holds
the two families against each other over 4096 pseudo-random bytes,
spoiled at the front, in the middle, at the end and not at all, each
also truncated, and requires them to answer alike: which reader a Vim
uses is decided by a has(), so what has to be true is that it cannot
matter. CI runs the whole suite against a Vim 8.0.0000 built from
source, which is what exercises the fallback end to end.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Blob reader and the xxd one answer the same four questions and a has() decides which a Vim uses, so the property that has to hold is that it cannot matter: a jump lands on the same byte either way. Four fixed cases each, and then both asked about 4096 bytes from a seeded generator - spoiled in the middle, at the very front, at the very end and not at all, each also truncated so a shorter side is covered - with any disagreement reported as the pair of answers that differed. Behind HexPairPagedBlobRangeSupported(), because a Blob literal is Vim 8.1.0735 and CI runs this suite against 8.0.0000: a legacy :if that is false does not evaluate what is inside it, and a function body is not parsed until it is called, which is why the walk lives in functions in a file of its own. That file carries no backslash continuation lines, and says so. The suite runs Vim WITHOUT -N, so 'compatible' is on and 'cpoptions' carries C, which makes a leading backslash an ordinary character again: the plugin gets away with continuations because it sets cpo&vim on the way in, and a file sourced after it has put cpo back does not. The symptom was E697 inside a function that had tested fine on its own. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
:HexPairFind read every block of the file as hex through xxd and matched
it with a regexp. With readblob() the block can be had as raw bytes for
a fortieth of that, which the comparison already takes advantage of -
but a search cannot simply follow, because Vim has no "find these bytes
in a Blob". What it has is index(), which finds ONE byte value.
So: walk every occurrence of one byte of the pattern and check the rest
by hand. That is a loop with a builtin call per candidate, which legacy
script is at its worst in and a compiled :def is at its best in - 45 ms
against 210 ms over an 8 MiB block of random data, where reading the
same block as hex and matching it costs 262 ms. Doing it in legacy would
have bought 20%; this buys five times, and it is the whole reason the
plugin now has a Vim9 file at all.
:HexPairFind over 256 MiB 9.10 s -> 2.02 s RSS 97 MB -> 23 MB
WHICH byte gets walked decides everything, because the walk costs one
step per occurrence of it. Anchor() samples 1024 bytes of the block and
picks the pattern byte that looks rarest there - at an ODD stride,
because binary files are full of powers of two and an even one would
look at the same column of every record. The case that makes the point
is a pattern of common bytes with one rare one in it:
"00 00 00 00 01" over 64 MiB of zeros 13.20 s -> 0.30 s
and the same walk anchored on the 00 instead would have been 6.2 s for
one 8 MiB block. When EVERY byte of the pattern is common in the block -
"00 00" in that same file - there is no anchor worth walking, so the
reader answers -2 and the block goes through xxd: the bad case costs
what it always did (measured: 0.24 s either way) and never more.
A pattern arrives as two Blobs, mask and value, built by
HexPairPagedFindByteFilter() from the same dotted hex the regexp is
built from, so the two forms of one pattern cannot drift apart. Byte k
matches when and(byte, mask[k]) == value[k], which is what keeps '?'
working at nibble level - a plain run of bytes could not express "d?".
A pattern with no fully specified byte has nothing to anchor on and
declines like a dense one.
autoload/hexpair.vim must never become load-bearing, and does not:
HexPairPagedBlobFindSupported() sources it by path relative to <sfile>
(so a plugin sourced from outside 'runtimepath' - which is how the suite
runs it - finds it too) and then asks it a question whose answer is
known, once per session. A file that was not copied, or a Vim that will
not load it, means the hex reader rather than an E117 in the middle of a
search. plugin/hexpair.vim still runs on Vim 8.0 with autoload/ deleted.
The suite checks the filter, the walk's own answers including both
refusals, and then the thing that actually matters: a dense fixture
where the byte reader must decline, a nibble wildcard, and a backward
scan all landing on the byte the hex reader lands on.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The branch that runs where there is no byte reader wrote four lines where the branch above it writes five, so every check after it read the wrong line on such a Vim - which the run against a simulated Vim 8.0 found and the run on this machine could not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
s:FileBlob() called readblob() with an offset and trusted the answer.
On native Windows past 2 GiB that answer is an empty Blob AND SUCCESS -
read_blob() declares a plain `struct stat` where the rest of Vim uses
stat_T, so st_size overflows and the computed length goes negative - and
an empty block is indistinguishable from a block of nothing. A
comparison would have called every such block identical and a search
would have found nothing in it: "no change" and "no match" for
everything past the 2 GiB mark of a large file, silently, on the one
platform this is not developed on.
The plugin already knew: s:CopyRange() says so in as many words and
routes around it, hexpair-windows-2gib documents it, and s:XxdCanSeek()
is the check every other reader makes. s:FileBlob() now makes it too.
Past that offset the bytes come from s:SeekReadBlob(), which is
s:SeekReadHex() with one line changed: PowerShell writes the range to a
temp file, as it already does for the page, and the block is read out of
THAT - no offset to seek to, so no limit to hit. It is strictly cheaper
than the hex reader it sits beside, by the xxd process, the pipe
carrying twice the bytes as text, and the line breaks taken back out. So
the fast comparison and the fast search apply past 2 GiB on Windows too;
what dominates there is the PowerShell start in front of them, as it
always did.
Pinned by the suite the way the hex seek reader already is: in the
has('win32') branch that only Windows CI runs, the two readers are asked
for the same range and must return the same bytes, and a range past the
end of the file must be empty on both.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
s:LiveHex() spelled the windowed text view's page by joining its lines
with a NL and walking the result a byte at a time. Both halves of that
were wrong.
WRONG ANSWER. Vim holds a NUL inside a line and getline() hands it back
as a line break, so joining with a NL makes a NUL and a real line break
the same character - and the speller then spelled both 0a. The buffer
had not lost the difference; only the join had. And s:PageBytes() has
always taken the bytes for a WRITE with writefile(lines, ..., 'b'),
which is the exact inverse of how Vim loaded the buffer, so the report
disagreed with what :w would put on disk. On an untouched page holding a
NUL, :HexPairModifiedShow said
hexpair: byte 3 (0x3): 0a here, 00 on disk
about a byte nobody had touched, while :w would have written 00.
TOO SLOW. Spelling a page a byte at a time in Vim script does not scale,
and not merely linearly - four times the page was fourteen times the
wait:
128 KiB page 3.63 s -> 0.009 s
512 KiB page 55.76 s -> 0.028 s
So it now does what the writer does: writefile() the lines to a temp
file and let xxd spell it, which is what s:HexFromFile() is for. One
mechanism instead of two, and the one that was already trusted with the
file.
Nothing else moves. The change is one branch of one function, whose only
caller is :HexPairModifiedShow - a command that reads and prints and
writes nothing. The hex view's branch is untouched, so the markings are
as they were. The temp file is one page, written at offset 0 and read
with no seeking, so it meets no 2 GiB limit on any platform: this is
about the buffer, not the file.
|hexpair-marking-views| keeps its blind spot and gains a sentence saying
whose it is: the MARKINGS compare in the text view's own spelling on
both sides, which is where a NUL and a line break really do read alike.
Neither this command nor the write was ever among them, and the help
said otherwise.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
HexPairModified, and the jumps over it, compared in the TEXT VIEW's own
spelling: line by line, against the page's bytes rendered the same way.
In that spelling a NUL and a line break are one character - Vim holds a
NUL inside a line and getline() hands it back as a line break - so
swapping one for the other was not an edit at all. The page answered
"nothing edited on this page" while :w would have written a different
byte. Two smaller errors came with it: an edit that changed the page's
length named the byte AFTER the first differing one, because a line's
span moved with the edit while the bytes it was held against did not;
and one edit came back as one run per line it touched, since the line
break between two of them is a byte no run covers.
Both views now take their runs from one comparison of real bytes - the
page as the buffer holds it now against the page as it was read, which
is what the hex view has always done - and the marking, the jumps and
:HexPairModifiedShow all read those same runs. They can no longer
disagree with each other, or with the write.
The cost is accepted rather than overlooked. The text view compares the
whole page per edit where it used to compare the lines on screen:
text view, per edit 128 KiB page 512 KiB page
before 4 ms 8 ms
after 30 ms 133 ms
which is exactly what the HEX view already cost for the same pages (30
ms and 65 ms, unchanged by this). The text view was cheap because it was
answering a cheaper question wrongly, and the two now cost the same
because they finally do the same thing. Nothing is recomputed between
edits, so scrolling and moving the cursor stay free, and a paged text
view is for reading rather than for typing in.
Neither this nor s:LiveHex() goes near the 2 GiB offset limit, and it is
worth saying so because that limit has already been walked into once
here. Both work on the BUFFER: the temp file s:LiveHex() writes is one
page, written from offset 0 and read by xxd with no -s at all, and
everything else on this path is b:hexpair_page_hex, already in memory.
The limit lives in one place only - reading a byte range OF THE FILE -
and nothing here does that.
s:TextComparePositions() stays for the 'diff' layer, which compares
against ANOTHER FILE, still in the text spelling, and keeps that blind
spot. It has an empty-hex case of its own - past the end of a shorter
file every byte differs - so it wants care rather than a copy of this.
The help now says whose blind spot is whose.
The suite gets the two directions the old comparison could not see; both
fail on the old code with the bug in as many words. They are not
symmetrical, and the asymmetry is the view's rather than the
comparison's: a NUL has a column of its own and is painted, a line break
has none and can only be jumped to.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…lings
:HexPairDiff's marking in the text view took the same wrong turn the
edited-bytes marking did: it compared getline() against the other file's
bytes rendered into the text view's own spelling, where a NUL and a line
break are one character. Two files differing in exactly that byte were
therefore identical there, while the hex view marked them - which is the
same disagreement between views that the last commit removed for the
other layer.
s:DiffRuns() answers it the same way, from real bytes on both sides, and
the marking clips those runs onto the lines on screen. Nearly free beside
the other layer: s:LiveHex() is cached per b:changedtick and both read
it, so a comparison running as well costs one more pass over the page's
hex, not another read of the buffer. Per edit in the text view, both
layers together:
128 KiB page 39-52 ms -> 47-48 ms
512 KiB page 148-176 ms -> 206-210 ms
and scrolling got cheaper, since the runs are cached where the old
comparison was redone for each new set of visible lines.
The cache is invalidated in s:LoadDiffHex(), which is the one place the
other file's bytes are set and the one event no tick reports: a page turn
and a second :HexPairDiff both change what is being compared without
touching the buffer.
The page-past-the-end-of-a-shorter-file case needed no special case, only
checking - HexPairPagedDifferingByteRuns() already counts bytes the other
run does not reach as differences of their own, which is exactly what
"every byte here differs because there is nothing over there" means. Held
to that by the block that already exists for it, and measured identical
before and after.
s:TextComparePositions(), s:BytesAsText() and HexPairPagedTextRuns() were
the spelling comparison and nothing else used them; 98 lines go with
them, and the unit test of the middle one goes too, since it was testing
a function the plugin no longer has.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… seam
A pre-release audit, driven by breaking the code on purpose and asking
whether the suite noticed. Six mutations went through green, and each
one is a hole this closes.
THE SCAN SEAM TESTS REACHED NO SEAM. A scan reads its first block from
the byte after the CURSOR, so the seam is at `from + block` and moves
with wherever the search began. The fixture put its needle a megabyte
into the file, which left it comfortably inside the first block of the
search meant to straddle it: taking the block overlap out entirely
changed nothing this suite could see. The offsets are computed from the
search that meets them now, and the backward direction - which reads
past its block by the pattern's span, a different line of code - gets a
check of its own. Both mutations are caught.
THE 2 GiB RULE COULD NOT BE CHECKED OFF WINDOWS AT ALL. s:XxdCanSeek()
asked has('win32') itself, so on Linux every range is xxd's and the
tests said so without the rule having to exist. It is now
HexPairPagedSeekableOffset(off, win32), pure and taking the platform as
an argument, exactly as HexPairPagedGateMessage() takes its boolean and
for the same stated reason. Both columns are asserted wherever the suite
runs, including the boundary itself and the byte past it, which the old
values missed.
AND WHETHER THE READERS ASK IT. Removing the guard from s:FileBlob()
passed everything on Linux and would break a large file on Windows in
silence - readblob() answers an out-of-range read with an empty Blob AND
success, xxd's strtol() saturates. That is not hypothetical: it happened
while this branch was being written. A mechanical check over the source,
like the packaging list, now holds every reader of a (file, off, len)
range to consulting the rule on the END of its range, and refuses a new
readblob() range anywhere else.
THE DIFF RUNS CACHE was invalidated by nothing the suite exercised. The
first attempt at a test did not catch it either, because a page turn
bumps b:changedtick by itself; the check now warms the cache at a tick
and then changes only what is being compared.
EVERY OPTION IS LISTED EVERYWHERE, mechanically. g:hexpair_verify_writes
was described in prose in all three places that list options and named
in none of them - it is read with get(g:, ...) rather than given a
default, so the eye that keeps the others in step slid past it. It has a
tagged help entry, a line in the plugin header and one in the README
block now, and a check that an option the plugin READS is an option all
three list, however it is read.
No dead code: the eight functions that look unreferenced are reached
through <SID> in <Plug> mappings or as a sort comparator.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The marking of bytes edited and not yet written is the one part of a hex page whose cost follows what was DONE rather than what is on screen. An overwrite marks the bytes that were typed and costs nothing anyone notices. An insert or a delete moves every byte after it, so every one of them differs from what the page was read as: the whole rest of the page is one run, marked correctly, and compared again on every keystroke. That is the moment the marking stops helping, and until now the only way out was to set an option and reopen. :HexPairModified toggles it, :HexPairModified! stops it. That shape is not invented here - :HexPairFind! and :HexPairDiff! are already the two ways to stop a marking, and this fills the third slot in the same table: there is nothing to SET for your own edits the way a pattern or a file is set, so the bare command toggles instead. The name is the plugin's one word for this - the option g:hexpair_show_modified, the highlight group HexPairModified, the commands :HexPairModifiedNext / Prev / Show - rather than a second noun for the same thing. :HexPairShowModified was rejected for reading as :HexPairModifiedShow, which is a different question entirely. It flips g:hexpair_show_modified rather than keeping a switch beside it, so a session and a vimrc cannot come to disagree about the same thing. s:ModifiedHighlight() now CLEARS when the option is off instead of returning early. The marks belong to the window, so every other window showing the page has to lose them too, and that branch is what carries the toggle to them; it costs nothing once they are gone, since the id list is empty and the loop does not run. <Leader>M in hexpair.vimrc - M for Modified, and the m prefix that marks use is lowercase, so nothing waits on a second key. One <Plug> target and not two: a toggle already reaches "off", and the shipped keys are what the <Plug> set is for; the bang stays typeable. The suite checks the option, both messages, that the bang does not toggle, and that the bytes to mark are the same after a round trip - the switch is about drawing them, not about what is compared. The drawing itself cannot be checked headlessly, a vim -es window having no geometry, which is the same half the diff marking settles for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A hex dump is hex digits and an ASCII column, and 'spell' underlines both as misspelt words: "de ad be ef" is four of them, and the ASCII column is worse, since whatever the bytes happen to spell is marked as prose. None of it is text a dictionary has any say about. The bundled ftplugin turns it off, which puts it exactly where it belongs: the DUMP alone. The windowed text view is not filetype xxd, so it goes on spelling as the window always did - which is the whole point, since that view IS text and is where a speller earns its keep. And a window that never had 'spell' on is never given it. 'spell' is window-local, unlike every other option this ftplugin sets, so the undo carries the value the window had rather than the `spell<` that would bring back the GLOBAL one - a window with 'spell' set on its own would otherwise lose it on the way out. That is why the undo string is built before the option is switched off. It stays on one line and uses no script variable, for the reason the file already gives: it may be sourced with a 'cpoptions' that disables line continuations. Both directions are walked in the suite - a window with 'spell' on through plain, hex, text, hex and back out, and a window with it off through the same - because only that catches the difference between restoring the value and restoring the global. The first attempt at that test toggled an unnamed buffer, where :HexPairUnhex has nothing to go back to, and read the confusion as two bugs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
:HexPairUnhex put the cursor back at the line and column the plain view was left at when hex mode was entered. That is the wrong end of the session. Hex mode is where a file gets WRITTEN - by this plugin, or by anything else while it was open - so a position taken before all that describes a file that may no longer exist; and where the cursor is when it leaves is, simply, where the user was looking. So the byte the hex view is on decides, and the remembered position becomes the fallback rather than the answer. Whether that byte can be pointed at is MEASURED, not deduced: the re-read buffer's length plus any BOM against the file's size. A transcoding 'fileencoding', a stripped BOM and folded CRLFs all move the count between a file's bytes and a buffer's, and comparing the two catches every one of them at once - and catches whatever else a Vim decides to do on the way in, which reading 'fileencoding' would not. The BOM is corrected for rather than refused, since it shifts every offset by exactly its own length. Asking for the byte is guarded twice, and both guards have a case behind them. A buffer with no page has none to give: s:AbandonSetup() leaves one holding the snapshot and no page state, and rescuing that buffer is something this has to keep doing. And a page whose dump no longer READS as one cannot say which byte the cursor is over, because counting them means parsing them - a line of prose appended to a dump gets E716 out of s:PagedLineBase. That page is precisely what :HexPairUnhex! is for, so this is the last place that may fail on it. The second guard was written, then removed as dead on the evidence that s:PagedByteOffset() contains no throw, and the suite put it straight back. Three cases in the suite: the byte lands where it should, the broken dump falls back, and a latin1 file whose forty bytes are forty-three in a utf-8 buffer falls back too. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The previous commit's landing worked on a Markdown file and did nothing at all on a binary - which is to say, it did nothing, since a binary is what hexpair is for. Reported from a real session, and the fixtures had hidden it: every one of them was text ending in a newline. Two things were wrong, and both are properties of the files that matter. A binary opened with a plain :edit is read by Vim as latin1 and transcoded, so a 200-byte file is a 293-byte buffer: no BYTE offset carries across, though every CHARACTER one does. And most binaries do not end in 0x0a, so Vim counts a final line ending the file has not got and the count is one over before anything else happens. Comparing line2byte() with getfsize() therefore never matched, and every such file fell back to the old behaviour. s:PlainPosForByte() works the position out instead of guessing whether it can, and it is the exact inverse of s:PreReloadPos() / s:PostReloadOffset(), which map the other direction for the ++bin reload: a line costs the file its characters where 'fileencoding' is single-byte and its bytes otherwise, plus the ending 'fileformat' gives it, and no ending on the last line when 'endofline' is off. The BOM is counted the same way it is on the way in. The walk totals the file as it goes, and that total against the size on disk is the check - so a file this model does not describe, a multi-byte 'fileencoding' that is not the internal one say, still falls back rather than landing somewhere invented. That is a stronger test than the size comparison it replaces, because it is the arithmetic that will be used to land. Fixtures now cover the three that were missing: no trailing newline, a latin1 text file, and an actual binary opened as text - which lands on line 1 column 46 for file byte 33, a column no byte count would have found. (The first attempt at this also compared a List with a number on its second time round the loop, which a probe hid by calling the command with silent!.) Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
'scrollbind' cannot follow a page turn, so hexpair moves every bound window to the page holding the same byte. A window whose own file did not reach that far stayed where it was, with a message - and the two windows then showed different offsets side by side, which in vimhexdiff is precisely the confusion binding them exists to prevent. The message is no help: it scrolls away, and what is left on screen looks exactly like a view that followed. Such a window goes to the page now and says the page is not there. A banner names it, says where the file really ends and on which page, and that is the whole content: no bytes. Deliberately s:LoadEmpty() with a different banner and a base. A page with no bytes is a shape this plugin already has, guarded in every place that counts them, so nothing new had to learn about it - the markings, the jumps and the inspector all find nothing here, which is the truth. The base is the one thing that is not zero: it is where the page would begin, so the two views agree about which page they disagree about and a turn back to a real one is an ordinary turn. b:hexpair_page_absent marks it for the one thing that has to refuse rather than shrug: a :w. Nothing would be written without that check either, the page being empty, but a command that silently does nothing is worse than one that says why. Only the bind reaches it. On its own, asking for a page that is not there is an error and stays one - showing an invented page to someone browsing a single file would be the confusion, not the cure. And only a page WHOLLY past the end: a last, partial page is a real page, the file does reach it, and it is shown exactly as before. That is the shape the maintainer asked for in place of the first sketch, which would have taught ninety-nine uses of b:hexpair_page_base and _len about pages with no file behind them - the write path among them - to gain some offset columns and rows of dashes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The vim80 job failed on "every reference in the help resolves to a tag", and it was right: |readblob()| went in four times during this cycle and readblob() is a function Vim gained in 8.2.2343. On a modern Vim every one of them resolves, which is why they passed here; on the 8.0 floor this plugin supports they are dead ends, and the job that builds 8.0 from source is the only place that shows. They are plain text now, the way the paged section already names the function when it says which patch added it. The rule is written down in both the test and CLAUDE.md, because nothing local can catch the next one. The rest is the release checklist CONTRIBUTING.md asks for, which found three more things. THE CHANGELOG DESCRIBED THE CYCLE, NOT THE RELEASE. Three separate entries reported the scan getting faster in the order the work happened - 8% here, 10.3 s to 8.4 s there, then 45x - each measured against the state before it rather than against v2.3.0. A reader of the release wants the release: searching and comparing are one entry now, with the numbers that matter (11.6 s to 2.0 s and 16.2 s to 0.25 s over 256 MiB), and the xxd reader that stays behind is the second. THE README'S COMMAND TABLE HAD LOST A FAMILY. :HexPairModifiedNext, :HexPairModifiedPrev and :HexPairModifiedShow were all missing from the table that is the command reference - one of them added a release ago - while being described in prose two screens above. A mechanical check now holds every command the plugin DEFINES to being named there, the way the packaging list is held to the repository. AND THE BINDING WAS DOCUMENTED AS IT USED TO BEHAVE. Three places still said a bound window whose file does not reach that far stays where it is, which is what the absent page replaced. Mutation testing on everything added since the last audit found three more holes, all now closed: the marking's clearing branch (the one that carries :HexPairModified to the other windows), the self-check that makes s:PlainPosForByte() fall back on a file it does not describe, and the clearing of b:hexpair_page_absent when a real page loads - which is what lets a page be written again after visiting one that is not there. The utf-16 fixture is the file no byte count describes: two file bytes a character, one or two buffer bytes, so neither walk adds up. Not bumped: Version: and Date: in plugin/hexpair.vim are the release step, and are the maintainer's. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two gaps that only show from outside the repository. The README told people to untar a release or clone into pack/, and nothing else - so anyone who installs Vim plugins the way most people do, with a manager, had to work out for themselves that the layout is a plain Vim package and that it therefore just works. It says so now, with the two lines for vim-plug and minpac, and states the floor and that there is no Neovim: this is tested against Vim 8.0.0000, current Vim and Vim on Windows, and nothing else is claimed. And CONTRIBUTING.md gains a Publishing section, because there is no manifest a Vim plugin registry reads - vim.org is a web form, GitHub topics are repository settings - so what has to be typed where was written down nowhere. It records the one-time vim.org fields, the per-release upload (the tarball the release already builds, the version from the plugin header, 8.0 as the floor, the CHANGELOG entry as the release notes), and the gh command for the topics and description. The vim.org description is deliberately kept short there. Everything that changes lives in the README and in :help hexpair, and a second full copy of either is a copy that drifts - which is the failure this cycle has already had twice. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Publishing section reached for `gh` as though it were the only way to set a description, a homepage and topics, and it is the one a maintainer who does not use that CLI cannot follow. The web interface is the shortest route and needs nothing installed; the REST API needs only curl and a token. Both are written out, with the limits GitHub applies to topics and the warning that the topics call REPLACES the list rather than adding to it. And it says plainly that these cannot live in a file in this repository, because that is the question the section was really being asked. There is a third-party app that reads .github/settings.yml, and it is named here only to be turned down: installing it escalates anyone with push to admin, since pushing to that file is how settings change - its own README says so. That is a standing risk in exchange for three fields that are set once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
vim.org refuses a POST body somewhere between 224 and 250 KiB. The limit
is documented nowhere and arrives as a bare 413 from the web server, or -
just under it - as an internal error, and the release tarball is 900 KiB.
So pack-release now builds a second artifact: the same files less
CLAUDE.md and CONTRIBUTING.md, which are written for somebody working ON
hexpair rather than with it, compressed with bzip2. 177 265 bytes.
CHANGELOG.md stays: what changed in a release is a user's business.
It is a release artifact in its own right, not a vim.org special, so it
is named for what it is.
THE COMPRESSOR WAS MEASURED, not chosen. On this content - one very large
and very repetitive text file plus smaller ones:
bzip2 -9 185 692 (without CLAUDE.md, for comparison)
xz -9e 189 112 = 7-Zip's "ultra", which is LZMA2
gzip -9 249 401
7-Zip PPMd, order 32 160 325
7-Zip's "ultra" preset LOSES here, which is worth knowing before someone
reaches for it. What wins is PPMd, by 14%, and it is not used: a .7z needs
7-Zip or p7zip to open, which a stock Linux, a stock macOS and Windows
before 11 do not have, and 177 KiB already uploads. The only thing that
saving could buy is a package some readers cannot unpack.
AND GZIP WOULD NOT DO for a second reason beyond the one CONTRIBUTING.md
gives about deflate streams differing between compressor builds: the gzip
header carries the source file's mtime and its name. The same bytes from a
file checked out at a different time compress to a different archive -
demonstrated - which is exactly what a second CI runner is. A bzip2 stream
has nowhere to put either: "BZh", one digit of block size, then blocks.
So the minimal package is expected to be byte-identical across platforms,
and expectation is not proof: CI now compares BOTH artifacts between the
Linux and the Windows build, by hash and by name, rather than the
canonical tarball alone. If bzip2 ever diverges, that check says so
instead of the maintainer finding out at release time.
The suite holds the omission list to being a subset of the packaging list,
so a rename or a typo cannot quietly omit nothing and push the package
back over the limit.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It was named in one line under the project links and explained nowhere, which leaves anyone looking at a release with two files and no way to choose between them. Installation now says which to take - the complete tarball - and what the other one is: the same plugin with CLAUDE.md and CONTRIBUTING.md left out and compressed, because vim.org refuses an upload somewhere between 224 and 250 KiB and the complete tarball is 900 KiB. It says what is NOT missing, which is the part a reader actually needs to trust: the plugin, the help, the mappings, the shell and Windows commands, the registry files, the icons, the licence, the README and the changelog. What is gone is the project's own notes and the contributor guide, and both are a click away. Verifying Releases covers both archives now, names the second signature file, and says that each is byte-identical whichever platform builds it - which is a claim CI checks rather than one this file makes. It also says that the copy on vim.org is that same archive and that its signature lives here, since vim.org hosts its own copy and nothing there vouches for it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
test-oldest-vim was right and the test was wrong, twice over.
It assumed a headless window has drawn no marks of its own. That holds on
a current Vim, where line('w0')/line('w$') in -es come out crossed and the
marking paints nothing, and it does NOT hold on 8.0, which answers them
usefully and draws. So the seeded count was 2 there rather than 1. Worse,
seeding REPLACED w:hexpair_mod_ids instead of adding to it, which orphaned
the id of the mark the plugin had drawn - so the clearing found one mark
where there were two and left one behind.
And the order was wrong for what it claimed to test: it toggled the
marking off and then looked, but the command clears the current window
itself, so the result was the same whether or not s:ModifiedHighlight()
has its clearing branch at all. Mutation testing had already reported that
branch as unreachable by the suite; this is why.
Now: off FIRST, so the command's own clearing is spent and cannot account
for the result and the window starts with no marks on any Vim; then a mark
put where another window's would be; then only a redraw. Removing the
clearing branch now fails this check, which is what it was for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
test-oldest-vim is the job that catches what nothing else does, and the
only way anyone here had of consulting it was to push and wait. Twice in
a row that meant spending a maintainer's CI cycle on a question that
could have been answered in ten minutes locally.
CONTRIBUTING.md now carries the recipe, flags and all. They are not
optional and have nothing to do with Vim: a 2016 source tree meets a
compiler that has since made implicit declarations, implicit int and
incompatible function-pointer arguments errors rather than warnings, so
8.0.0000 does not build without saying so. Pinned by the same commit the
workflow pins, and using the distribution's xxd for the reason the
workflow already gives.
The finding that sent me looking is in CLAUDE.md, because it will bite
again: in vim -es a CURRENT Vim answers line('w$') below line('w0') and
therefore draws nothing, which several tests quietly lean on, while 8.0
answers usefully and draws. A test that counts marks or positions has to
establish the state it starts from rather than assume a headless window
is empty.
Verified: 728 checks pass against 8.0.0000 as well as against 9.2.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It is the third file in that package that is reading matter rather than plugin, and on vim.org - which is what the package exists for - it is reading matter the site already has: every version carries its own release notes in a field of its own, pasted from this very file. Shipping all 62 KiB of it inside the archive as well is the same text twice, in the one place where space is the constraint. 164 475 bytes now, against 177 265. What is left is what the plugin needs to run and the README that explains it; the changelog, the contributor guide and the project notes are a click away on GitHub, and the complete tarball beside it has all three. README.md, CHANGELOG.md and CONTRIBUTING.md all described the contents and all three said something now untrue, so all three are corrected. The promise they make is narrower and more honest for it: not "nothing a user needs", which invites an argument about what a user needs, but "nothing the plugin needs to run", followed by the list of what is gone and where to find it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The README was 1590 lines, and the two longest things in it were a second copy of documentation that exists elsewhere. The 2 GiB section was 109 lines against :help hexpair-windows-2gib, which says the same things and more - and the copy HAD ALREADY DRIFTED. Its "what runs where" table still claimed searching and comparing go through `xxd -s -l`, which stopped being true when the byte readers went in this cycle. The pre-release audit did not catch it, because nothing mechanical can: the words are all still words the help uses. It is fourteen lines now, saying what a user needs - it works, it is slower, one thing is actually faster - and pointing at the help for the rest. The context menu was 156 lines, of which about a hundred were why the shipped .reg files look the way they do: REG_EXPAND_SZ and the hex(2) UTF-16LE encoding that forces them to be generated, the %VAR%-before-%1 expansion order that makes them safe, ExtendedSubCommandsKey keeping everything out of HKLM, and why the console window is allowed to flash. That is maintainer material and it moved to CONTRIBUTING.md, beside the generator script it explains. What is left in the README is how to add the menu, what it looks like, and how to regenerate it for another path. And the block-size measurements went the same way as the 2 GiB table: they are reference material, :help g:hexpair_scan_block has them, and a README does not need a second set to keep in step. 1590 lines to 1381, 79 KiB to 67, and not one sentence of it is gone from the project - only from its second and third copies. The compressor and package figures were re-measured against the shorter tree rather than left as they were, since they are measurements and this is the tree they now describe. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The reproducibility check has been failing on identical archives. Both platforms produced the same two SHA-256 sums, and the job rejected them: linux: ...aacce81 dist/hexpair.v2.4.0-devel.tar windows: ...aacce81 hexpair.v2.4.0-devel.tar sha256sum echoes back the path it was given and PowerShell's .Name is the bare file name, so the two sides have always written the name differently. It never mattered while the comparison took only the hash. It started mattering when the minimal package arrived and the name was added to the comparison, so that a missing archive could not pass by having its line compared against the other file's. So cut the name down to its last component before comparing, and sort after that rather than trusting each producer to have sorted whichever form it happened to write. And require two lines a side. Equal lists prove nothing about the packaging if the packaging produced no archives at all, and a reproducibility check that can pass vacuously is worse than none. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
It was the last thing in Installation, below the whole table of default mappings - a hundred and twenty lines after the `tar xf` a reader has by then already run. A signature check noticed at that point has been noticed too late. It now follows the two archives it is about, before the routes that do not involve an archive at all. That needs headings for what used to trail after it, or a plain `git clone` would read as part of verifying a signature: the repository and the plugin managers become "Other Ways to Install", and the mapping table becomes "Mappings", which it should arguably have been anyway - it is the longest thing in the section and had no name. The Vim-and-xxd sentence moves up to open the section instead of sitting between two install routes, since it is true of all of them. And the notice goes inside the snippet, on the line above the `tar xf`, because that is the part that gets copied. The outline's section count is a canary for the parser having stopped seeing headings, so it is raised with the two headings, not because the list itself changed - the outline checks either side of it prove that. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
:HexPairFind read the file and ignored the buffer, so its answers were wrong in both directions at once. Bytes typed into the page were "not found in this file" while they were on the screen; bytes typed OVER were still found, and the cursor jumped to an offset where they were no longer to be seen. The maintainer hit the first half by inserting a string and searching for it. The plugin already held the opposite principle everywhere it wrote something: :HexPairReplace decides on the buffer's bytes - "the file still holds the pattern where this buffer no longer does" - and answered a jump of the second kind with "the cursor is not on a match". :HexPairDiff's marking has always compared the buffer's own digits against the other file. It was the two file-wide WALKS that disagreed with all of it, and the help disagreed with itself in the same place: :HexPairDiff says "what is marked is what is on SCREEN", and :HexPairDiffShow said, four paragraphs later, "the page as it was READ". So the page in view is laid over each block as it is read. One page is all that can ever be modified - turning a page needs an unmodified buffer or a bang that discards - which is what makes one contiguous overlay enough, and s:SearchPageHex() the single source of those bytes: the two scans take it through s:FindInBlock() and s:CmpPair(), the two chokepoints each already had, and the marking takes it through s:PageHexForSearch(), or the cursor would land on a match nothing highlighted. CLAMPED to the length the page was read with, which is what keeps every offset a file offset. An insert grows the page past its own end on disk and those bytes have no offset to report or to jump to; they are searched once :w has given them one. That is the boundary :HexPairModifiedShow already names, and the tests pin it rather than leaving it to be discovered. :HexPairDiffShow now reports the buffer's byte on this side, which is the one the marking beside it compares and the one the cursor is on. The other file stays the file on disk, in a bound pair of views too: how what is here differs from that file is the question these ask. The Blob for the byte reader is built on first use, not with the page: a scan that finds its answer before it reaches the page needs none, and it is 0.14 s on a 128 KiB page. Twelve checks cover it, and all twelve fail without the overlay: both readers (a dense pattern is handed back by the byte reader and goes through the hex one), both views, the forward scan, the backward one, the wrap, the growth boundary, and the comparison walk. The overlap arithmetic is asked directly, since through a real scan the smallest block the setting allows is a megabyte. The README's own example of :HexPairFindText was PK\x03\x04, which is not a string it can find: there are no escapes, so that is eight characters and not four bytes. It is a real string now, with a second one to show that the bytes are what 'encoding' spells it as. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The size ladder and the compressor table in pack-release.py, and the figures quoted from them in CONTRIBUTING.md and CHANGELOG.md, were measured before this cycle's documents changed. Every one of them had moved. That is the second time, so the numbers are now marked as the snapshot they are, and the suite checks the thing they exist to support: it builds the minimal package in memory and fails if it is not under 200 000 B, comfortably inside the 224-250 KiB band where vim.org starts answering 413. One of the re-measurements is worth reading: compressing the COMPLETE tarball now gives 225 869 B, which is no longer under that band. The smaller package is not a convenience any more. The 7-Zip PPMd figure could not be re-measured - no p7zip here - so it is stated as the 14% it wins by rather than as a byte count from a tree that no longer exists. The LZMA2 comparison is xz -9e, which is the same algorithm at the same setting as 7-Zip's "ultra", and measured now. The changelog gains the search and comparison fix. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The new comparison block cut the temporary directory off the other file's name with a substitute() on "$WORK/", and that is a POSIX path. A native Windows Vim reports the same file as ~\AppData\Local\Temp\tmp.X\dirty2.bin - backslashes, and the home directory shortened away - so nothing was cut and four checks failed on that job alone. The suite already has the answer, in the block that compares across a scan seam: the script says what THIS Vim calls the file, the checks build their expectation from that line, and check_path compares with the separators normalised. Same thing here. And the check that builds the minimal package ran `python3` directly, thirty lines after this file explains that Windows installers name it `python` and that a `python3` which merely opens the Microsoft Store is a common decoy - which is why $PY exists and is found by running it. Use it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The two lists printed identically and compared as different: linux: f42a65...ce6be hexpair.v2.4.0-devel.minimal.tar.bz2 windows: f42a65...ce6be hexpair.v2.4.0-devel.minimal.tar.bz2 Tee-Object writes CRLF, so every line of windows.sha256 ends with a \r that awk leaves on the last field. A \r immediately before a \n prints as nothing, which is why the evidence in the log looked like proof that the check itself was broken. It was invisible for the same reason the path prefix was: while only the hash was compared, the \r sat on the field nobody looked at. Adding the file name to the comparison exposed both, one after the other. So tr -d '\r' before the awk, and on a mismatch print the diff through cat -A rather than exiting bare. An archive that genuinely differs announces itself in the hash; every failure this check has actually had was something a diff would not show. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things the release checklist asks for, and found. CLAUDE.md still said the Explorer menu need not clean up after its own earlier shapes because "nothing was ever released with those", and extended that licence to "the next restructure while v2.3.0 is unreleased". v2.3.0 shipped that submenu. A user can have it installed now, so the next restructure has to REMOVE what that release put in the registry rather than orphaning it - which is the opposite of what a future session would have read there. CONTRIBUTING's release process gained the minimal package in steps 4 and 5 and did not gain it in step 6, which still said to attach "both files (.tar and .tar.asc)". There are four. The README sends readers to the smaller package when the complete tarball is more than they want, so a release carrying only the .tar leaves that pointing at nothing. And the suite's new packaging check imported pack-release.py, which left a __pycache__ in the repository root on every run. It is gitignored, so nothing would have caught it; sys.dont_write_bytecode stops it. The rest of the checklist verified rather than changed: the Added entries name things no earlier release had (:HexPairModified as a COMMAND is new even though the highlight group of that name is not), the Fixed entries name behaviour v2.3.0 actually shipped - its :HexPairFind read the file and nothing else - and no runtime message describes the search as reading the disk any more. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
hexpair.vimrc is the file a user is told to copy into their vimrc, so its Options block is where an option is actually found and set. It had eleven of the fifteen the plugin reads. Missing: g:hexpair_scan_block and g:hexpair_show_inspect, never added; g:hexpair_verify_writes, which an earlier audit added to the three places it was checking and not to this one; and g:hexpair_insert_encoding, which this file MENTIONS in prose - "in g:hexpair_insert_encoding (utf-8 unless you say otherwise)" - a hundred lines above the block that never gave the line to uncomment. Being described is not being findable. The four were invisible because the completeness check knew about three places and this was not one of them. It is now the fourth, which is the part of this commit that matters: the same drift produced all four omissions, one at a time, over several cycles. The check compares NAMES only, deliberately. The README writes each option at its default and the vimrc writes the value that changes the behaviour - "Every line below is the NON-default value" - so the two files disagree about every line by design, and a check on the values would have to encode which convention each file follows. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Options got a completeness check last week and the two other hand-kept lists did not, so they had drifted the same way. Both omissions are the newest entry of their kind, which is how these fail: what is added last is added to whichever files the author had open. <Plug>(HexPairModified) reached the README and hexpair.vimrc and never the help - no tag among the others, and no line in the help's own copy of the mapping set, which otherwise lists every target. A reader of :help hexpair-mappings could not find the target for a command this release adds. The HexPairInspect highlight group reached the README and the help and not hexpair.vimrc, whose Colours block listed the other seven. That file is the one a user copies, so the group with no line there is the group nobody overrides. Both are now covered by a check of the same shape as the options one, and it fails on either omission by name. Names only, each file matched in its own dialect: the help tags a group as <name> or <name>-highlight, the README writes a commented `" highlight`, the vimrc `"highlight`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…pped `make-context-entry-reg.py --help` took --help as the INSTALL PATH and wrote both shipped .reg files for a menu pointing at a directory called "--help". Found by running it during the release audit; the files were restored from git, but a maintainer who did that and did not look would have committed them. It now prints usage for -h/--help, rejects any other option and more than one argument, and writes nothing in either case. The suite gains the check that would have caught such a file: the committed .reg files must be byte-identical to what the generator writes. The checks beside it read the committed files and decode them, which passes just as well on a hand-edit, while CONTRIBUTING.md calls the generator their source. The script writes beside itself, so it is copied into the work directory and run there - a check that rewrites what it checks is not a check. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The .reg check held the "Icon" values to ending in .ico and stopped there. Explorer draws nothing at all for an icon path that is not there, and reports no error, so a renamed or dropped .ico would reach a user as a blank menu entry and reach nobody here. Three files, four references - one is shared - and the check now says so by name when one goes missing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…hout README.md goes into both archives. It linked relatively to CHANGELOG.md, CLAUDE.md and CONTRIBUTING.md - the three the minimal package leaves out on purpose - so the copy uploaded to vim.org, which is the one most readers will ever unpack, carried three dead links. And to demo/hexpair-demo.gif, which is 5.7 MB and is in NEITHER archive: the picture at the top of the README has never rendered for anyone reading it outside GitHub. The README's own sentence about the omissions says they are "reading matter that is a click away here", meaning GitHub, so the fix is the one that text already describes: those four are absolute URLs now. LICENSE.md and doc/hexpair.txt stay relative, because they ship in both. The rule is now a check, and it is about every shipped .md rather than this one: a relative link must resolve inside every archive that carries the document it is in. A file that lives on GitHub and not in the tarball is an absolute URL, or it is a dead link for whoever unpacked it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Third time. The ladder in pack-release.py and the compressor comparison quoted from it were re-measured a week ago and were stale again by this morning - the README edits in the commit before this one moved the minimal package by 117 bytes, and a version string that loses "-devel" at release will move it again. Precision to the byte buys nothing here. The ladder answers "what do I drop next if vim.org refuses this", and the compressor table answers "is bzip2 the right choice", and neither question is decided in the third significant figure. So the ladder is rounded to KiB and the compressors are given as percentages against bzip2, which is what the comparison was always about and does not move when a document does. What is exact is the suite: it builds the package and fails if it is over 200 000 B. That is the promise these numbers were being asked to keep, and it keeps itself. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The release checklist draws the line at what a RELEASED version did wrong, so each of the audit's findings was put to v2.3.0 rather than judged by when it was noticed. Two were already shipped and are now in Fixed. v2.3.0's hexpair.vimrc listed eleven of the fourteen options its plugin read and seven of the eight highlight groups - and one of the missing three, g:hexpair_insert_encoding, is MENTIONED in prose in that same file, so a reader learned the option exists and then could not find the line. And the README's opening picture has never rendered for anyone who unpacked a release: it is a relative link to a 5.7 MB GIF that no tarball has ever carried. Three were introduced and fixed inside this cycle - the help never getting <Plug>(HexPairModified), hexpair.vimrc never getting g:hexpair_scan_block, and the new minimal package's README linking to the three files that package omits - so by the same rule they name no released code and go to CLAUDE.md's bug table instead, with the checks that now catch them. They are one failure written three times: what is added last is added to whichever files the author had open. The .reg generator's --help goes there too. It is development-only and ships in nothing, so it is not a changelog entry, but it did overwrite two released files and the guard is worth naming. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…e repo The previous commit made four of README.md's links absolute in the repository, which fixes the tarball by making the file worse everywhere else: on GitHub and in a checkout a relative link is the right form, and those four are files sitting next to the README. So the repository keeps them relative and pack-release.py rewrites them as it packs - per ARCHIVE, which is the part a one-off edit could not do. CHANGELOG.md, CLAUDE.md and CONTRIBUTING.md stay relative links in the complete tarball, which carries them, and become URLs in the minimal package, which does not. demo/hexpair-demo.gif becomes one in both, since 5.7 MB ships in neither. Images get raw.githubusercontent.com and everything else the blob viewer. Pointed at main and not at the release tag: a tag exists only after step 3 of the release process and never for a -devel build, so a tag URL would ship 404s from every intermediate package - and for a changelog or a contributor guide the current file is the more useful answer anyway. The check moves with it. It reads the PACKAGED documents now, building both archives in memory and holding every relative link in every .md they contain to resolving inside that same archive - which checks the rewriting, where reading the repository's own files only checked the file list. Removing the rewrite fails it with all five links named. Only README.md changes; no other shipped document has a relative link that leaves its archive. Both archives stay byte-identical across runs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A package is a fixed set of bytes. A link out of it should reach the files those bytes were built beside, not whatever main has become by the time someone unpacks it - so the rewriting names the release tag. The tag comes from the version in the plugin header with "-devel" removed: that suffix is this project's marker for the cycle in progress and no part of any tag, so 2.4.0-devel packs links to v2.4.0. Any other suffix is left alone, and a 2.4.0-rc1 would link to v2.4.0-rc1. The cost is that a package built mid-cycle carries links that resolve only once the tag is pushed, in step 3 of the release process. That is the right way round: the packages that reach users are release packages, and theirs point at exactly the tree they were cut from. Pinned by a check, because it is a guess about a name that does not exist yet when the package is built - the derivation over four version shapes, and the tag that actually comes out in the packed README. Get it wrong and every release ships dead links that nothing else here would notice; the check fails on a packer left pointing at main, and on a version whose suffix is mishandled. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.