Repository navigation
Fix: cairn prime --budget-bytes bounds only triggers, payload runs 2.5x over (crn-s3749) - #136
Merged
Merged
Conversation
added 2 commits
August 27, 2026 12:47
…5x over (crn-s3749)
budgetBytes was enforced only against Items. TopicCounts was exempt as part of
the "index view", justified by a comment reasoning that it is "as cheap as
TotalVisible ... independent of what the byte budget lets into Items". That
reasoning is about COMPUTE cost -- NFR-2's no-I/O requirement, which still
holds -- and it was being used to exempt the field from a PAYLOAD BYTE budget.
Cheap to compute, expensive to emit.
Measured on the mayor's live store (287 entries visible), default flags:
topic_counts 18,350 chars 88.2% 278 keys, 277 of them count == 1
triggers 3,466 chars 16.7%
TOTAL 20,807 chars against an 8,192-byte budget
topic_counts is not an aggregation at that shape -- it is the topic-key list
wearing an aggregation's shape, growing by one row every time an agent learns
anything. Payload size was linear in distinct-topic count with nothing in the
path to bound it.
This makes budgetBytes bound the whole itemized payload:
- A reserve (budgetBytes/4) is carved OUT of the item budget before itemizing,
so items cannot starve the index to nothing. It is carved out rather than
added on top -- adding it would let items spend the full budget and push the
total over, which is the bug in a different place.
- Anything items leave unspent flows back to the index.
- The index is ranked count-descending before the cap, so topics holding more
than one entry -- the only rows carrying information the per-entry Items list
does not -- survive ahead of singletons. Kept rows still PRINT key-ascending,
preserving the ordering callers had.
- TopicCountsTruncated and a Warnings line report what was dropped. A silent
cap would be worse than the overrun it replaces: a short index would be
indistinguishable from a complete one.
The aggregate scalars (TotalVisible, fresh/stale/unknown) stay unconditional.
They are O(1) bytes, and bounding them would convert an honest overrun into an
undercount.
Verified against the same live store the bug was measured on:
20,813 -> 5,068 chars, 38 topics listed, 240 dropped and named in warnings,
total_visible still 287.
TestPrimeIndexBreakdownIndependentOfByteBudget is AMENDED, not deleted: its
budget=1 case asserted the old exempt-forever contract, which was the defect.
It now pins what FR-1/NFR-2 actually needed -- cost independent of entry
content size, scalars covering the whole visible set -- plus the new
report-what-you-dropped guarantee, and keeps a generous-budget case asserting
the full breakdown still lists everything when it fits.
CI lint failed on the previous commit with four issues, all introduced by it: cyclop (Prime at 16 > 15), intrange x2, and unparam on the test helper. Fixing cyclop by extracting applyIndexBudget then tripped funlen (81 > 80), because Prime was already at the limit before this branch touched it. - applyIndexBudget: the index-budget fill plus its report-what-you-dropped warning, which belong together anyway. - orderByPriority: the itemization comparator. Load-bearing and worth naming -- byte-budget truncation walks this slice in order and stops at the first entry that does not fit, so the ordering decides what survives a cap. - Test helper drops its always-200 parameter for a named const, and the two index loops use Go 1.22 range-over-int. No behaviour change: verified against the same live store, 5,068 chars, 38 topics listed / 240 dropped, total_visible 287. golangci-lint run ./... reports 0 issues and the full suite is green. Lesson worth recording: golangci-lint was installed locally the whole time and I pushed without running it, so CI found what a 30-second local command would have.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes
crn-s3749.The defect
--budget-byteswas enforced only againstItems.TopicCountswas exempt as part of the "index view", justified by this comment:That reasoning is about compute cost, and it is still correct — NFR-2 holds. It was being used to exempt the field from a payload byte budget. Cheap to compute, expensive to emit.
Measured on the mayor's live store, 287 entries visible, default flags:
topic_countstriggersAt that shape
topic_countsis not an aggregation — 278 keys, 277 withcount == 1. It is the topic-key list wearing an aggregation's shape, gaining a row every time an agent learns anything. Payload was linear in distinct-topic count with nothing in the path to bound it.This matters because
primeis aSessionStarthook: the overrun is paid on every session, forever.The fix
budgetBytes/4) is carved out of the item budget before itemizing, so items cannot starve the index to nothing. Carved out rather than added on top — adding it lets items spend the full budget and pushes the total over, which is the same bug one line later. (I wrote it that way first; the test caught it at 2459/2048.)Itemsdoes not already have — survive ahead of singletons. Kept rows still print key-ascending, preserving the ordering callers had.TopicCountsTruncatedplus aWarningsline report what was dropped.A silent cap would be worse than the overrun it replaces, so it says what it dropped. A short index that looks complete is the same false-success shape the store warns about elsewhere.
The aggregate scalars (
TotalVisible, fresh/stale/unknown) stay unconditional — they are O(1) bytes, and bounding them would turn an honest overrun into an undercount.Verified on the store the bug came from
Full repo suite green.
One test is amended, not deleted — please review this specifically
TestPrimeIndexBreakdownIndependentOfByteBudgetasserted atbudget=1that every topic was still listed. That assertion was the defect — it pinned the exemption. It now pins what FR-1/NFR-2 actually needed (cost independent of entry content size; scalars covering the whole visible set), adds the report-what-you-dropped guarantee, and keeps a generous-budget case asserting the full breakdown still lists everything when it fits.If crn-3476 intended the enumeration itself to be unconditional, then this is a deliberate contract change and worth an explicit ruling rather than a silent merge.
Not changed
defaultPrimeBudgetBytesstays at 8192. The effective content of that budget changes here, so the default may deserve revisiting — but that is a sizing decision, separate from the accounting bug.Same measurement reported upstream in the Memory Beads discussion: gastownhall/beads#5877 (issuecomment-5443702739).