diff --git a/.github/pargates-shard-weights.json b/.github/pargates-shard-weights.json index f16c8b459..407adc134 100644 --- a/.github/pargates-shard-weights.json +++ b/.github/pargates-shard-weights.json @@ -293,6 +293,7 @@ "manifestcheck.sh": 62.9, "mapdatasectioncheck.sh": 55.0, "mapdiffcheck.sh": 3.3, +"mapinventorycheck.sh": 2.5, "matchcapturecheck.sh": 11.8, "matchgrammarcheck.sh": 0.6, "maxfilesizecheck.sh": 6.8, @@ -378,6 +379,7 @@ "oswin32logiccheck.sh": 30.0, "outlinecheck.sh": 1.5, "overbudgetcommentcheck.sh": 0.5, +"ownerhopcheck.sh": 53.2, "ownerscheck.sh": 5.2, "packcallersharecheck.sh": 0.9, "packtaskcheck.sh": 20.8, diff --git a/.ripwire_quality_acks b/.ripwire_quality_acks index e029c5a56..b5bcda42a 100644 --- a/.ripwire_quality_acks +++ b/.ripwire_quality_acks @@ -7,6 +7,7 @@ ack api-surface 060a064b6ffa7775 7 cid=605cbb1f768828e0 P2.2 register-macro dead ack api-surface 085e3d408c2c4a35 2 cid=55e7936512b0563d A3: TestRunnerIndex takes the run's crawl root so run= is spelled relative to root= (one absolute root per document, test/rootrelemitcheck.sh ARM 9). The parameter is the fix; every caller passes the root it already holds. ack api-surface 0a20676f246f053a 5 cid=d4f1ceec6378a259 t14-cleanup: heals stale ack-ledger identities (kParserVer moved since these rows were written) and acks this lane's own minor footprint — printSkillScanArtifact's +1 defaulted fullLegend param, packTaskBundleText's bodyless-placeholder clause, kHelpHead's --readability validation note, kCompactAttributeReadings' merge-scout anchoring= entry — all additive/self-contained, none foreign ack api-surface 0a3d16d6f3139408 10 cid=e2b9873df9888866 by=src/* answer-safe --edit-check window: the three contract-change rows are ONE defaulted paging pair (pageLimit/pageOffset, 0/0 = the verb's own default cap) plumbed through the ONE assembler and its two front doors — editCheckBundleText, editpreview::run, editCheckText — rather than a second capped emitter, because two emitters would drift and a page that drifted could drop the flagged caller that IS the answer. The complexity/verbosity growth in the assembler is the partition-preserving row loop (the window advances on UNFLAGGED rows only, so a flagged row and its sites_l= ride every page uncut) plus the in-band legend that says what pages and what never does; the churn rows are this lane's own footprint across cli.h/mcp*/editcheck*, none foreign. | prior: card A1 pre-apply preview: preview= is a DEFAULTED flag on the ONE edit-check assembler rather than a second emitter — two emitters could drift, and a preview that disagrees with the post-hoc answer is worth nothing (test/editpreviewcheck.sh compares the two documents byte-for-byte). The +20 LOC and the churn are the legend sentence that tells a reader the numbers describe bytes that were never written. +ack api-surface 0d64cc4708da0ab4 5 cid=104918b8416000b6 now=5 was=3 facet=contract-change p=src/ingest_crawl.h:1656 cli-perf-068: probeIgnoreSet is a section-local function of the ingest translation unit (ingest_crawl.h) with one caller, collectSources. It gains the ingest's IgnoreProbeSpeculation (the git ignore probe running beside the walk, checked against the fresh answer before the parse) and the canonical root the speculation's sidecar is keyed by; both are passed by collectSources, whose own new parameter is defaulted. No other caller exists to break. ack api-surface 0d6e1106ea5949d5 4 WAVE-2 close (2026-08-19), finding 3 of 3: the 76 remaining gating rows, ONE change. All of them are W2-E's root-relative p= landing (9beaa2c/fccea68/a271e6c/b3fe074 plus the f9108b7 correction), measured for the first time at WAVE granularity. The per-lane acks written during W2-E covered only the correction round's own diff (working-tree-vs-HEAD at that moment), so the original ~30-verb landing was never QD-acked; this ack closes that gap rather than re-accepting anything. Verified by reading the whole 20cdc04..860291c src/main.cpp diff line by line: 282 of 448 added lines match the root-relative predicate directly and every one of the remaining 127 is an existing std::printf rewritten from ing.files[...] to the root-relative rp local, plus four extracted emit helpers (computeDirModules, printJsonSymbolRows, writeOversizeRows, writeDropRows). No unrelated logic rides in. By kind: api-surface 28 = the +1 rootArg/rootPrefix parameter on the emitters that must now be TOLD their root (writeAbiCheck/Ref/Struct, emitColumnar*, packBodies/Deps/Lego/Outline/Signatures[Json], serialize[Json], writeLayout*, packConnect, buildD1Row and the report writers) - defaulted wherever a caller could stay unchanged. complexity 31 and verbosity 16 = the single-root-condition ternary and its guarded root= clause applied per emitter, with no new nesting level and no new control flow beyond that one conditional; the large absolute numbers (runStructureText 207->231, runLint 313->323, runMaintenanceViews 174->190, runCallHierarchy 71->80) are pre-existing dispatcher size the wave adds to, not creates - decomposing them is its own round and is recorded as a wave-2 follow-up. params 1 = writeNonLocalStateReport 4->6, the same contract. The three sibling lanes are individually clean: --quality-delta at f6ec56d..1732fd8 (W2-J), 1732fd8..9a41c74 (W2-K) and 9a41c74..20cdc04 (W2-F) each report gating=0. Full suite green at this head: gates=429 pass=427 skip=2 fail=0, ASan+LSan clean, byte-deterministic, xmllint clean. churn= is unavailable in ref-pair mode by construction (both trees materialized out of the repo), so short-horizon-churn is silent here and that silence is not evidence. ack api-surface 1039e3c8e0fc3667 4 cid=4efcfe9cb7f739a2 M12 (capture-audit L9): path-spelling fixes — collectUseSites gained a root parameter (default-valued, back-compat) to root-relativize in_id=; runVerify grew from adding root=/verPathRel/the multi-root roots table it never had; short-horizon-churn rows are every function this finding's fix touched this session. ack api-surface 105c48e20c80c896 3 cid=720fab31ea99ebde A2: unmeasuredHintNote gained the AbsHintFrame parameter one commit after this lane introduced it (4db6fb3). It is a header-inline helper in namespace mcpedit with exactly one caller, resolveOneForEdit, in the same file; no consumer outside this lane ever saw the 2-arg form. The widening is what makes the never-parsed disclosure and the symbol scan agree about which files a hint names -- two copies of that rule is the defect this replaces. diff --git a/CHANGELOG.md b/CHANGELOG.md index f4e190411..6825b4d10 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -15,6 +15,158 @@ not published here — see `docs/EVALS.md` for the instruments behind the headli ## [Unreleased] +### Fixed — `--for`: the `` budget is charged for the ``/`` stub it serves, not the full render + +The default answer collapses ``/`` to a ~120 B counted stub, but the `` budget was charged the full +render before the collapse. On a corpus whose question names many interfaces, the full `` filled the budget: one +measured Python corpus served 4 of 40 signature rows in a ~4.6 KB answer under the 7.5 KB ceiling. + +- A section served as a stub is charged at the stub's size (the CLI lens and the MCP `for` tool share one rule). +- Without an explicit ceiling, `--sections=lego,compose` keeps the stubbed answer's `` budget, so the stub's `next=` + restores both sections with the same `` rows, byte-identically. Under `--token-budget` (MCP `budget_tokens`) + the restored sections are charged in full, so the hard bound holds. +- Measured on 32 `--for` questions over 8 corpora: 19 answers byte-identical, 13 carry more signature rows + (4 to 29-33 on the corpus above), and every row the old answer showed is still shown. Gate: `forsectioncollapsecheck` (13). +- Under `--token-budget` with `--with-graph` or `--detail`, the `` budget now leaves room for the graph block, or for + the first `--detail` body (whole up to a quarter of the budget, else cut with a `next=` for the rest). Before, they rode + past the budget and the first body shrank to its first line. With BOTH flags nothing is reserved: on the measured + questions that cut a needed signature row, so those answers keep their rows, serve the first body at its floor, and say + `over_ceiling="1"` when they overshoot. Without `--token-budget` nothing changes. Gate: `estchargecheck` #11 A7R. +- By-design break, owner ruling 2026-10-09: `--token-budget=2000 --detail=20 --with-graph` on this repo's `src` no longer + fits its allowance (6029 B vs 5428 B). It keeps every signature row the same budget shows without the two flags, says + `over_ceiling="1"`, and the cut first body carries a `next=` that returns exactly the lines it cut. The over-budget note + now names the kept rows, the graph block and the first body's floor. It used to blame the header floor, which was false + here: the header alone is ~1.8 KB. The old bar still holds for each flag alone and for both flags when the rows fit. + Gates: `estchargecheck` #11 A7 twin / A7B / A7R kill+next, `ceilingverdictcheck` (2b). +- Under `--token-budget`, a spent ceiling no longer drops the compact `` section without a trace: it ships as + ``, where `next=` is the same question without the budget. A partly cut + `` carries the same `next=`, and a cut `` carries the `--for … --limit=N` file page. These bytes are reserved + from the signature rows, so measured at 2000 tokens over 32 questions: 192 hop rows named on 32/32 answers (none before), + 65 signature rows fewer (still disclosed by ``'s own `next=`). Gates: `estchargecheck` #11 A7D, `fordisclosurecheck` + #1b/#1c. + +### Changed — `--for`: the function a question names gets the first hop row, with all its callee names + +The compact `--for` answer hop-expanded only the six best-ranked rows, inside a ~1 KB allowance. A question that names +the function it is about ("How does the JWT middleware verify a token?") often ranks that function below a dozen symbols +that repeat the question's other words, so its callees — the names an agent needed next — were never served. + +- A ranked function or method DEFINITION (not a test symbol, not a C prototype) whose whole name, leading `_ # $ @` + removed and ASCII case folded, equals a word of the question is that question's owner, when the word has such a + definition in the top 24. It gets a `` row first in ``, past the six-row cap; every same-named + definition in the ranked pool is its own row. At most three owner rows; the rest are disclosed as + ``. A small list of common English words (get, then, write, …) ranks after the + other owner words, so verbs a question merely uses do not take the owner's slot. +- An owner row lists its callees up to 100 names (a runaway guard); past it, or under an explicit budget that cuts it, + `` names the full list. The hop-slot rule still applies: an owner with no proven callee + edge has no row and is counted in `noedge=` once. Name-only callees keep `via="name"`. +- By name, not by resolution: `qword=` says the name equals the word, never that the symbol is the only or the true + implementation. Default regime: owner rows ride on top of the hop allowance, so every earlier `` row is still + served (a moved owner can only free room), and ``/`` are unchanged: the legend clauses the owner rows + switch on (`qword=`, and `via="name"` when only an owner row carries it) are not charged to the signature budget. Under `--token-budget` they are funded first. A question that names no + owner, `--json --for` and the MCP `for` verb are byte-identical; `RIPWIRE_NO_OWNER_HOP=1` restores the previous + answer. Measured on the 32 `--for` rows of the round-1 comparison set: the two in-scope rows now name 5 of 5 gold + callee items they missed (were 0 of 5); 23 of 32 answers are byte-identical, 1 differs only in `noedge=`, and 8 grow by 0.2–1.1 KB (+1.4% over all 32). Gate: + `test/ownerhopcheck.sh`. + +### Changed — `ripwire wrap` recommends `--for` at its default budget + +The CLI-first paste block (claude, codex, opencode, openclaw, hermes) and the aider repo-map line now read +`--for=""` instead of adding `--token-budget=2000`. That budget is a ceiling. Over 40 `--for` questions in +eight open-source repositories (one release binary, `--no-cache`) it halved the median answer (4.6 KB against 9.2 KB). +It dropped signature rows on 35 questions and call-hop rows on 24. Only the signature cut carried `capped="1"` and a +`next=`. The file tail and two partial hop cuts carried `capped="1"` alone, and on 24 questions the whole call-hop block +was left out with no marker (22 of them had hop rows). The default budget and every other knob are unchanged, and +`--token-budget=N` still works when passed by hand. Only the printed `wrap` text changes. +### Changed — `--expand` / `--outline` ship no ride-along map when every definition they name is served + +`--expand=NAME` on a name with several definitions, and `--outline=NAME`, used to carry the generic top-200 ranked map +beside bodies that already answered the request (on this repository `--expand=subtokens`: 26,752 B, of which 19,444 B +was map rows that never mention the symbol; `--expand=emitTo` served the whole 24,788 B file because the whole-file +comparison priced the bundle with that map). When every requested definition is served the map now defaults to +`top-k=0`: the root carries `topk_default="0"` and `map_next="ripwire ROOT..."` (the call that prints the map, for a session +that skipped orienting: every root of the call plus its crawl-scope flags, shell-quoted). When a byte budget cut some +definitions the root names exactly those in `unserved_total=N` (the true count) and a shell-quoted +`unserved_next="--expand=FILE:LINE:NAME,..."` instead of shipping the map; past 16 cut definitions it serves the first 16, +`unserved_listed="16"` says so, and a larger `--pack-budget-bytes` serves the rest. The whole-file comparison therefore prices the +payload only. An explicit `--top-k=N` keeps the previous shape. Language-neutral: the decision reads the record of which +requested definitions were emitted. + +### Fixed — the cross-branch verbs refuse when the blob batch cannot run + +`--stray-content`, `--stray-content --abi`, `--stray-content --plan`, `--eval-stray` and the MCP `stray_content` tool read +every branch's content through one `git cat-file --batch` whose sha list is written under the cache directory. With `TMPDIR` +naming a directory that does not exist the list could not be written, every blob read as absent, and the answer was +`merged="1"` for a branch holding unmerged work (`--abi`: exit 0, no drift). Each now exits 1 (MCP: `-32603`) naming the +cause and `TMPDIR`, with nothing on stdout, the way `--whereis` refuses on the same condition. + +### Fixed — a git history walk that could not be read says so, and is retried + +A `git log --name-only` that failed (an unreadable object, a bad ref) came back as an empty stream or as the newest part of +the window and was stored under the (repo, HEAD, window) key, so `churn=` and `amp=` read as "no change history", or as the +whole window's count, until HEAD moved, with no tell. A git that would not run at all (not on `PATH`) in a repository read +the same way, as an unborn branch. A walk git could not finish is now never cached, so the next call walks again (a blob an earlier build stored from such a +walk is not read either: the history-walk cache scheme moves, see the versions note), and every +document that prints `churn=` or `amp=` (the map under `--metrics`, `--for`, `--pack-task` and `--grep` with `--metrics`, +`--around`) carries `history_unread="1"` on its root (`"history_unread":true` in JSON) beside a comment that defines it and +names the cause. It means `churn=` and the co-change half of `amp=` count only the commits that were read, so both are +floors: when git did not run, `churn=` is absent and `amp=` counts callers only; when git stopped part-way, they count the +newest part of the window it read. It does not mean the other numbers are affected, and a walk that read an empty window (a +repository with no commit) stays silent; a HEAD that is corrupt rather than unborn still reads as that empty window. Not yet +covered: `--cochange` (and the MCP `cochange` tool), `--rank-by=churn` and `--owners` run their own git walks and do not yet tell a +failed one from an empty one (`commits="0" partners="0"`; "found no commits in its window"; exit 1 with "git unavailable / +no history"). + +### Fixed — a non-ASCII `--help-task` prompt no longer trips the sanitizer build + +`lowerAscii` converted every byte of the prompt through an implicit `char` to `unsigned char` conversion, which the +sanitizer build (`implicit-integer-sign-change`) aborts on for any byte at or above 0x80. The same shape sat in the +router's name-shape and `FILE:LINE` predicates, the LSP header parser and the extension test; each now narrows +explicitly. + +### Changed — `--whereis` labels another ref's row by parsing its blob, lists a line several refs share once, and reads refs from tree objects + +A row on another ref used to take `kind="def"` from a lexical reading of its one line, and any such reading errs both ways: +a looser one called call sites definitions (`if( !task.empty() ) { out += escapeXml( task, esc ); }`, `go f(a)`, `@f(1)`), +a stricter one lost real definitions (`#define NAME(`, `decltype(auto) NAME(`, `@Test(…) public void NAME()`, +`#[derive(Debug)] struct NAME`). Measured on this repository (162 local refs), `--whereis=escapeXml`: the default page +went from 60 rows of `total="2197"` (54 of them call sites) to 2 rows, both the declaration. + +- **Another ref's `kind="def"` comes from a parser.** A ref holding HEAD's own blob of a path takes HEAD's index labels, + line for line. Any other blob is parsed once per distinct blob and path by the index's own extraction — the same + language rule, tags query and symbol model — and a definition it captures for the name labels the nearest row the way + HEAD's index labels HEAD's. Only a blob holding a line a loose lexical test calls worth parsing is parsed; a blob + without one reads `kind="ref"`. The root says `ref_labels="parsed"` whenever another ref's row is in the answer. + Arms: one well-formed file per language family (C, C++, ObjC, Python, TypeScript, JavaScript, Java, Ruby, Go, Rust, + Swift, C#, Bash, Markdown) whose every marked line reads as its marker says and as HEAD's index reads the same bytes; + the twelve definition shapes the stricter reading lost; a branch-only `#define` and annotated Java method. +- **A runaway guard on that parse, disclosed.** At most 256 blobs or 64 MB per answer, most rows first, under every + listing, the default page included. A blob the guard leaves, or one the parser cannot finish, confirms nothing: its + parse-worthy lines read `kind="text"` (listed beside the definitions, never `kind="def"`) and an `` element counts them; `--detail=1` lifts the guard. Its `next=` replays the same question with + `--detail=1` (an explicit `--whereis-listing=` rides with it); an MCP `kind` ref filter is not carried, as `` does not carry it: the CLI has no whereis ref filter. +- **A definition line several refs hold is one row.** Under `listing="defs"` a `kind="def"` (or `kind="text"`) row outside + the checkout whose path and line text other refs hold too is printed once, for the first of those refs by name, with + `refs="N"`; the root's `folded=` counts the rows it stands for (on the first page, `shown + more + folded + refs count = + hits`). `hits=` still counts every copy and `--whereis-listing=all` prints each ref's own row. +- **Faster.** Every ref's tree is read from git's content-addressed tree objects, each distinct tree once and a level at a + time through one `git cat-file --batch`, instead of one `git ls-tree -r` per distinct tip (the same rows in ls-tree's + order and spelling — an arm compares quoted and unquoted names with ls-tree itself); a blob several refs share is + scanned once. The parallel per-tip listing stays the fallback, used when ripwire runs on a directory below the top of + the work tree. + +What stays a floor: a definition the loose lexical test misses (a signature wrapped before its parameter list closes, a +name alone on the line under its return type, or a definition with no parameter list and no declaration keyword before +the name: `name: function (`, `pub const NAME`, a C# property) is a counted ref when nothing else in its blob earns the +blob a parse. A parsed blob is labelled as HEAD's index labels the same bytes, so a file its grammar cannot parse +cleanly (a fragment, a file mid-edit) can lose a definition or show a false one exactly as HEAD's index would. +MCP `whereis` takes no `detail` argument, so there the guard's `next=` names the CLI flag. Its description keeps its 701 +characters, the sentence calling a branch row's label lexical replaced by the parser one, and the +`tools/list` manifest shrinks by 5 B, to 47,109 B on this release's tree (the dropped em dash is six bytes of JSON; every +other tool is unchanged). Gates: `test/crossrefcheck.sh` (5b–5e), +`test/selectorhonestycheck.sh` (HEAD's rows, labelled by the index), `test/pagingsweepcheck.sh`. + ### Changed — a call bound by name alone keeps its rows, marked `via="name"`; typed receivers resolve A member call `x.m()` (or, where the receiver is implicit, a bare `m()`) bound to every in-repo definition spelled @@ -69,7 +221,7 @@ in a Java class with an outside base drawn to an unrelated class's `flush`. change); the `--max-tokens=3000` maps of the eight repositories keep 337 rows (331 before the change). The gate is `test/receiverevidencecheck.sh` (every surface, CLI/MCP parity, census conservation, and near misses for every evidence rule). Ingest records the receiver chain and the Java/C#/Kotlin/Swift declarations - (`kParserVer` 156 in this release, see the versions note; `kCacheVersion` 29), so a cache written by an earlier build is re-parsed. + (`kParserVer` 158 in this release, see the versions note; `kCacheVersion` 29), so a cache written by an earlier build is re-parsed. ### Added — `--quality-delta` gains a twelfth kind, `defect-shape`: four defect shapes code review kept finding by hand `kind="defect-shape"` rows name a known defect SHAPE the change added, in `defect=` (`src/defectshape.h`): @@ -121,6 +273,92 @@ can be a false format-arity row; and only a literal `#if 0` is skipped — `#if `#else` of `#if 1` are read as live code. The qsnap blob scheme moves (19 in this release, see the versions note); the legend dictionary gains `r defect=`. +### Changed — a warm CLI call costs about a quarter less, `--whereis` on a many-branch checkout about 70% less, and `--whereis` refuses instead of printing a zero it never measured + +Every answer is byte-identical to before except the one `--whereis` refusal below. Measured on 2026-10-08 on a +10-core arm64 Mac, Release builds, median of 5 warm calls, against main at 60dd3b3f: + +| call | before | after | +| --- | ---: | ---: | +| `--callers` on a 3,100-file snapshot of this repository, no `.git` | 319 ms | 269 ms | +| the same snapshot as a one-commit git repository | 437 ms | 341 ms | +| five verbs on this repository's checkout (median of medians) | 448 ms | 342 ms | +| `--whereis` on a checkout with 139 local branches | 12.3 s | 3.7 s | + +- **The git ignore probe runs beside the crawl.** Every call on a git checkout asked `git ls-files --others --ignored` + before the crawl could start. The crawl now walks with the previous call's answer for that root while git computes + this call's, and the two are compared byte for byte before anything is parsed; when they differ, the crawl and the + cache load are redone on git's answer. Every answer is still git's own answer for this call. Under `--no-cache` the + probe runs before the crawl, as it did. +- **No shell is started for a markitdown that cannot run.** The doc pass started `/bin/sh -c markitdown …` for every + pdf/docx/pptx/xlsx on every call, and on a machine without markitdown the shell's "not found" was never cached. The + PATH is now searched the way the shell searches it, once per ingest, so installing markitdown is still noticed on + the next call. +- **The resolver reads dense arrays for a name with many definitions.** A module-scope call has had a caller node since + 0.6.2, so the test scripts' top-level `ok "…"` calls (about 650 shell definitions of `ok`) reach the candidate walks. + Those walks now read per-symbol arrays instead of whole symbols. +- **`--whereis` reads each git listing in blocks and lists each distinct tip once**, the listings in parallel, and + scans each blob once however many branches share it. + +### Fixed — `--whereis` refuses when it could not read the blobs, instead of answering `hits="0"` + +`--whereis` reads every blob through one `git cat-file --batch`, whose sha list is a file under the cache directory. +When that directory was unusable (for example `TMPDIR` naming a directory that does not exist), the batch never +started and the answer said `on-head="0" hits="0"`, with a near-miss note suggesting the name it had been given, on +a repository whose HEAD defines it. The CLI now exits 1 and names the cause on stderr, and the MCP `whereis` tool +answers error -32603. A genuine zero over a working cache directory is answered as before. +The cross-branch verbs read blobs the same way and now refuse the same way (the entry above); both verbs, CLI and MCP, +say it in one sentence that differs only in what was not read and names the cache directory and `TMPDIR`. + +### Added — a Ruby `db/schema.rb` renders its columns as Section definitions +A Rails schema is recognized BY CONTENT, never by path: a Ruby file whose tree holds a +`create_table "x", … do |t| … end` call — the call's do-block carries one block parameter — is a +rendered schema, wherever it lives. Each `t. "name"` / `t. :name` in a table block mints +ONE `SymKind::Section` def at `Lang::Ruby` — the data-kind slot a doc heading or YAML key already +occupies (model.h: a Section's isolation is a language property; the Ruby ones are still no call +edges) — span = the name token, both quote spellings and the `:symbol` form. Rails-generated attribute +uses (`product.price`) then resolve to the column as a definition: it appears in `--uses`, `--grep`, +`--whereis` and the map. THE COLUMNS ARE DEFINITIONS ONLY, by maintainer decision: buildGraph's +byName skips Section-Ruby, so a column admits NO call edge (it still takes the restart share of the ranking +that every definition gets, so a Rails app's `k=` values are smaller; its map rows do not change) — an untyped +`response.code` must not become a table's caller, and 200 `name` columns must not damp a model's real +`def name` (both measured at review; edges may return behind an evidence rule, e.g. a receiver proven +to be the owning model). The id rule covers the grounded spellings: no id column/spelling → an +implicit `id` def (the anchor is the table-name string — no `id` token exists there); `id: false` → +no id def; `id: :uuid` → an `id` def at the pair key (the uuid is a TYPE of the id column, never a +column of its own; the type is not modelled); `primary_key: "x"` → a def named the string instead. +An ARRAY or symbol `primary_key` (a composite key, or a hand-written migration) mints NO implicit id — +the rendered composite columns are the block's own explicit `t.` lines. `t.timestamps` mints +`created_at` AND `updated_at` (the DSL names them literally, symmetric with the id rule) — the two +defs anchor at the method token's edges. The DSL CALLS keep their reference posture exactly like the +attr family: `string`/`datetime`/`create_table` stay external-surface names. Duplicate columns across +tables stay SEPARATE defs (same name, distinct identity bytes): `id` on N tables is `defs="N"`, and +the map row merges them with `overloads`, never hiding the multiplicity. MIGRATIONS never interfere: a +class-wrapped create_table — the migration shape, string- or symbol-named — is refused by a +class/module-nest gate (a rendered schema's tables live at file level or under `Schema[].define`, never +inside a class body), so the rendered `db/schema.rb` stays the ONE source of column names and indexing +the migration beside its schema cannot double any def; `add_column` / `remove_column` / +`change_column` are argument data and read nowhere — a column added then dropped never registers +(pinned by the three migration fixtures). Rails 8's `db/queue_schema.rb`, `db/cache_schema.rb` and +`db/cable_schema.rb` are matched by the same content gate. Ingest records the schema columns (`kParserVer` 158 in this release, see the versions note; carried by the branch as +122 → 126 → 144 → 149 across its merges with `main`), so a cache written by an earlier build is re-parsed; no record +layout change (`kCacheVersion` and `kQSnapCacheScheme` unchanged). +Gate: `test/rubyschemacheck.sh` on `test/rubyschemafix/` — schema text from a real, running Rails +8.1 app's `bin/rails db:schema:dump` (verified `db:schema:load`-able), original domain tables scrubbed +to `spike_*` names; two spellings no dump emits are restored by hand and marked in-file (`id: :uuid`, +which that dump actually failed on, and a literal `t.timestamps`). Stated capture floors, each pinned +by a gate arm: `t.index` / `t.references` / `t.belongs_to` / `t.polymorphic` / `t.check_constraint` / +`t.exclusion_constraint` / `t.unique_constraint` name no column (a `t.references`/`t.belongs_to` mints +no `_id`, and a `create_join_table` mints nothing — Rails dumps render these as explicit columns); +a column call must be ON the block parameter (`helper.string "x"` names nothing); a symbol-named table +(`create_table :users`) is a migration spelling, not a schema surface; `where("price > ?")` string +fragments stay opaque; the picker splits honestly (`rec.name` on a plain local and a rich `X.new.name` +receiver are `graph_ambiguous`, `self.name` inside the defining class pins its own def). The +before-evidence: on the pre-change binary every column was `defs=0 external=1` — the fixture's +`--uses=name` read `defs="10"` (9 attr/def defs plus the YAML key) with the schema columns invisible; it +now reads `defs="19" external="0"` and the column uses resolve to definitions only. + + ### Changed — a Ruby call to self, or on an instance the code builds, answers from its own side: an instance never reaches `def self.m` The entry below left one floor open, (i). A call to self, or on a receiver the code builds, read a class's defs from both @@ -1445,6 +1683,24 @@ false-confidence rate, next-clue usefulness) and the rule that answering comes b gains §9.2, which fixes the two roles a size limit may have (runaway guard, stair-step target) and the capped-versus-uncapped grading rule. Documentation only; no behaviour change. +### Changed — Go import edges come from the shared `@import.path` capture (#358, Go slice) + +Go is the second language onto the shared import-capture vocabulary that the C-family slice introduced. A Go +`import_declaration` — single or grouped, aliased (`import f "fmt"`), dot (`import . "x"`), blank (`import _ "x"`), and +cgo's `import "C"` — is now matched by `(import_declaration) @import.path` in `queries/go/tags.scm` and read by one +`DepDialect::Go` normaliser in `src/ingest_importcap.h`; `captureIncludes` no longer walks Go files. The captured node is +the whole declaration because that is the unit the Include record has always been built from: its target is the +declaration's clause (`importClauseTarget`, shared with Swift, which still reads it from the walk), so a grouped block +stays ONE record and `resolveGoImport` still takes the first quoted token. Splitting a group into one record per spec, +and moving the alias onto `@import.alias`, would add rows and are left for their own slice. + +**Output is byte-identical.** Dependency and resolve output over the Go fixtures and repos in the lane report is +unchanged, the go.mod `module` / `replace` resolution reads the same records, and a cache written by the previous binary +is accepted untouched, so no parser-version bump is owed. `test/importcapcheck.sh` gains six Go arms (embedded capture, +21 byte-exact rows covering every form above plus tab, semicolon, raw-string, UTF-8 cut and error-recovery spellings, no +widening for a function-body import, replace-based resolution with a bogus-target negative, the import-role use-site +ref, and the Go cache round-trip). + ### Changed — the C-family import edges come from one shared capture instead of two per-language extractors (#358, C-family slice) `src/ingest_relations.h` held about 21 per-language import extractors (`csharpUsingTarget`, `phpUseTarget`, @@ -1495,6 +1751,53 @@ on both `--no-cache` and `--deps`. The new `test/importcapcheck.sh` pins the sha `#import` spellings, a macro include, guarded arms, both dead arms, the `#pragma`/`#error` gate, the import-role use-site half, cache round-trip, determinism) over a 14-edge fixture it builds itself. +### Changed — the TypeScript / JavaScript import edges come from the shared capture too (#358, JS/TS slice) + +The second language family onto the `@import.path` vocabulary the C-family slice introduced. `import … from 'x'`, the +re-exports (`export * from`, `export * as n from`, `export { a as b } from`, `export type { T } from`) and the CommonJS / +dynamic `require('x')` / `import('x')` are now three patterns on the one capture name in `queries/{typescript,tsx, +javascript}/tags.scm`, normalised by a `DepDialect::Web` arm in `src/ingest_importcap.h`. `captureIncludes` no longer +walks a TypeScript or JavaScript file at all, and the three branches of `directiveTargetOf` that read it are gone. +`jsModuleLoadTarget` stays: it is the text gate for the call spelling, and `captureJsImportFacts` (the bindings a +destructured `require` makes) still reads it. + +A query cannot say "the callee's text is `require`" (tags-pass predicates never run), so the pattern matches any bare +call with a leading string and the three guards that keep an ordinary call out of the graph stay C++: the callee is +the bare name `require`/`import`, there is exactly one argument, and it is a string. Two more things the walk owned +are re-derived from the captured node's ancestry, not dropped: the **container reach** (the walk entered only an +allowlist, so `require('x').y`, `[require('x')]`, `a ? require('x') : b`, a class method body and a `declare module` +body were never edges, and still are not) and the **lazy bit** (a `require`/`import` call written inside a function +body is `lazy="1"` on `--impact`'s import tier, from the same ancestor read that checks the reach). + +**Not changed, deliberately.** `import x = require('y')` (TypeScript) is not an edge, before or after: the grammar +gives it its own `import_require_clause`, which the walk never read and no pattern here matches. `@import.alias` and +`@import.names` are not used: the Include record of a JS/TS import names a module and nothing else; the bound names +are a different record (`captureJsImportFacts`, untouched). + +**Disclosure differences from the walk, in `--skipped` only** (the C-family slice's two, plus a third in the function both languages share). +`extract-partial` for import nesting now fires only when a captured import is actually cut at the bound. A `require('x')` +is itself a container to the old walk, so it announced at 254 and 255 nested blocks although the edge was kept; this +does not. A call the pattern captures but the `require`/`import` text gate drops (`foo('x')`) under 300 nested blocks is +not announced either, and neither is a C-family `#pragma once` under 300 `#ifdef` — the text gate now runs before the +bound test, in the one function both languages share, so an import was never cut. In the other direction, an import +under 300 nested blocks inside a class method is announced (the ancestor read reaches the bound before it can learn the +walk would never have entered the class). All three are pinned in `test/importcapcheck.sh`. + +**The ancestry is read in one descent from the root.** `ts_node_parent` is itself a descent from the root, so walking up +k ancestors costs k of them. The first version of this slice did that, bounded to 256 hops, and a synthetic file of +2 000 nested `foo( "x", foo( "x", … ) )` calls took 152 s against 0.5 s for the walk it replaced (Apple M2 Pro, one run +each, loaded machine). The final version reads reach, depth and the lazy bit off one `ts_node_child_with_descendant` +descent, after the text gate, and the same file takes 0.9 s. On three TypeScript repositories and the CommonJS tree of a +global npm install the `--deps` wall time is unchanged to within run-to-run noise. + +**Dependency edges are byte-identical.** Verified with both binaries against unchanged trees: `--deps`, `--uses`, +`--impact`, `--callers`, `--zoom`, `--report` and `--for` over three TypeScript repositories, the CommonJS tree of a +global npm install, two npm cache trees, a `.astro` tree and this repository's `test/` fixtures. `test/importcapcheck.sh` +gains a JS/TS section (J1–J8): an exact edge list, in source order, for `.ts .js .tsx .mjs .cjs .jsx` and a `.astro` +frontmatter (68 edges, with the `neg_*` shapes that must stay out), the lazy bit, the use-site half, the depth bound at +254/255/256 blocks, the disclosure differences, and cache round-trip with a pre-change cache accepted unchanged +(no `kParserVer` bump). + ### Changed — `--affected` and `--test-gate` mark which tests to run first (`run_first=`) A tests-to-run answer listed every reaching test in evidence order (changed, partner, then `hops=` ascending), 80 to @@ -1613,7 +1916,7 @@ per page instead. Gate: `test/impactdepthcheck.sh`. ### Changed — the versions this release moves, stated once -`kParserVer` 124 → 156 (the function-literal fix takes 128; #338 and #325 take 129; the body-less C/C++ type-specifier +`kParserVer` 124 → 158 (the function-literal fix takes 128; #338 and #325 take 129; the body-less C/C++ type-specifier span fix and the TypeScript `await f(x)` / `!f(x)` calls each took a number of their own on their branches, as did the false-edge resolution (134, 135), the value-reference rows (140) and the Ruby method-lookup changes (#373: one per step from 130 to 137 — bare-word calls, mixins, typed receivers, RSpec targets, Rails declared calls, Rake and Jbuilder @@ -1623,12 +1926,15 @@ shadow and the value-reference slot text take 143, above 141's full-use file tag 148, renumbered from its branch's 145: other branch builds in flight have used up to 147, and 148's full-use file tag is 149. The Go named-type kinds and the train 25 re-review's extraction fixes each took 145 on their branches, and the receiver-evidence records took 145 to 154 on theirs; all three land together at 156: branch builds have used up to 154, -whose full-use file tag is 155, so no cache such a build wrote is read as this release's), `kCacheVersion` 25 → 29 +whose full-use file tag is 155, so no cache such a build wrote is read as this release's; the Rails `db/schema.rb` +columns (#339, carried as 122 to 149 on its branch) land at 158, above 156's full-use file tag 157), `kCacheVersion` 25 → 29 (the function-literal fix's record changes, then the false-edge fix's member-call fields, then the receiver chain `memberPath`/`memberCtor`; the Ruby branch's appended binding kinds need no bump of their own) and `kQSnapCacheScheme` 15 → 19 (the `--quality-delta` error-masking and placeholder changes, then the dead kind agreeing with `--dead-code` on functions -held as values, then on a function that only stores itself, then the defect-shape records of the twelfth kind). Every ingest cache written by an earlier build is refused and re-indexed once, and every -cached quality snapshot is recomputed. The session legend dictionary is `dictv=593769ab691623d8 entries=806`. +held as values, then on a function that only stores itself, then the defect-shape records of the twelfth kind), and the +history-walk cache's `kQChurnCacheScheme` 2 → 3 (an earlier build stored a `git log` walk that git could not finish as if it +were the window). Every ingest cache written by an earlier build is refused and re-indexed once, every +cached quality snapshot is recomputed, and every repository's history is walked once more. The session legend dictionary is `dictv=74f30d013d02c2a2 entries=829`. ### Fixed — a call the language resolves outside the tree no longer binds to a same-named in-repo definition @@ -2210,6 +2516,37 @@ five times, round-robin across the sizes, and compares medians. The thresholds a stall (one 640 KB run delayed 0.6 s) fails the old arm and passes the new one. A deliberately quadratic per-word rescan in `scanAsanWordBoundaries` still fails B1, B2 and B3 (medians 1033 / 14094 ms / timeout). +### Added — the budgeted-bundle candidate page: `--for`/`--pack-task` under a budget take a resumable `--limit`/`--offset` window; MCP `explore` gains `limit`/`offset` (issue #294, PR #362) + +`--for=TASK --token-budget=N` with an explicit `--limit`/`--offset` now answers with the **candidate +page**: one `` window over the same ranked candidate set the bundle cuts — the pageview quintet +(`shown=/total=/capped=/has_more=/next_offset=`) on the root, `above_cliff=` the head-tier count, +`tier=` one tier per page (a head window ends at the cliff), `r=` each row's GLOBAL candidate rank, +`at=` the git index answered from, `next=` the pasteable continuation argv (dropped when +`has_more="0"`), `over_ceiling="1"` when the budget cannot fit the window (fewer rows served, never +silently more bytes), and the page's own legend in both dialects (`--legend=full|compact`, the central +compact table). `--pack-task` under a budget takes the same window; a window with no budget stays the +`--for` file page; `--partition` and a window refuse together. The un-paged bundles are byte-identical +(`forbudgetmonotoncheck`), and `--offset=0` alone remains the un-paged answer — its resume point is +its own ``. + +MCP `explore` declares `limit`/`offset` (the same bounded pair `for` takes) and answers the same +page shape; previously it silently ignored `budget_tokens` beside a window and served the budgetless +file page. All five page paths — CLI `--for`, CLI `--pack-task`, MCP `for`/`explore`/`pack_task` — +page ONE candidate list: the CLI pages force the ranking's full distribution under a window (the +rule `--for`'s bundle path already followed), exactly the exhaustive scoring the MCP twins always +used, so the `ranking=` attribute the first round carried is gone — there is nothing left to +distinguish. Every page, CLI or MCP, carries the same pasteable `next=`/`next_tier=`: an MCP page's +handle is a CLI argv now, and it walks the same list. Every accepted ranking flag +(the CLI's `--no-route`, `--no-mention-boost`, `--no-doc-mention`, `--cochange-boost`; the twins' +`no_route`) rides `next=`, so a pasted continuation re-ranks identically — the +`test/pagingsweepcheck.sh` walk arms pin all five surfaces to `has_more="0"` with identical row +sequences and pin each flag's echo. The `tools/list` manifest grows 46,930 → 47,109 B (measured +on this release's tree: the +184 B of schema is `explore`'s `limit`/`offset` alone — `for` already +declared the pair, `pack_task` is a dispatch alias, not a listed tool — and `whereis`'s description +is 5 B shorter, the parser-label entry above), and the ceiling moves 46,950 → 47,150 B +- the one sanctioned case that moves it: `explore`'s two declared schema properties +(`test/mcpmanifestcheck.sh` arm `(1b)` asserts the figure against a live measurement). ### Fixed — the Linux G1 sanitizer ritual completes: five string_view comparator lambdas stop wrapping, and the GCC ASan path builds (#342) `LSAN_OPTIONS=… ./asan/ripwire .` — the sanitizer ritual AGENTS.md requires before a PR — aborted on any diff --git a/README.md b/README.md index 0f9f2afd6..f0f163196 100644 --- a/README.md +++ b/README.md @@ -190,6 +190,16 @@ you need — no follow-on grep, no three more whole-file reads to fill in what i followed by three greps is the same search paid for twice: it does not save you tokens and it does not make the coding faster. +**Two jobs, one goal.** An agent meeting a codebase does two things, and ripwire is shaped for both: + +1. **Orient — `ripwire .`** One call gives the ranked map: the main subsystems, the program entry points, + the symbols that matter most, and a count of every file the ranking left out. The most important + information about a new tree, in as few calls as possible. +2. **Ask — `--for`, `--expand`, `--callers`, `--path`, …** After that, each question gets a lean answer: + as few bytes as it can spend while still answering the question fully, so no follow-up grep is needed. + +That is the goal, and not every answer reaches it yet; the stair-steps below are how we get there. +
The two stair-steps that make it reachable — honest about what is missing, priced in what it spends @@ -215,6 +225,9 @@ clue where to look next. So every change is judged on more than "was it right": - **False-confidence rate:** wrong answers that claim to be complete — the worst case, held near zero. - **Next-clue usefulness:** for partial and wrong answers, whether following the answer's first suggested next step reaches what the question needed, within one hop. Graded blind. +- **Recoverability:** for every cut answer, whether following its `next=` returns everything that was cut. +- **Tokens per complete answer:** total tokens an agent spends across a scenario, divided by the questions it got fully answered. +- **Cut-row recoveries:** how often a needed item arrived only through a continuation. **Answering comes first; efficiency is how we get there.** We aim for answers about as lean as the leanest tools, but a byte budget is never allowed to cost an answer. We have gone too far toward short @@ -222,6 +235,27 @@ answers before and cut information that was needed, so now every size limit has runaway guard far above typical answers (it catches an output bug, and says so when it trips), or a stair-step target where anything over it must be explained. A capped answer is always measured against the same answer uncapped, and the one that answers better wins. + +**A cut is only honest if you can get the rest back.** Disclosing a cut is the floor, not the bar: a capped +answer plus the `next=` it prints must return everything the uncapped answer had. A test that checks this for every +budget option on every command is being built, so that "the answer was cut" always comes with "and here is the rest". + +**What a budget may spend first.** When a budget is tight, ripwire drops bytes that carry no answer before +it drops answers: repeated legend text, duplicated paths, bodies whose signature already says what you need. +Only then the lowest-ranked rows, each counted. A whole section never disappears without a marker. + +**Budgets are yours to set, never ours to impose.** The defaults and the setups `ripwire wrap` writes carry no +budget that cuts a typical answer; a budget is a choice the caller makes, with its cost disclosed. + +**Efficiency is counted per complete answer, across the whole task.** Bytes per answer can fall while the task +gets more expensive, if the agent then has to ask again. So the efficiency figure is tokens per complete answer +over a whole scenario — orient, then the questions that follow — not bytes per call. + +**Ranking is the tool's best guess; the next= is the safety net — and we measure how often the net is needed.** ripwire cannot always +know which row answers the question; it goes by ranking. When a needed item appears only after following an answer's next=, the +honesty layer saved the answer but the ranking put that item below the cut. We count those per round: +- **Recovered through next=:** the item was cut but recoverable — one extra call. Driving this down is ranking work. +- **Lost:** the item was cut with no marker or no working next= — a bug, fixed first.
### Same answer, a fraction of the tokens — read this table first if your agent is on a budget @@ -243,7 +277,7 @@ Ordered understand → navigate → review-the-change: | "Where is X handled?" | `ripwire . --for="…"` | **~2.1K tok** | ~4.9K–20K tok — `grep -rn src/`, then read the file it points at | 2.3×–9.3× | | "What do I already know?" | `ripwire . --recall="…"` | **~15K tok** | ~445K tok — read all 119 markdown docs this repo carries | 29.2× | | "Set me up for this task" | `ripwire . --pack-task="…"` | **~2.1K tok** | ~16K–80K tok — read every relevant file, whole | 7.7×–37.7× | -| "Show me this one function" | `ripwire . --expand=SYM --top-k=0` | **~260–16.5K tok** body (+~5.7K for the ranked-neighborhood bundle) | ~43K–174K tok — read the whole file it lives in | 2.6×–670× | +| "Show me this one function" | `ripwire . --expand=SYM` | **~260–16.5K tok** body (+~5.7K for the ranked-neighborhood bundle) | ~43K–174K tok — read the whole file it lives in | 2.6×–670× | | "Who calls this function?" | `ripwire . --callers=SYM` | **~580 tok** | ~40K–52K tok — `grep -rn SYM src/` (mostly noise), then open 2–3 files to sort real calls from mentions | 69.2×–89.1× | | "Is it safe to change this?" | `ripwire . --impact=SYM` + `--uses=SYM` | **~1.3K tok** | ~18K tok — open every direct-use file, whole | 14.4× | | "I have a stack trace" | `ripwire . --from-trace=FILE` | **~1.4K tok** | ~124K–298K tok — grep all 7 frame names, then open the innermost file(s) | 86.9×–208.6× | @@ -260,7 +294,7 @@ three `docs/ARCHITECTURE.md` names as central; the `--for` row lands `mcpStale` (`src/mcpindex.h:633`), the actual staleness check, 5th-ranked; `--recall` lands the container-rule doc (`AGENTS.md`) that states, verbatim, the same "no `std::map`" rule `CONTRIBUTING.md` explains in full; `--pack-task` names the same three touch points a human would — `cachelint.h`, `mergeCachePack` -(`src/main.cpp:1787`), the `lintrules.h` helpers it reuses; `--expand --top-k=0` hands back the +(`src/main.cpp:1787`), the `lintrules.h` helpers it reuses; `--expand` hands back the requested function's complete, unmodified body — the ranked-neighborhood addition costs the same ~22.6 KB regardless of which function you ask for, confirmed on two (a fixed floor, not per-function variance); `--callers` on `langOfPath` names its 2 real callers, the same ones a `grep` hit-list @@ -799,7 +833,7 @@ to do with it.** Two shapes get the most out of it, and one gets nothing. |---|---|---| | `ripwire . --for="…" --top-k=5` | `ripwire . --for="…" --signatures-only` (or `--token-budget=N`, `--detail=N`) | `--top-k` is inert on `--for`: the run warns on stderr and emits the full bundle anyway, so the agent *believes* it narrowed the output and did not. | | `ripwire . --query="…"` as the default lens | `ripwire . --for="…"` | `--query` is the raw BM25 ranking — the binary's own help calls it debug and says "use --for". It is the right tool for hunting a vocabulary, the wrong default for a task. | -| `ripwire . --expand=SYM` where SYM is an **ambiguous** bare name | `ripwire . --expand=SYM --top-k=0` (or name it exactly: `--expand=FILE:NAME`) | A multi-match name keeps the ranked map — there IS something to disambiguate — so ~9K est_tokens of map ride along with the bodies. An **unambiguous** single match already defaults to `--top-k=0` on its own (disclosed as `topk_default="0"`); no flag needed there. | +| `ripwire . --expand=SYM --top-k=0` (a habit from before the lean default) | `ripwire . --expand=SYM` | `--expand` and `--outline` ship no ride-along map whenever every definition asked for is served — one exact match or an ambiguous bare name alike (disclosed as `topk_default="0"`, with `map_next=` naming the call that prints the map if you never oriented). Definitions a byte budget cut are named in `unserved_total=` / `unserved_next=`. An explicit `--top-k=N` still brings the map back. | | `--callers=` expecting routes | find the URL in the project's own docs (e.g. a feature map), then `--expand` the handler | Framework route handlers have no callers in the graph — the decorator reaches them, not project code. Empty `--callers` on a handler is the design, not a bug. | | `ripwire …` (ONE root that happens to contain checkouts) | `cd` into ONE checkout first | Nothing refuses this: the crawl silently walks the nested repos and merges them into one corpus, so you pay for a map of everything and rank across unrelated codebases. Distinct from the real multi-root feature, which is N **explicit** positional roots (`ripwire dir1 dir2 … `, 2–16 checkouts merged on purpose). | @@ -948,9 +982,9 @@ Full retrieval tables — including the MRR figures behind the router numbers ab ## What it answers
-188 long flags across seven families, plus the MCP server — and --help-task names the ONE command a task wants, or abstains honestly when the evidence is too thin +189 long flags across seven families, plus the MCP server — and --help-task names the ONE command a task wants, or abstains honestly when the evidence is too thin -Around the core sit 188 long flags advertised in `--help`, across seven families — plus an MCP +Around the core sit 189 long flags advertised in `--help`, across seven families — plus an MCP server, so a coding agent can call any of them mid-task instead of grepping and reading whole files. `--help` prints one line per flag (~4.5K tokens); `--help=--FLAG` prints that flag's full entry with every caveat, `--help=SECTION` one family, and `--help=all` the whole catalog. @@ -1065,7 +1099,7 @@ every dependency is vendored in-tree, so the build completes with the network of git clone https://github.com/redhat-et/ripwire.git cd ripwire cmake -S . -B build && cmake --build build -j -./build/ripwire . --max-tokens=3000 # the ranked map — start here on an unfamiliar repo (bare it is ~22 KB here; this keeps the head at ~6 KB) +./build/ripwire . # orient once: the ranked map of an unfamiliar repo (add --max-tokens=N only if your context is tight — it cuts the map and says so) ```
@@ -1109,7 +1143,7 @@ skills/install.sh --codex # Codex CLI: the task-shaped skills that say whe Four commands worth learning first: ```bash -ripwire . --max-tokens=3000 # the ranked map — start here (bare it is ~22 KB / ~9K est_tokens on this repo; this keeps the head at ~6 KB / ~2.5K) +ripwire . # orient once: the ranked map of an unfamiliar repo (add --max-tokens=N only if your context is tight — it cuts the map and says so) ripwire . --for="incremental cache invalidation" # the task lens: what to touch, ranked ripwire . --callers=someFunction # who calls it ripwire . --test-gate # before you commit: which tests must run @@ -2085,9 +2119,9 @@ wrong, and it has. These are the results that say so, all in-tree, all published ### In the tests
-678 gate scripts, five contracts no unit test can hold, and the house rule: write the gate before the code it measures +681 gate scripts, five contracts no unit test can hold, and the house rule: write the gate before the code it measures -`test/regression.sh` names **678 gate scripts** and is the authoritative list; +`test/regression.sh` names **681 gate scripts** and is the authoritative list; `python3 test/pargates.py . ./build/ripwire -j 6` runs the same set in parallel. On top of them sit the contracts that do not fit a unit test: two runs byte-identical, warm output identical to cold, output that pipes clean through `xmllint --noout`, a sanitizer build with `-fno-sanitize-recover=all`, and a @@ -2609,12 +2643,14 @@ every request cross the network in the clear: put a TLS-terminating reverse prox Change to the repository. Run the following command: ```bash -ripwire . --max-tokens=3000 # start here on an unfamiliar repository +ripwire . # orient once on an unfamiliar repository ``` -The command writes the top of the ranked map within the token budget. Remove `--max-tokens=3000` -for the complete map. The complete map is larger. On a large tree, use `--top-k=N` to limit the row -count. +The command writes the ranked map: the main subsystems, the entry points, the most important symbols, +and a count of every file the ranking left out. Read it once per session, then ask lean questions +(`--for`, `--expand`, `--callers`). If your context is tight, `--max-tokens=N` keeps only the head of +the map; the cut is disclosed (`shown=` of `symbols=`), but it can drop what you need, so prefer the +full map when it fits. The first call on a tree parses it; every call after that reads a cache and answers in a fraction of the time. Judge the cost of the tool by the second call, not the first. If the first call fails, or @@ -2625,7 +2661,7 @@ identity of the index, and says which one is at fault. | Task | Command | | --- | --- | -| Orient in a new tree | `ripwire . --max-tokens=3000` | +| Orient in a new tree | `ripwire .` | | Prepare for a task | `ripwire . --for=""` | | Get everything for one task in a single call | `ripwire . --pack-task=""` | | Ask which command to run | `ripwire . --help-task=""` | @@ -2647,7 +2683,7 @@ identity of the index, and says which one is at fault. ### 5. Command families -The `--help` output groups 188 long flags advertised in `--help` into seven families. The `--help=` +The `--help` output groups 189 long flags advertised in `--help` into seven families. The `--help=` column below is the argument that prints one family: `ripwire --help=navigate`. `ripwire --help=--FLAG` prints one flag's full text — the caveats, the units, what it refuses and why. `ripwire --help=all` is the whole catalog, about 46,000 tokens. @@ -2841,7 +2877,7 @@ python3 test/pargates.py . ./build/ripwire -j 6 A new gate script must be added to `test/regression.sh` in the same change. The gate `test/manifestcheck.sh` enforces this rule. -Another gate derives the cap inventory. The tool has 240 compile-time caps and 7 ranking parameters. +Another gate derives the cap inventory. The tool has 245 compile-time caps and 7 ranking parameters. `docs/LIMITS.md` lists each cap, its value, and whether the file discloses a truncation when the cap fires, and `python3 docs/limits_build.py --check` proves that list against `src/`. `docs/TUNING.md` lists the measured cost of each cap. diff --git a/docs/COMMANDS.md b/docs/COMMANDS.md index b4d03761b..cb44daa21 100644 --- a/docs/COMMANDS.md +++ b/docs/COMMANDS.md @@ -9,7 +9,7 @@ python3 docs/docs_commands_build.py --bin build/ripwire The flag surface below is read from `ripwire --help`, so it cannot disagree with the shipped binary. `test/docscommandscheck.sh` fails if it ever does — in either direction. -Sample output is lifted from a real recorded run (`docs/captures/COMMANDS_showcase_2026-10-08.md`), trimmed to the first few lines and +Sample output is lifted from a real recorded run (`docs/captures/COMMANDS_showcase_2026-10-10.md`), trimmed to the first few lines and scrubbed of local paths. It is illustrative, not a golden: run the command yourself for the current shape. @@ -38,7 +38,7 @@ Two limits apply to nearly everything here and are not repeated in every section ## Contents -**understand a codebase cold** — [`--top-k`](#--top-kn) · [`--max-tokens`](#--max-tokensn) · [`--token-budget`](#--token-budgetnkmg) · [`--help-task`](#--help-tasktask) · [`--for`](#--fortask) · [`--sections`](#--sectionslegocompose) · [`--signatures-only`](#--signatures-only) · [`--auto-bodies`](#--auto-bodies) · [`--no-route`](#--no-route) · [`--adaptive`](#--adaptive) · [`--no-mention-boost`](#--no-mention-boost) · [`--no-doc-mention`](#--no-doc-mention) · [`--lego`](#--legotype) · [`--exemplar`](#--exemplartaskkind) · [`--recall`](#--recalltask) · [`--tree`](#--tree) · [`--html`](#--htmlfile) · [`--color-by`](#--color-bymode) · [`--order`](#--ordermode) · [`--no-stable`](#--no-stable) +**understand a codebase cold** — [`--top-k`](#--top-kn) · [`--no-inventory`](#--no-inventory) · [`--max-tokens`](#--max-tokensn) · [`--token-budget`](#--token-budgetnkmg) · [`--help-task`](#--help-tasktask) · [`--for`](#--fortask) · [`--sections`](#--sectionslegocompose) · [`--signatures-only`](#--signatures-only) · [`--auto-bodies`](#--auto-bodies) · [`--no-route`](#--no-route) · [`--adaptive`](#--adaptive) · [`--no-mention-boost`](#--no-mention-boost) · [`--no-doc-mention`](#--no-doc-mention) · [`--lego`](#--legotype) · [`--exemplar`](#--exemplartaskkind) · [`--recall`](#--recalltask) · [`--tree`](#--tree) · [`--html`](#--htmlfile) · [`--color-by`](#--color-bymode) · [`--order`](#--ordermode) · [`--no-stable`](#--no-stable) **navigate / answer a question** — [`--around`](#--aroundsym) · [`--callers`](#--callerssym) · [`--callees`](#--calleessym) · [`--uses`](#--usessym) · [`--graph-query`](#--graph-queryexpr) · [`--external-surface`](#--external-surface) · [`--path`](#--pathsrcdst) · [`--connect`](#--connectabc) · [`--impact`](#--impactsym) · [`--verify`](#--verifyclaim) · [`--mentions`](#--mentionssym) · [`--affected`](#--affectedf1f2sym) · [`--exercises`](#--exercisestestfile) · [`--situ`](#--situf1f2) · [`--handoff`](#--handoff) · [`--test-gate`](#--test-gatef1f2) · [`--grep`](#--grepstr----regexpat) · [`--grep-context`](#--grep-contextn----grep-beforen----grep-aftern) · [`--and`](#--andstr) · [`--not`](#--notstr) · [`--grep-scope`](#--grep-scopelinefile) · [`--grep-in`](#--grep-incodeany) · [`--handles`](#--handles) · [`--match`](#--matchquery) · [`--pattern`](#--patternpat) · [`--query`](#--queryterms) @@ -68,21 +68,21 @@ _Same map, capped to the 5 highest-ranked symbols._ ``` $ ./build/ripwire . --top-k=5 - + - - + + - + - + -... [11 more line(s); run it to see the whole thing] +... [17 more line(s); run it to see the whole thing] ``` **Shaped by:** `--token-budget`, `--for`, `--recall`, `--graph-query`, `--pack-signatures`, `--expand`, `--from-trace`, `--run-trace` @@ -92,6 +92,40 @@ $ ./build/ripwire . --top-k=5 - --for's OWN signature/lego/compose bundle self-limits via --pack-top-n instead — --top-k is INERT there (documented, not fixed — a real fix is a behavior change); - to WIDEN a --for answer use --limit=N, the file-grain page (one row per file), not --top-k. +### `--no-inventory` + +**Answers:** drop the default map's tier, the files and entries its ranked rows leave out. + +The default map appends after its ranked rows whenever they left out a file with code: rows (program entries by each language's own entry convention) and rows naming or counting every unshown file by directory. This flag drops it (the rows are unchanged either way). The tier never rides a map fitted to a token budget, nor a payload verb's ride-along map. + +**Try it** + +_The same capped map without its inventory tier: --top-k=5 leaves files out, so the default map appends (program entries, and the files the ranked rows left out) after its rows; --no-inventory drops that tier, and the ranked rows are the same._ + +``` +$ ./build/ripwire . --top-k=5 --no-inventory + + + + + + + + + + + + + + +... [11 more line(s); run it to see the whole thing] +``` + +**Caveats (stated by the binary):** + +- The default map appends after its ranked rows whenever they left out a file with code: rows (program entries by each language's own entry convention) and rows naming or counting every unshown file by directory. +- The tier never rides a map fitted to a token budget, nor a payload verb's ride-along map. + ### `--max-tokens=N` **Answers:** budget the map to ~N tokens (binary-search top-K) — SHAPES the map to fit. @@ -106,16 +140,16 @@ _SHAPE the map to fit ~1500 tokens (binary-search top-K)._ $ ./build/ripwire . --max-tokens=1500 - + - + ``` -**Shaped by:** `--token-budget`, `--recall`, `--detail`, `--pr-context`, `--from-trace`, `--run-trace`, `--limit` +**Shaped by:** `--token-budget`, `--recall`, `--detail`, `--pr-context`, `--from-trace`, `--run-trace` **Caveats (stated by the binary):** @@ -136,10 +170,10 @@ _GATE form: exit 3 if the map's own est_tokens exceeds the budget (over-budget f ``` $ ./build/ripwire . --token-budget=100 - + ``` -**Shaped by:** `--top-k`, `--max-tokens`, `--for`, `--recall`, `--handoff`, `--pr-context`, `--from-trace`, `--run-trace` +**Shaped by:** `--top-k`, `--max-tokens`, `--for`, `--sections`, `--recall`, `--handoff`, `--pr-context`, `--from-trace` **Caveats (stated by the binary):** @@ -173,7 +207,7 @@ $ ./build/ripwire . --help-task="write a cheerful release announcement" **Answers:** the task lens: say what you are doing, get the signatures and bodies to read first the task lens: ranked signatures + metrics framed for reuse. -The bundle enforces a ~7.5KB default payload budget (tail entries trim first; marks it: T rows handed to the trim, S printed). The r=1 (top-ranked) row carries next="--expand=FILE:NAME" — the one pasteable follow-up, the body that ends the search (defined here, not in the bundle's own header, which the token ladder does not charge). An explicit --token-budget=N overrides the default at the conservative byte rate (SHAPES, exit 0; see --token-budget above) and the header reports the delivered est_tokens. TERMINAL BY DEFAULT: after the signatures, the top-ranked symbols' FULL bodies ride inline (CDATA + callee signatures, the --expand shape) under a fixed extra body allowance — whole-body-or-not-at-all, rank-first, capped at the --pack-task candidate cap (6). The root discloses it: bundle="auto" bodies="N" (bodies="0" reason="budget" when none fit) — on EVERY auto-mode run: a ceiling the signatures alone exhaust still carries the attribute (legend and empty shell dropped there; only the attribute has reserved bytes), and --for --json, which serves no bodies by design, says so with "bundle":"sigs". Only the caller-chosen postures (--signatures-only, --detail=N) are attribute-free. ANCHOR-ONLY when the route names one: a query that NAMES a symbol gets THAT symbol's own body or NO body — never a same-named doc section, type stub or re-export shim from another file standing in for it. If the anchor's own body does not fit, the bundle serves nothing and says so, and the per-item over-budget comment names what was dropped. COMPACT ON THE CONCEPTUAL ROUTE: a query that anchors nothing (subtoken+body) gets the ranked map plus a section — the same candidate head's ONE-HOP callee signatures, the block a body carries — and NO body CDATA, disclosed as bundle="compact" bodies="0" reason="compact-route". Read the map, then --expand=SYM the one you want. --auto-bodies restores the body walk there. That shape discloses on every run too: a ceiling the signatures alone exhaust carries bundle="compact" bodies="0" reason="budget" — three distinct reasons, never collapsed (compact-route = the route chose edges, no_candidates = nothing scored, budget = the ceiling was spent). An explicit --token-budget=N is a hard ceiling, split so a wider ceiling never buys less: the signature side's claim is capped at the DEFAULT ~7.5KB sig budget and every byte beyond it flows to the enrichment — at any ceiling at or above the default's effective total the block is byte-identical to the default run's, so every body (or hop row) the default serves still fits. An explicit --pack-top-n is an explicit SIG posture and keeps the whole-ceiling sig claim. --compress composes: the served bodies (auto/anchor and --detail=N alike) go through the same comment-strip --expand uses, disclosed as compress="1" on the element (nothing to strip on the compact route). RANKING CONFIDENCE, disclosed not scored: the root always carries confidence="high|low" margin_pct="N" — derived from the SAME relevance-cliff gap statistic --adaptive cuts at (no new scorer, no behavior change; the --json dialect carries the same two keys). low means the ranking is FLAT (no material score cliff and more positive matches than the head shows) — treat the set as a starting point, not an answer; high means a material cliff inside the served head (margin_pct= is that drop as a whole percent) or every positive match already shown. COVERAGE, and the widening page: on a THIN answer only, the root also carries coverage="N" — the IDF-weighted share (whole percent) of the task's subtokens found in the top-ranked symbol's name, doc or body; an unmatched subtoken weighs as the rarest, so a "(#12147)" token honestly lowers it. THIN = coverage under 50, or a ranked head spread over fewer than 3 files: coverage= and its legend clause ride the root (present-only: a confident answer carries neither) and the r=1 row's next= names --for=TASK --limit=40 instead of --expand — the FILE-GRAIN WIDENING PAGE: …, ONE row per positive-score file (~100 B each), ranked file-first by score= = the IDF-weighted share of the task's subtokens the file's top 8 symbols cover between them (a term counts once however often it recurs, so one huge file cannot monopolise; ties by the best symbol's lens score, then path), n= its positive-score symbols, sym= its top symbols. --limit=N sets the rows, --offset=M pages (the house quintet + next= on a cut page). The page takes NO bundle-shaping flag (--json, --format=candidates, --detail, --signatures-only, --auto-bodies, --adaptive, --token-budget, --top-k …): each is refused beside --limit, never ignored. Measured on the routing-loop ladder (RocksDB, frozen 30): the follow-up that completes answers where a body cannot. / COLLAPSE TO A COUNTED STUB BY DEFAULT (L2, a disclosed cut, never a silent one): when the ranked set names an interface with implementors, or a HAS-A field edge into/out of it, the bundle used to spend the full // method contract or row list on it — the STUB instead says only how many rows the section held: (same shape for ). No stub for a section that would be EMPTY (an absent section stays absent). total= is that section's own pre-cap row count (packLego's post-dedup interface count; every matched HAS-A edge for compose, which caps nothing). next= carries the one restoring spelling that returns BOTH sections byte-identical to the un-stubbed render, in ONE call: --sections=lego,compose. +The bundle enforces a ~7.5KB default payload budget (tail entries trim first; marks it: T rows handed to the trim, S printed). The r=1 (top-ranked) row carries next="--expand=FILE:NAME" — the one pasteable follow-up, the body that ends the search (defined here, not in the bundle's own header, which the token ladder does not charge). An explicit --token-budget=N overrides the default at the conservative byte rate (SHAPES, exit 0; see --token-budget above) and the header reports the delivered est_tokens. TERMINAL BY DEFAULT: after the signatures, the top-ranked symbols' FULL bodies ride inline (CDATA + callee signatures, the --expand shape) under a fixed extra body allowance — whole-body-or-not-at-all, rank-first, capped at the --pack-task candidate cap (6). The root discloses it: bundle="auto" bodies="N" (bodies="0" reason="budget" when none fit) — on EVERY auto-mode run: a ceiling the signatures alone exhaust still carries the attribute (legend and empty shell dropped there; only the attribute has reserved bytes), and --for --json, which serves no bodies by design, says so with "bundle":"sigs". Only the caller-chosen postures (--signatures-only, --detail=N) are attribute-free. ANCHOR-ONLY when the route names one: a query that NAMES a symbol gets THAT symbol's own body or NO body — never a same-named doc section, type stub or re-export shim from another file standing in for it. If the anchor's own body does not fit, the bundle serves nothing and says so, and the per-item over-budget comment names what was dropped. COMPACT ON THE CONCEPTUAL ROUTE: a query that anchors nothing (subtoken+body) gets the ranked map plus a section — the same candidate head's ONE-HOP callee signatures, the block a body carries — and NO body CDATA, disclosed as bundle="compact" bodies="0" reason="compact-route". Read the map, then --expand=SYM the one you want. A ranked function or method whose whole name is a word of the query (by name, not resolved) gets the first rows, past that head: up to 3, each listing its callees to 100 names (capped=1 next= past it); more are counted as qword_cut= with next=. --auto-bodies restores the body walk there. That shape discloses on every run too: a ceiling the signatures alone exhaust carries bundle="compact" bodies="0" reason="budget" — three distinct reasons, never collapsed (compact-route = the route chose edges, no_candidates = nothing scored, budget = the ceiling was spent). An explicit --token-budget=N is a hard ceiling, split so a wider ceiling never buys less: the signature side's claim is capped at the DEFAULT ~7.5KB sig budget and every byte beyond it flows to the enrichment — at any ceiling at or above the default's effective total the block is byte-identical to the default run's, so every body (or hop row) the default serves still fits. An explicit --pack-top-n is an explicit SIG posture and keeps the whole-ceiling sig claim. --compress composes: the served bodies (auto/anchor and --detail=N alike) go through the same comment-strip --expand uses, disclosed as compress="1" on the element (nothing to strip on the compact route). RANKING CONFIDENCE, disclosed not scored: the root always carries confidence="high|low" margin_pct="N" — derived from the SAME relevance-cliff gap statistic --adaptive cuts at (no new scorer, no behavior change; the --json dialect carries the same two keys). low means the ranking is FLAT (no material score cliff and more positive matches than the head shows) — treat the set as a starting point, not an answer; high means a material cliff inside the served head (margin_pct= is that drop as a whole percent) or every positive match already shown. COVERAGE, and the widening page: on a THIN answer only, the root also carries coverage="N" — the IDF-weighted share (whole percent) of the task's subtokens found in the top-ranked symbol's name, doc or body; an unmatched subtoken weighs as the rarest, so a "(#12147)" token honestly lowers it. THIN = coverage under 50, or a ranked head spread over fewer than 3 files: coverage= and its legend clause ride the root (present-only: a confident answer carries neither) and the r=1 row's next= names --for=TASK --limit=40 instead of --expand — the FILE-GRAIN WIDENING PAGE: …, ONE row per positive-score file (~100 B each), ranked file-first by score= = the IDF-weighted share of the task's subtokens the file's top 8 symbols cover between them (a term counts once however often it recurs, so one huge file cannot monopolise; ties by the best symbol's lens score, then path), n= its positive-score symbols, sym= its top symbols. --limit=N sets the rows, --offset=M pages (the house quintet + next= on a cut page). The page takes NO bundle-shaping flag (--json, --format=candidates, --detail, --signatures-only, --auto-bodies, --adaptive, --token-budget, --top-k …): each is refused beside --limit, never ignored. Measured on the routing-loop ladder (RocksDB, frozen 30): the follow-up that completes answers where a body cannot. / COLLAPSE TO A COUNTED STUB BY DEFAULT (L2, a disclosed cut, never a silent one): when the ranked set names an interface with implementors, or a HAS-A field edge into/out of it, the bundle used to spend the full // method contract or row list on it — the STUB instead says only how many rows the section held: (same shape for ). No stub for a section that would be EMPTY (an absent section stays absent). total= is that section's own pre-cap row count (packLego's post-dedup interface count; every matched HAS-A edge for compose, which caps nothing). next= carries the one restoring spelling that returns BOTH sections byte-identical to the un-stubbed render, in ONE call: --sections=lego,compose. **Try it** @@ -181,19 +215,19 @@ _Name-shaped query: the router picks name-exact BM25 (header says which/why)._ ``` $ ./build/ripwire . --for="rankGraphTeleport" - + - + PageRank with an explicit teleport / personalization vector p (Σp = 1). The prior is name-quality-biased through biasPrior() so all rank modes share one weighting seam; the transition matrix (edgesinline RankedGraph rankGraphTeleport( const Graph& g, const std::vector<float>&am … [line truncated: 31 more bytes on this line] #### The convergence disclosure contract -### Wave-2 adversarial verification (2026-08-19) — six probes against `aa97c9e` +### Wave-2 adversarial verification (2026-08-19) — six probes against `aa97c9e` - + & p, float alpha = 0.85f ) ... [17 more line(s); run it to see the whole thing] ``` @@ -210,7 +244,7 @@ $ ./build/ripwire . --for="rankGraphTeleport" **Answers:** (with --for) opt back into the full / render the stub above replaces (with --for=TASK) restore the / sections the ranked bundle collapses to a counted stub by default — this IS the stub's own next= spelling, so pasting it back unchanged is enough. -A comma-separated, order-insensitive, closed set (lego and/or compose, each named at most once); every other section of the bundle is unaffected. +A comma-separated, order-insensitive, closed set (lego and/or compose, each named at most once); every other section of the bundle is unaffected — the budget is the stubbed answer's. Under an explicit --token-budget the restored sections are charged in full, so can carry fewer rows there. **Try it** @@ -218,14 +252,14 @@ _Restore the / sections the --for bundle collapses to a counted s ``` $ ./build/ripwire . --for="tree-sitter parse of a source file" --sections=lego,compose - + - - + + inline bool topLevelEvidence( std::string_view source, const TSLanguage* language, const Predicate& predicate ) + The crawl's per-file byte ceiling. A text file larger than this is skipped: at this size it is o…constexpr std::size_t kDefaultMaxFileBytes = 4u * 1024u * 1024u -TSTree* parseTree( TSParser* parser, std::string_view src ) +TSTree* parseTree( TSParser* parser, std::string_view src ) Exercise every registered grammar and its embedded query, reporting loaded and expected totalsinline DoctorGrammarProbe doctorProbeGrammars() ... [21 more line(s); run it to see the whole thing] @@ -245,14 +279,14 @@ _T3 opt-out: the signatures-only lens (no auto bodies, no bundle="auto" attribut ``` $ ./build/ripwire . --for="rankGraphTeleport" --signatures-only - + - + PageRank with an explicit teleport / personalization vector p (Σp = 1). The prior is name-quality-biased through biasPrior() so all rank modes share one weighting seam; the transition matrix (edgesinline RankedGraph rankGraphTeleport( const Graph& g, const std::vector<float>&am … [line truncated: 31 more bytes on this line] #### The convergence disclosure contract -### Wave-2 adversarial verification (2026-08-19) — six probes against `aa97c9e` +### Wave-2 adversarial verification (2026-08-19) — six probes against `aa97c9e` @@ -277,17 +311,17 @@ _Opt OUT of compact conceptual serving: restore the rank-first auto wal ``` $ ./build/ripwire . --for="tree-sitter parse of a source file" --auto-bodies - + - - + + inline bool topLevelEvidence( std::string_view source, const TSLanguage* language, const Predicate& predicate ) + The crawl's per-file byte ceiling. A text file larger than this is skipped: at this size it is o…constexpr std::size_t kDefaultMaxFileBytes = 4u * 1024u * 1024u -TSTree* parseTree( TSParser* parser, std::string_view src ) +TSTree* parseTree( TSParser* parser, std::string_view src ) Exercise every registered grammar and its embedded query, reporting loaded and expected totalsinline DoctorGrammarProbe doctorProbeGrammars() -struct FileHealth +struct FileHealth ... [20 more line(s); run it to see the whole thing] ``` @@ -310,16 +344,16 @@ _Same query with routing forced OFF (plain subtoken+body BM25) — contrast with ``` $ ./build/ripwire . --for="rankGraphTeleport" --no-route - + - - - + + + uniform-teleport PageRank (the defaultinline RankedGraph rankGraph( const Graph& g, float alpha = 0.85f ) - + PageRank with an explicit teleport / personalization vector p (Σp = 1). The prior is name-quali…inline RankedGraph rankGraphTeleport( const Graph& g, const std::vector<float>& p, float alpha = 0.85f ) -inline ChurnRanking churnRankedGraph( const MainDispatch& d ) - +inline ChurnRanking churnRankedGraph( const MainDispatch& d ) + L10 (2026-09-04): the old wording claimed "the same corpus ranked by pagerank orders differently…inline constexpr const char* kChurnRankLegend = "<!-- rank_by=churn: k= is PageRank re-run with the teleport BIASED by git CHANGE-FREQUENCY over window= " "(a c… ... [20 more line(s); run it to see the whole thing] ``` @@ -342,14 +376,14 @@ _Cut the result at the relevance cliff (Adaptive-k) — on a flat ranking nothin ``` $ ./build/ripwire . --for="tree-sitter parse of a source file" --adaptive - + - - + + inline bool topLevelEvidence( std::string_view source, const TSLanguage* language, const Predicate& predicate ) + The crawl's per-file byte ceiling. A text file larger than this is skipped: at this size it is o…constexpr std::size_t kDefaultMaxFileBytes = 4u * 1024u * 1024u -TSTree* parseTree( TSParser* parser, std::string_view src ) +TSTree* parseTree( TSParser* parser, std::string_view src ) Exercise every registered grammar and its embedded query, reporting loaded and expected totalsinline DoctorGrammarProbe doctorProbeGrammars() ... [21 more line(s); run it to see the whole thing] @@ -373,10 +407,10 @@ _Same task with the anchor disabled — the contrast the flag exists for._ ``` $ ./build/ripwire . --for="why does src/lexical.h chooseForRanker pick name-exact BM25" --no-mention-boost - + - + The name-exact ranker AS THE RETRIEVAL LENS SERVES IT: whole-name BM25 plus the definition-over-…inline std::vector<float> lexicalScoresNameExactRanked( const IngestResult& ing, std::string_view query, const std::vector<float>* symbolScoreMul ) @@ -404,11 +438,11 @@ _The same task with doc-mention surfacing OFF — the contrast the flag exists f ``` $ ./build/ripwire . --for="quality delta acks ledger rubber stamp" --no-doc-mention - + -inline QualityDeltaOutcome computeQualityDelta( const std::string& root ) +inline QualityDeltaOutcome computeQualityDelta( const std::string& root ) std::optional<int> refuseForeignAckSelection( const rw::Config& cfg, const rw::quality::Scope& scope, const std::vector<rw::quality::Regression>& outOfSc … [line truncated: 14 more bytes on this line] runQualityViews was NOT a dispatch chain — it held two branches, one of which was 298 lines. T…std::optional<int> runQualityDelta( const MainDispatch& d ) @@ -434,7 +468,7 @@ _Interface -> implementors view: every existing impl of the named interface; the $ ./build/ripwire . --lego=Vehicle - + @@ -464,7 +498,7 @@ _The repo's best-in-class instance to imitate before writing new code (picked by $ ./build/ripwire . --exemplar="format byte sizes for humans" - + @@ -658,7 +692,7 @@ _Stable (path/id) emit order — provider KV-cache hits across re-runs._ ``` $ ./build/ripwire . --order=stable --top-k=5 - + @@ -672,7 +706,7 @@ $ ./build/ripwire . --order=stable --top-k=5 -... [11 more line(s); run it to see the whole thing] +... [17 more line(s); run it to see the whole thing] ``` **Shaped by:** `--no-stable` @@ -689,20 +723,21 @@ _--no-stable outside --mcp: what the flag does (or says) when there is no stable ``` $ ./build/ripwire . --no-stable --top-k=3 - + - - + + - + - + - + +... [17 more line(s); run it to see the whole thing] ``` --- @@ -724,8 +759,8 @@ $ ./build/ripwire . --around=rankGraphTeleport - - + + @@ -775,8 +810,8 @@ _What SYM calls (1-hop out-edges)._ ``` $ ./build/ripwire . --callees=rankGraphTeleport - - + + @@ -806,16 +841,16 @@ _The resolvable use-sites (call/read/write/import/extends) with file:line; count ``` $ ./build/ripwire . --uses=rankGraphTeleport - + - - + + - - - - - + + + + + ``` @@ -840,17 +875,17 @@ _Composable node-set query: functions within 2 caller-hops of rankGraphTeleport. ``` $ ./build/ripwire . --graph-query='and(callers(name("rankGraphTeleport"),2),kind(all,fn))' - + - + + - - + - - + + ... [17 more line(s); run it to see the whole thing] @@ -877,19 +912,19 @@ _Names referenced but never defined in-corpus (stdlib/third-party surface). The ``` $ ./build/ripwire . --external-surface - - - - - - - + + + + + + + - - - - - + + + + + ... [17 more line(s); run it to see the whole thing] ``` @@ -912,12 +947,12 @@ _Shortest directed call-path SRC -> DST. CHANGED: now reports from_p/to_p/from_d ``` $ ./build/ripwire . --path=main,rankGraphTeleport - - - - - - + + + + + + ``` @@ -934,10 +969,10 @@ _Minimal connecting subgraph over 3 symbols (finds shared-caller joins)._ ``` $ ./build/ripwire . --connect=rankGraphTeleport,runEval,getIndex - + - + @@ -962,16 +997,16 @@ $ ./build/ripwire . --impact=rankGraphTeleport - - - - - + + + + + - + ... [17 more line(s); run it to see the whole thing] ``` @@ -1044,7 +1079,7 @@ _Test files that transitively reach the changed file._ ``` $ ./build/ripwire . --affected=src/graph.h - @@ -1106,7 +1141,7 @@ _Mid-task situational report for the current git diff — recorded against a CLE $ ./build/ripwire . --situ ripwire situational-awareness — 0 changed file(s), 0 symbols in them root: . -at: f1e5c2e76 +at: 6009ca693 (0 changed files — working tree is clean, nothing to analyze) ``` @@ -1124,19 +1159,19 @@ _The continuation packet for the NEXT session: disk truth (branch/sha ``` $ ./build/ripwire . --handoff - - + + - + - - - + + + + ... [1 more line(s); run it to see the whole thing] ``` @@ -1160,7 +1195,7 @@ _Pre-PR gate on a CLEAN tree: no obligations, exit 0._ ``` $ ./build/ripwire . --test-gate - ``` @@ -1185,15 +1220,15 @@ _Literal trigram-indexed search. Each hit carries its MATCHED line as the $ ./build/ripwire . --grep=DISCLOSE - + - + - + @@ -1487,10 +1522,10 @@ _Importance-weighted detail: FULL bodies for top-2, signatures for the rest._ ``` $ ./build/ripwire . --for="pagerank power iteration" --detail=2 - + - - + + NodeId order. Renumbering an otherwise-identical graph (the exact "same probe file, different so…inline constexpr float kScoreTieAbsEps = 1e-6f @@ -1522,12 +1557,12 @@ $ ./build/ripwire . --pack-signatures --top-k=10 - + - + - + @@ -1553,7 +1588,7 @@ $ ./build/ripwire . --outline=rankGraphTeleport --top-k=0 signature rows. est_tokens=: price as emitted (an upper bound under compact). root=: p= relative to it. --> - + & p, float alpha = 0.85f ) { PROFILE_SCOPE_DESCRIBE( "rankGraph: PageRank (power iteration)" ); @@ -1572,7 +1607,7 @@ $ ./build/ripwire . --outline=rankGraphTeleport --top-k=0 **Answers:** print these symbols' full bodies, plus the signatures of what they call full bodies of A,B,... -Selector grammar per item (the tail after the LAST ':' decides; a tail STARTING WITH A DIGIT is a range, anything else is a name): NAME every def of that name | FILE:NAME that file's def NAME:START-END body slice | FILE:NAME:START-END selector + slice FILE:LINE:NAME paste a row's p="path:line" straight from --callers/--lint/--grep (NOT --hotspots: its p= is a BARE path — build FILE:LINE:NAME from its own p=/top_l=/top= instead, since top= is just the worst function's name) path::scope::name the id composed from a row's p= sc= n= (the map, --for, --pack-task) Scope::name the sym= spelling edit-check and grep's in= rows print — matches the name under any scope whose ::-boundary SUFFIX is Scope (Box::lid, deep::Box::lid); a wrong scope refuses, it never falls back to the bare-name union Scope.name|Scope#name the same match, dotted (Python/JS/Java) or Ruby-style; tried only when no other spelling matched (START-END is 1-based within the def's OWN body — lines="lo-hi/total" marks the slice partial; out-of-range clamps. FILE matches any path substring, like --callers/--lego.) EXACT-NAME DEFAULT (one token, one unambiguous match, no explicit --top-k): the ranked map defaults to top-k=0 — you already named the exact symbol, so the ~200-row orientation map is pure overhead in front of the one body it exists to summarize. Disclosed on the root as topk_default="0" (self-describing: the change is visible without reading source). A MULTI-match name (an ambiguous bare name) or a multi-token --expand keeps the map — there IS something to disambiguate. An EXPLICIT --top-k=N (0 included) always overrides this default. Each body also carries sibs="a,b,..." sibs_total="N" [sibs_capped="1"] (the file's OTHER symbols, names only, capped at 40) and inc="x.h,..." inc_total="N" [inc_capped="1"] (the file's own #include/import targets, capped at 24) — both absent when the count is 0 (a documented zero, not a degrade), so a body no longer needs a second --outline call just to learn what else lives in its file. CHEAPEST-COMPLETE-ANSWER SERVING (no explicit --top-k, no range slice): the verb ALSO measures the (possibly map-less) bundle against the requested symbols' whole FILE(s) and emits the SMALLER, disclosed on the root as mode="bundle|whole-file" reason="the two byte counts" — on a small file the old bundle was 5.65x the file itself; on a big file the bundle saves ~26x. The whole-file form is with the file CDATA-wrapped (redacted as usual) and every requested symbol's line anchor kept. An EXPLICIT --top-k=N (including 0) opts out of BOTH the exact-name default and mode= auto-selection and keeps the classic undecorated shape; a SYM:START-END slice opts out of mode= auto-selection only (serving the whole file would invert an explicit narrowing) but still gets the exact-name top-k=0 default when it applies. When the bundle wins AND no explicit --top-k was given, the requested bodies are served BEFORE the ranked map (not buried after it), and the map's escape hatch also rides the root as note="...--top-k=0 for the payload alone..." (not stderr-only): a caller reading only stdout sees both the answer and how to drop the map on the next call. Absent whenever the bundle carries no map at all (top-k=0, exact-name default) or loses to whole-file, and whenever --top-k was explicit — the classic shape above carries no note= either. +Selector grammar per item (the tail after the LAST ':' decides; a tail STARTING WITH A DIGIT is a range, anything else is a name): NAME every def of that name | FILE:NAME that file's def NAME:START-END body slice | FILE:NAME:START-END selector + slice FILE:LINE:NAME paste a row's p="path:line" straight from --callers/--lint/--grep (NOT --hotspots: its p= is a BARE path — build FILE:LINE:NAME from its own p=/top_l=/top= instead, since top= is just the worst function's name) path::scope::name the id composed from a row's p= sc= n= (the map, --for, --pack-task) Scope::name the sym= spelling edit-check and grep's in= rows print — matches the name under any scope whose ::-boundary SUFFIX is Scope (Box::lid, deep::Box::lid); a wrong scope refuses, it never falls back to the bare-name union Scope.name|Scope#name the same match, dotted (Python/JS/Java) or Ruby-style; tried only when no other spelling matched (START-END is 1-based within the def's OWN body — lines="lo-hi/total" marks the slice partial; out-of-range clamps. FILE matches any path substring, like --callers/--lego.) EXACT-NAME DEFAULT (one token, one unambiguous match, no explicit --top-k): the ranked map defaults to top-k=0 — you already named the exact symbol, so the ~200-row orientation map is pure overhead in front of the one body it exists to summarize. Disclosed on the root as topk_default="0" (self-describing: the change is visible without reading source). The same default applies when --expand names SEVERAL definitions (an ambiguous bare name, or several tokens) and EVERY one is served: the bodies carry p= l= t= for each, the root adds map_next= (the shell call that prints the map you did not get — orient once per session, then look up lean). If a byte budget cut some, the root adds unserved_total=N (the true count) and unserved_next="--expand=FILE:LINE:NAME,…" (the call serving them; past 16 it serves the first 16 and unserved_listed="16" says so — a larger --pack-budget-bytes serves the rest) — never the generic map. map_next= and unserved_next= are quoted for the shell; map_next= names every root plus the crawl-scope flags (--exclude, --max-file-size, --no-ignore, --ignore-tests). --outline follows the same rule for its one token (several --outline tokens keep the ordinary map). An EXPLICIT --top-k=N (0 included) always overrides this default. Each body also carries sibs="a,b,..." sibs_total="N" [sibs_capped="1"] (the file's OTHER symbols, names only, capped at 40) and inc="x.h,..." inc_total="N" [inc_capped="1"] (the file's own #include/import targets, capped at 24) — both absent when the count is 0 (a documented zero, not a degrade), so a body no longer needs a second --outline call just to learn what else lives in its file. CHEAPEST-COMPLETE-ANSWER SERVING (no explicit --top-k, no range slice): the verb ALSO measures the (possibly map-less) bundle against the requested symbols' whole FILE(s) and emits the SMALLER, disclosed on the root as mode="bundle|whole-file" reason="the two byte counts" — on a small file the old bundle was 5.65x the file itself; on a big file the bundle saves ~26x. The whole-file form is with the file CDATA-wrapped (redacted as usual) and every requested symbol's line anchor kept. An EXPLICIT --top-k=N (including 0) opts out of BOTH the exact-name default and mode= auto-selection and keeps the classic undecorated shape; a SYM:START-END slice opts out of mode= auto-selection only (serving the whole file would invert an explicit narrowing) but still gets the exact-name top-k=0 default when it applies. When the bundle wins AND no explicit --top-k was given, the requested bodies are served BEFORE the ranked map (not buried after it), and the map's escape hatch also rides the root as note="...--top-k=0 for the payload alone..." (not stderr-only): a caller reading only stdout sees both the answer and how to drop the map on the next call. Absent whenever the bundle carries no map at all (top-k=0, exact-name default) or loses to whole-file, and whenever --top-k was explicit — the classic shape above carries no note= either. **Try it** @@ -1584,7 +1619,7 @@ $ ./build/ripwire . --top-k=0 --expand=rankGraphTeleport @@ -1689,10 +1724,10 @@ $ ./build/ripwire . --expand=readAckRecords --top-k=0 --no-redact + - - + + - + - + - + @@ -1756,11 +1791,11 @@ _File->file dependency graph (god-files, cycles)._ ``` $ ./build/ripwire . --deps - - - - - + + + + + @@ -1768,7 +1803,7 @@ $ ./build/ripwire . --deps - + ... [17 more line(s); run it to see the whole thing] ``` @@ -1793,18 +1828,18 @@ _Complexity x recent git churn (maintenance pain)._ $ ./build/ripwire . --hotspots - - - - - - - + + + + + + + - - - - + + + + ... [17 more line(s); run it to see the whole thing] ``` @@ -1821,7 +1856,7 @@ _Token-normalized duplicate bodies._ ``` $ ./build/ripwire . --clones - + @@ -1852,12 +1887,12 @@ _Per-function size ranking, largest Halstead volume/token entropy/lines first ``` $ ./build/ripwire . --biggest-first --limit=8 - - - - - - + + + + + + @@ -1885,7 +1920,7 @@ _Per function, the non-local MUTABLE state it can reach (transitively), most wri ``` $ ./build/ripwire . --nonlocal-state --limit=8 - @@ -1920,18 +1955,18 @@ _The family join: per function, which of four orthogonal evidence families fire, ``` $ ./build/ripwire . --ensemble --limit=8 - + - + - + - + ... [17 more line(s); run it to see the whole thing] ``` @@ -1951,18 +1986,18 @@ _THE single wide-angle quality read: six families in one pass, an eligible/ranke ``` $ ./build/ripwire . --quality-panel - + - + - + - - + + ... [17 more line(s); run it to see the whole thing] ``` @@ -1988,13 +2023,13 @@ _The local-reasoning lens: to understand this symbol, how much must you know tha ``` $ ./build/ripwire . --context-ratio --limit=8 - - - - - - + + + + + + ... [23 more line(s); run it to see the whole thing] ``` @@ -2019,8 +2054,8 @@ _Score the naming-* rules against this repo's own rename history: proxy=old/(old ``` $ ./build/ripwire . --naming-calibration - - + + @@ -2030,7 +2065,7 @@ $ ./build/ripwire . --naming-calibration

-

+

... [14 more line(s); run it to see the whole thing] ``` @@ -2054,18 +2089,18 @@ _The corpus's OWN case-convention vote per (language, kind) group; off-conventio ``` $ ./build/ripwire . --naming-consistency --limit=8 - - - - + + + + - + - + - + - + ... [17 more line(s); run it to see the whole thing] ``` @@ -2091,19 +2126,19 @@ _The opt-in --lint modifier: naming predicates over LOCAL variable names too, C/ ``` $ ./build/ripwire . --lint --naming-locals - - + + - - + + - + ... [17 more line(s); run it to see the whole thing] ``` @@ -2126,9 +2161,9 @@ _Functions WITH a doc comment, most name-restating first: c_coeff (high = the co ``` $ ./build/ripwire . --comment-coherence --limit=8 - + - + @@ -2159,16 +2194,16 @@ _Files that change together in git (hidden coupling)._ ``` $ ./build/ripwire . --cochange - + - + - - - - - + + + + + ... [22 more line(s); run it to see the whole thing] ``` @@ -2191,16 +2226,16 @@ _Only pairs whose co-change RECURS in 2+ sub-windows of the mined window (sub_wi ``` $ ./build/ripwire . --cochange --cochange-recur=2 - + - + - - - + + + ... [22 more line(s); run it to see the whole thing] ``` @@ -2217,17 +2252,17 @@ _Modularity-violation GROUPS instead of pairs: "X co-changes with {A,B,C}, none ``` $ ./build/ripwire . --cochange --cochange-groups - - + + - + - + ... [17 more line(s); run it to see the whole thing] @@ -2246,19 +2281,19 @@ _Hotspots scoped to RECENT churn (the regression lens)._ ``` $ ./build/ripwire . --hotspots --since="2 weeks ago" - - - - - - - - - - - - - + + + + + + + + + + + + + ... [17 more line(s); run it to see the whole thing] ``` @@ -2284,7 +2319,7 @@ _Enforce layering rules (exit 2 on violation) — run against the repo's own tes $ ./build/ripwire . --arch=test/archfix/rules.txt - @@ -2314,7 +2349,7 @@ _Enforce layering rules (exit 2 on violation) — run against the repo's own tes $ ./build/ripwire . --arch=test/archfix/rules.txt - @@ -2340,7 +2375,7 @@ _Enforce layering rules (exit 2 on violation) — run against the repo's own tes $ ./build/ripwire . --arch=test/archfix/rules.txt - @@ -2367,19 +2402,19 @@ _Built-in AST checks (c-cast, goto, unsafe-c-fn, ...)._ ``` $ ./build/ripwire . --lint - - + + - - + + - + ... [17 more line(s); run it to see the whole thing] ``` @@ -2476,19 +2511,19 @@ _DROP two families, applied after selection; the raw select=/ignore= you passed ``` $ ./build/ripwire . --lint --lint-ignore=naming-,cache- - + - - + + - + ... [17 more line(s); run it to see the whole thing] ``` @@ -2590,19 +2625,19 @@ _Cluster the call graph into cohesive modules._ ``` $ ./build/ripwire . --communities - + - + - + ... [17 more line(s); run it to see the whole thing] ``` @@ -2621,8 +2656,8 @@ _Drill into ONE call-graph community by id — the drill= the --communities outp ``` $ ./build/ripwire . --community=0 - + ``` @@ -2646,19 +2681,19 @@ _Nested module hierarchy (multi-level Louvain) + cross-module bridges — levels ``` $ ./build/ripwire . --zoom - - + + - + - + - + - + - + ... [17 more line(s); run it to see the whole thing] ``` @@ -2683,16 +2718,16 @@ $ ./build/ripwire . --report # ripwire architecture report -2688 files · 25558 symbols · 40392 edges · 2494 modules (11469 call-graph isolated) +2787 files · 26903 symbols · 41775 edges · 2640 modules (11994 call-graph isolated) Root: `.` -Call-graph isolate provenance: 3087 declaration, 1678 header, 2119 source, 4585 document; 0 connected Louvain singletons +Call-graph isolate provenance: 3221 declaration, 1908 header, 2215 source, 4650 document; 0 connected Louvain singletons -## Modules (call-graph clusters; showing 12 of 2494) -- **src::emitTo@infra/emit.h:53:2760 [write,run,emit]** — 1685 symbols -- **src::append@elixir_resolve.h:111:4875 [add,resolve,compute]** — 1021 symbols -- **src::emplace@infra/svector.h:408:22477 [resolve,scan,find]** — 605 symbols +## Modules (call-graph clusters; showing 12 of 2640) +- **src::emitTo@infra/emit.h:53:2760 [write,run,resolve]** — 1714 symbols +- **src::ASSUME@infra/Diagnostics.h:386:25510 [resolve,collect,add]** — 934 symbols +- **src::append@elixir_resolve.h:111:4875 [resolve,scan,read]** — 644 symbols ... [17 more line(s); run it to see the whole thing] ``` @@ -2709,13 +2744,13 @@ _Cross-module call seams no test reaches. NOW carries seam_pairs/shown/capped._ ``` $ ./build/ripwire . --seams - - - + + + + - @@ -2740,17 +2775,17 @@ $ ./build/ripwire . --mermaid %% ripwire --mermaid: module (directory) dependency graph — node = dir (symbol count), edge = inter-module calls (>= 3). Render at mermaid.live. flowchart LR subgraph sg0 ["src"] - n101["src
7731"] + n101["src
7915"] n102["src/infra
1280"] end subgraph sg1 ["test"] - n103["test
5057"] + n103["test
5126"] + n392["test/legochargefix
546"] n333["test/fixtures/recallpassage
314"] + n448["test/ownerhopfix/py/pkg
158"] n280["test/expandmodefix
151"] n282["test/expandsibsfix
149"] - n535["test/rubyattrsfix
85"] - n411["test/massfix
77"] - n492["test/receiverevidencefix/js/lib
74"] + n589["test/rubyschemafix
99"] ... [17 more line(s); run it to see the whole thing] ``` @@ -2769,19 +2804,19 @@ _Bus-factor: recency-weighted author ownership per file._ ``` $ ./build/ripwire . --owners - - - - + + + + - + - + ... [17 more line(s); run it to see the whole thing] ``` @@ -2800,7 +2835,7 @@ _Internal functions with no caller found in the index — a name-based graph rea ``` $ ./build/ripwire . --dead-code=src - + ``` @@ -2865,7 +2900,7 @@ _On a CLEAN tree: nothing got worse, exit 0. The gating shape is in the sandbox ``` $ ./build/ripwire . --quality-delta - @@ -2902,7 +2937,7 @@ _On a CLEAN tree: nothing got worse, exit 0. The gating shape is in the sandbox ``` $ ./build/ripwire . --quality-delta - @@ -2939,7 +2974,7 @@ _The Delta Maintainability Model scalar for the WORKING TREE vs HEAD — recorde ``` $ ./build/ripwire . --dmm -

@@ -3009,7 +3044,7 @@ _OWNERSHIP partition for a shared tree: every regression here lives in src/infra ``` $ ./build/ripwire . --quality-delta --scope=src/graph.h - @@ -3039,13 +3074,13 @@ _Fast per-symbol post-edit contract check vs git HEAD (unchanged on a clean tree ``` $ ./build/ripwire . --edit-check=rankGraphTeleport - - - - - - + + + + + ``` @@ -3093,12 +3128,12 @@ $ ./build/ripwire . --insert-before-symbol=nonNegativeFloatDescKey --edit-payloa {"n":"benchAdaptive","p":"bench/bench_radix_ab.cpp:157","l":[162]}, {"n":"radixSortNonNegativeFloatsDesc","p":"src/infra/sortutil.h:105","l":[114]}, {"n":"radixSortByScoreDescId","p":"src/infra/sortutil.h:120","l":[180]}], -"graph_ambiguous":12495,"graph_unresolved":12552,"graph_unindexed":255,"counts_floor":true},"tests_to_run":[{"p":"test/verify_radix.cpp","hops":1,"run":"bash test/greptiercheck.sh"}, +"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true},"tests_to_run":[{"p":"test/verify_radix.cpp","hops":1,"run":"bash test/greptiercheck.sh"}, {"p":"test/adaptivecutshapefix/adaptive_cut_shape_test.cpp","hops":2,"run":"bash test/adaptivecutshapecheck.sh"}, {"p":"test/includeprecise_unit.cpp","hops":2,"run":"bash test/includeprecisecheck.sh"}, {"p":"test/verify_csr.cpp","hops":2,"run":"bash test/a9disclosurecheck.sh"}, {"p":"test/rustimport_unit.cpp","hops":4,"run":"bash test/rustimportprecisecheck.sh"}], -"order":"evidence","partners":0,"run_first":1,"tests":5,"script_gates_unmodelled":743,"graph_ambiguous":12495,"graph_unresolved":12552,"graph_unindexed":255,"counts_floor":true,"next":"--uses=src/infra/sortutil.h:nonNegativeFloatDescKey"} +"order":"evidence","partners":0,"run_first":1,"tests":5,"script_gates_unmodelled":746,"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true,"next":"--uses=src/infra/sortutil.h:nonNegativeFloatDescKey"} ``` ### `--insert-after-symbol=TARGET` @@ -3114,12 +3149,12 @@ _Insert immediately AFTER one uniquely-resolved definition; replaced_bytes=0 bec ``` $ ./build/ripwire . --insert-after-symbol=lessByScoreDescId --edit-payload=/aux/payload_note.h {"applied":"insert_after_symbol","symbol":"lessByScoreDescId","file":"src/infra/sortutil.h","span":{"start":2375,"end":2463},"lines":{"start":56,"end":58},"replaced_bytes":0,"old_file_bytes":10824,"new_file_bytes":10912,"file_eol":"lf","eol_normalized":false,"trailing_newline_folded":true,"separator … [line truncated: 652 more bytes on this line] -"graph_ambiguous":12495,"graph_unresolved":12552,"graph_unindexed":255,"counts_floor":true},"tests_to_run":[{"p":"test/verify_radix.cpp","hops":1,"run":"bash test/greptiercheck.sh"}, +"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true},"tests_to_run":[{"p":"test/verify_radix.cpp","hops":1,"run":"bash test/greptiercheck.sh"}, {"p":"test/adaptivecutshapefix/adaptive_cut_shape_test.cpp","hops":2,"run":"bash test/adaptivecutshapecheck.sh"}, {"p":"test/includeprecise_unit.cpp","hops":2,"run":"bash test/includeprecisecheck.sh"}, {"p":"test/verify_csr.cpp","hops":2,"run":"bash test/a9disclosurecheck.sh"}, {"p":"test/rustimport_unit.cpp","hops":4,"run":"bash test/rustimportprecisecheck.sh"}], -"order":"evidence","partners":0,"run_first":1,"tests":5,"script_gates_unmodelled":743,"graph_ambiguous":12495,"graph_unresolved":12552,"graph_unindexed":255,"counts_floor":true,"next":"bash test/greptiercheck.sh"} +"order":"evidence","partners":0,"run_first":1,"tests":5,"script_gates_unmodelled":746,"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true,"next":"bash test/greptiercheck.sh"} ``` **Caveats (stated by the binary):** @@ -3165,12 +3200,12 @@ $ ./build/ripwire . --insert-before-symbol=nonNegativeFloatDescKey --edit-payloa {"n":"benchAdaptive","p":"bench/bench_radix_ab.cpp:157","l":[162]}, {"n":"radixSortNonNegativeFloatsDesc","p":"src/infra/sortutil.h:105","l":[114]}, {"n":"radixSortByScoreDescId","p":"src/infra/sortutil.h:120","l":[180]}], -"graph_ambiguous":12495,"graph_unresolved":12552,"graph_unindexed":255,"counts_floor":true},"tests_to_run":[{"p":"test/verify_radix.cpp","hops":1,"run":"bash test/greptiercheck.sh"}, +"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true},"tests_to_run":[{"p":"test/verify_radix.cpp","hops":1,"run":"bash test/greptiercheck.sh"}, {"p":"test/adaptivecutshapefix/adaptive_cut_shape_test.cpp","hops":2,"run":"bash test/adaptivecutshapecheck.sh"}, {"p":"test/includeprecise_unit.cpp","hops":2,"run":"bash test/includeprecisecheck.sh"}, {"p":"test/verify_csr.cpp","hops":2,"run":"bash test/a9disclosurecheck.sh"}, {"p":"test/rustimport_unit.cpp","hops":4,"run":"bash test/rustimportprecisecheck.sh"}], -"order":"evidence","partners":0,"run_first":1,"tests":5,"script_gates_unmodelled":743,"graph_ambiguous":12495,"graph_unresolved":12552,"graph_unindexed":255,"counts_floor":true,"next":"--uses=src/infra/sortutil.h:nonNegativeFloatDescKey"} +"order":"evidence","partners":0,"run_first":1,"tests":5,"script_gates_unmodelled":746,"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true,"next":"--uses=src/infra/sortutil.h:nonNegativeFloatDescKey"} ``` **Shaped by:** `--insert-after-symbol` @@ -3229,11 +3264,11 @@ _The same plan committed: per-file locks, re-verify-before-write, atomic rename, ``` $ ./build/ripwire . --edit-plan=/aux/edit_plan.json --apply -{"schema":"ripwire.edit-plan/v1","mode":"apply","edits":1,"files":1,"callers_union":4,"graph_ambiguous":12495,"graph_unresolved":12552,"graph_unindexed":255,"counts_floor":true,"applied":1,"atomic_files":1,"atomic_scope":"per-file","rollback_on_write_error":true,"recheck_before_each_write":true,"mul … [line truncated: 529 more bytes on this line] +{"schema":"ripwire.edit-plan/v1","mode":"apply","edits":1,"files":1,"callers_union":4,"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true,"applied":1,"atomic_files":1,"atomic_scope":"per-file","rollback_on_write_error":true,"recheck_before_each_write":true,"mul … [line truncated: 538 more bytes on this line] {"n":"benchAdaptive","p":"bench/bench_radix_ab.cpp:157","l":[162]}, {"n":"radixSortNonNegativeFloatsDesc","p":"src/infra/sortutil.h:107","l":[116]}, {"n":"radixSortByScoreDescId","p":"src/infra/sortutil.h:122","l":[182]}], -"graph_ambiguous":12495,"graph_unresolved":12552,"graph_unindexed":255,"counts_floor":true}}]} +"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true}}]} ``` **Shaped by:** `--edit-check` @@ -3279,19 +3314,19 @@ _Bare --slice=SYM: the INVENTORY of sliceable locals (), so a calle ``` $ ./build/ripwire . --slice=rankGraphTeleport - - - - - - - - - - - - - + + + + + + + + + + + + + ... [3 more line(s); run it to see the whole thing] ``` @@ -3316,17 +3351,17 @@ _Forward flow: which statements the seed's value reaches, at the default depth b ``` $ ./build/ripwire . --slice=rankGraphTeleport:teleport --slice-flow=fwd - - + + - + teleport( pw.begin(), pw.end() );]]> - + - + ... [22 more line(s); run it to see the whole thing] @@ -3370,10 +3405,10 @@ $ ./build/ripwire . --slice-depth=3 _Hold a LOCATION, not a name: the enclosing-definition chain at FILE:LINE (a compiler error, a diff hunk, a stack frame), outermost -> innermost._ ``` -$ ./build/ripwire . --at=src/graph.h:7926 +$ ./build/ripwire . --at=src/graph.h:8051 - - + + ``` @@ -3397,7 +3432,7 @@ _No-LLM review-evidence bundle for the working-tree diff (clean tree = empty)._ ``` $ ./build/ripwire . --pr-context - ``` @@ -3423,11 +3458,11 @@ _Pairwise cross-arm conflict sites + suggested landing order (any committish sha ``` $ ./build/ripwire . --merge-scout=HEAD~2,HEAD~1 - - + + - + @@ -3502,7 +3537,7 @@ _Which refs of the `lane/` ref family still hold divergent authored work vs HEAD ``` $ ./build/ripwire . --stray-content=lane/ - + @@ -3536,7 +3571,7 @@ _Select the genuinely-unmerged refs of the `lane/` ref family and feed them to m ``` $ ./build/ripwire . --stray-content=lane/ --plan - + @@ -3547,7 +3582,7 @@ $ ./build/ripwire . --stray-content=lane/ --plan - + ... [17 more line(s); run it to see the whole thing] ``` @@ -3572,20 +3607,20 @@ _Cross-branch ABI-break gate over the `lane/` ref family: struct byte-contract d ``` $ ./build/ripwire . --stray-content=lane/ --abi - - - - - - + + + + + + - - - + + + -... [2 more line(s); run it to see the whole thing] +... [17 more line(s); run it to see the whole thing] ``` **Caveats (stated by the binary):** @@ -3598,7 +3633,7 @@ $ ./build/ripwire . --stray-content=lane/ --abi **Answers:** find which branch's tree defines or mentions SYM, HEAD first which REF's tree defines or mentions SYM — HEAD first, then every local branch, with on-head="0" naming the case the verb exists for: content that lives only on a branch. -Each distinct blob is read ONCE (content-addressed), so N branches cost ~one tree. kind="def" on a HEAD row is the PARSED index's answer (head_labels="index"); on a REF row it is a LEXICAL heuristic — ref blobs are raw text, never ingested, so a doc quoting a signature still reads as a definition. head_labels="lexical" ⇒ HEAD fell back to that heuristic too (no indexed def of the name, or a working tree that drifted from HEAD). refs_scanned= is the SCAN denominator (refs read besides HEAD), not a matched count. Read-only; single-root only. A checkout that differs from HEAD is read from disk: each changed path's rows say ref="worktree" and replace HEAD's, at= gains +dirty, and worktree= says whether every one was read. LIMITS: a TREE scan finds only what some ref STILL carries, so hits="0" alone cannot tell a name this repo never had from one it deleted, and content dropped by every tree is invisible. Add --with-history: a row then says v="never" or v="removed" with the commit, date and file that removed it. Remote-tracking refs are excluded (they mirror local ones); refs are capped, narrow with --stray-content=SUBSTR. LISTING: by default only the kind="def" rows are listed and the kind="ref" rows counted in one element, when that page lists MORE definitions than the --whereis-listing=all page under the row cap, or the same ones in strictly fewer bytes; otherwise every hit is listed. +Each distinct blob is read ONCE (content-addressed), so N branches cost ~one tree. kind="def" on a HEAD row is the PARSED index's answer (head_labels="index"); on another ref's row (ref_labels="parsed") it is the same extraction run over that ref's blob, once per blob and path, and only for a blob holding a line a loose lexical test calls worth parsing (it only CHOOSES blobs): a blob without one reads kind="ref". kind="text" is such a line no parse read ( counts them; --detail=1 lifts the parse guard), never a def. head_labels="lexical" ⇒ HEAD fell back to that lexical test (no indexed def of the name, or a working tree that drifted from HEAD). refs_scanned= is the SCAN denominator (refs read besides HEAD), not a matched count. Read-only; single-root only. A checkout that differs from HEAD is read from disk: each changed path's rows say ref="worktree" and replace HEAD's, at= gains +dirty, and worktree= says whether every one was read. LIMITS: a TREE scan finds only what some ref STILL carries, so hits="0" alone cannot tell a name this repo never had from one it deleted, and content dropped by every tree is invisible. Add --with-history: a row then says v="never" or v="removed" with the commit, date and file that removed it. Remote-tracking refs are excluded (they mirror local ones); refs are capped, narrow with --stray-content=SUBSTR. LISTING: by default only the kind="def" (and kind="text") rows are listed and the kind="ref" rows counted in one element, when that page lists MORE definitions than the --whereis-listing=all page under the row cap, or the same ones in strictly fewer bytes; otherwise every hit is listed. On that page a definition line several refs hold (same path, same text) is ONE row with refs="N", and folded= on the root counts the rows it stands for (on the first page, shown + more + folded + refs count = hits); listing=all prints each. **Try it** @@ -3606,29 +3641,29 @@ _Which ref's tree defines or mentions SYM — HEAD first, then every local branc ``` $ ./build/ripwire . --whereis=rankGraphTeleport - - - - - - -... [24 more line(s); run it to see the whole thing] + + + + + + + +... [14 more line(s); run it to see the whole thing] ``` **Shaped by:** `--whereis-listing`, `--with-history`, `--limit` **Caveats (stated by the binary):** -- on a REF row it is a LEXICAL heuristic — ref blobs are raw text, never ingested, so a doc quoting a signature still reads as a definition. -- head_labels="lexical" ⇒ HEAD fell back to that heuristic too (no indexed def of the name, or a working tree that drifted from HEAD). +- --detail=1 lifts the parse guard), never a def. - refs_scanned= is the SCAN denominator (refs read besides HEAD), not a matched count. +- LIMITS: a TREE scan finds only what some ref STILL carries, so hits="0" alone cannot tell a name this repo never had from one it deleted, and content dropped by every tree is invisible. ### `--whereis-listing=WHICH` **Answers:** with --whereis: which rows to list, defs, refs or all (default: more defs, else shorter). -The default lists defs when that page lists MORE definitions than all under the same row cap (references can fill a capped all page before the branch definitions arrive), whatever its bytes; when both list the same definitions, defs only when strictly shorter in bytes, else all (a tie lists all). defs lists every kind="def" row and COUNTS the kind="ref" rows in one element whose next= lists exactly them (refs). kind="def" is the parser's label: a definition it does not model (define_method, setattr, a name bound by assignment) is a counted ref. With no ref row, or no def row (the mentions are then the answer), every hit is listed and the root carries no listing=. all lists every row, the whole hit list. shown=/capped= and --limit/--offset window the LISTED rows; hits= counts every row. Refused without --whereis, and on an unknown value. +The default lists defs when that page lists MORE definitions than all under the same row cap (references can fill a capped all page before the branch definitions arrive), whatever its bytes; when both list the same definitions, defs only when strictly shorter in bytes, else all (a tie lists all). defs lists every kind="def" (and kind="text") row and COUNTS the kind="ref" rows in one element whose next= lists exactly them (refs). kind="def" is the parser's label: a definition it does not model (define_method, setattr, a name bound by assignment) is a counted ref. With no ref row, or no def row (the mentions are then the answer), every hit is listed and the root carries no listing=. all lists every row, the whole hit list. shown=/capped= and --limit/--offset window the LISTED rows; hits= counts every row. Refused without --whereis, and on an unknown value. **Try it** @@ -3636,13 +3671,13 @@ _The whole hit list asked for by name: under the same 60-row cap HEAD's referenc ``` $ ./build/ripwire . --whereis=rankGraphTeleport --whereis-listing=all - - - - + + + - - + @@ -3749,8 +3784,8 @@ _The cache-locality lens over every aggregate: fields READ TOGETHER but declared ``` $ ./build/ripwire . --field-affinity - + @@ -3761,7 +3796,7 @@ $ ./build/ripwire . --field-affinity - + ... [17 more line(s); run it to see the whole thing] ``` @@ -3784,17 +3819,17 @@ _Which of this repo's doc claims are now false. CHANGED: row attribute at= renam ``` $ ./build/ripwire . --doc-drift - - - - - - + + + + - - - - + + + + + + ... [19 more line(s); run it to see the whole thing] ``` @@ -3819,17 +3854,17 @@ _Which of this repo's doc claims are now false. CHANGED: row attribute at= renam ``` $ ./build/ripwire . --doc-drift - - - - - - + + + + - - - - + + + + + + ... [19 more line(s); run it to see the whole thing] ``` @@ -3853,18 +3888,18 @@ _Same report, with git history splitting stale mentions into deleted-by-commit v ``` $ ./build/ripwire . --doc-drift --with-history - - - - - - - + + + + + - - - - + + + + + + ... [18 more line(s); run it to see the whole thing] ``` @@ -3889,7 +3924,7 @@ _The house PLAN/DESIGN format's STRUCTURE check — never semantics; exit 2 when ``` $ ./build/ripwire . --plan-lint=test/planlintfix/wave.md - + @@ -3924,10 +3959,10 @@ $ ./build/ripwire . --from-trace=- AddressSanitizer:DEADLYSIGNAL ================================================================= ==41337==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000018 (pc 0x000102f4a1c8 bp 0x00016d2f1a40 sp 0x00016d2f19e0 T0) - #0 0x102f4a1c8 in rw::rankGraphTeleport(Graph const&, std::vector const&, float) src/graph.h:7926 - #1 0x102f3e884 in rw::rankGraph(Graph const&, float) src/graph.h:7967 - #2 0x102e11f30 in runDefaultMap(MainDispatch const&) src/main.cpp:1634 - #3 0x102e01a44 in main src/main.cpp:3999 + #0 0x102f4a1c8 in rw::rankGraphTeleport(Graph const&, std::vector const&, float) src/graph.h:8051 + #1 0x102f3e884 in rw::rankGraph(Graph const&, float) src/graph.h:8092 + #2 0x102e11f30 in runDefaultMap(MainDispatch const&) src/main.cpp:1751 + #3 0x102e01a44 in main src/main.cpp:4169 #4 0x1a2b3c0dc in start+0x9dc (dyld:arm64e+0x60dc) ==41337==ABORTING ``` @@ -3954,7 +3989,7 @@ _A command that exits 0: a minimal success record (exit, measured duration, disc $ ./build/ripwire . --run-trace="true" - + ``` @@ -4054,7 +4089,7 @@ $ ./build/ripwire . --pack-task="add a new output format flag to the CLI" (run= when derivable). window: shown= total= capped= (capped=1 cut). run_unknown=1: no runner derivable (a guess would be worse). est_tokens=: price as emitted (an upper bound under compact). root=: p= relative to it. : that callee matched by name alone (receiver unprove … [line truncated: 552 more bytes on this line] - + Print the authoritative CLI usage and flag catalog to the caller-provided output streaminline void printUsage( std::FILE* out ) noexcept claude's `mcp add … -- ripwire --mcp`: the flag goes on the end of that command line, ahead of its newlineinline std::string wrapCliAddPost( const AgentTarget& row, std::string_view toolsArg ) @@ -4081,13 +4116,13 @@ _Fan-out form: one shared core + 3 per-agent slices carved along call-graph comm ``` $ ./build/ripwire . --pack-task="add a new output format flag to the CLI" --partition=3 - wraps a ; bundle role=core|partition i= symbols= modules= bytes= tokens=: one agent's ctx, symbols= ids assigned, bytes= its size; tokens= = est_tokens= = bytes/2.36 (flat, the densest rate; the inner ctx est_tokens= is language-weighted). Inner ctx task= root= budget_tokens= dropped_positive=: the task, p= base, the slice's token target, ranked candidates its budget cut. of_top=: rows ranked (far) / bodies (callers); s rel=caller|callee shared=: 1-hop edge, bodies reached (absent at 1). window: shown= total= capped= (capped=1 cut). run_unknown=1: no runner derivable (a guess would be worse). est_tokens=: price as emitted (an upper bound under compact). : that callee matched by name alone (receiver unproven); it does NOT mean the edge is false. : rank N in this ranking, rows in r= order. : cyclomatic/cognitive complexity. : N callers in the index (absent: not measured). route=: the ranker: name-exact(X) = the task names symbol X (anchors: its evidence), subtoken+body = conceptual BM25 (:broad = 1-2 plain words, plain rg may also win; :declined(...) = a name hit refused as a common name). sc=: enclosing scope; the full id is p::sc::n (p= of the row or its ) and selectors take it. next=: the one pasteable follow-up. pure=1: const/constexpr signature (Swift: non-mutating) and no transitive side effect found; a hint. partitions=/requested=: partitions carved / asked for. modules=/split=: call-graph groups found / cuts forced to split one. core_symbols=/surface=: ids in the shared core / core plus the assignable remainder. budget_per_agent_tokens=: one agent's budget, core_budget_tokens= plus partition_budget_tokens=. total_bytes=N: bytes of all bundles together. overlap_mean=/overlap_max=: pairwise Jaccard over the ids partitions name, before trimming. shared_symbols=/union_symbols=: ids two or more partitions name / ids any partition names. core_overlap=: share of the core surface a partition reaches anyway. --> - + Print the authoritative CLI usage and flag catalog to the caller-provided output streaminline void printUsage( std::FILE* out ) noexcept claude's `mcp add … -- ripwire --mcp`: the flag goes on the end of that command line, ahead of…inline std::string wrapCliAddPost( const AgentTarget& row, std::string_view toolsArg ) @@ -4114,10 +4149,10 @@ _Task lens + a compact Mermaid flowchart of the top anchors' 1-hop edges._ ``` $ ./build/ripwire . --for="pagerank power iteration" --with-graph - + - - + + NodeId order. Renumbering an otherwise-identical graph (the exact "same probe file, different so…inline constexpr float kScoreTieAbsEps = 1e-6f @@ -4188,14 +4223,14 @@ _Environment self-check: binary staleness, grammars, cache dir, git, tracked-bin ``` $ ./build/ripwire . --doctor - - + - - + + - - + @@ -4222,14 +4257,14 @@ _--doctor plus a LIVE integration inspection for one agent: PATH binary, install ``` $ ./build/ripwire . --doctor --agent=claude - - + - - + + - - + @@ -4257,7 +4292,7 @@ $ ./build/ripwire . --skipped - @@ -4345,15 +4380,15 @@ _Rank by git change-frequency prior instead of PageRank._ $ ./build/ripwire . --rank-by=churn --top-k=5 - - + + - + - + @@ -4378,22 +4413,22 @@ a trailing slash is ignored; absolute paths and '..' are refused). The global block stays byte-identical, a second page follows it — n=/of= are its counts (of= IS the total, so the paging half carries no total=), capped="1" has_more="1" next_offset= offset= limit= page it, and next= replays THIS run's own corpus flags (--since/--exclude) so the page it names is a page of the same answer. The symbol map collapses to a disclosed stub — the map was not asked for and was not ranked at all, which is why the header carries no pr_iters=. merge_bombs_skipped= stays on the global block: it counts the window's skipped commits, not the directory's._ ``` -$ ./build/ripwire . --rank-by=churn-decay --since=HEAD~4 --exclude=test --exclude=docs --exclude=skills --in=src --limit=3 +$ ./build/ripwire . --rank-by=churn-decay --since=HEAD~7 --exclude=test --exclude=docs --exclude=skills --in=src --limit=3 - - - - - - - - + + + + + + + - - - -... [11 more line(s); run it to see the whole thing] + + + + +... [9 more line(s); run it to see the whole thing] ``` **Shaped by:** `--external-surface`, `--doctor`, `--legend`, `--max-memory` @@ -4455,7 +4490,7 @@ _The same gating report under --legend=full — same rows, same exit 2; the defa ``` $ ./build/ripwire . --quality-delta --legend=full - @@ -4483,7 +4518,7 @@ _The session legend dictionary the MCP server serves as ripwire://legend-dict/fu ``` $ ./build/ripwire . --legend-dict -ripwire legend dictionary ripwire.dict/v1 dictv=593769ab691623d8 entries=806 +ripwire legend dictionary ripwire.dict/v1 dictv=74f30d013d02c2a2 entries=829 : the answer's rows come first; its root keeps only task= changed= from= to=, and this LAST child carries every other root attribute unchanged (schema= included); legend="ref": a definition is sent once per session (this dictionary's core, or the first answer that ne … [line truncated: 83 more bytes on this line] schema=ripwire.KEY/v1: the line ripwire.KEY/v1 below reads the answer's rows window: shown= total= capped= has_more= next_offset= offset= limit= page a list (capped=1 cut; next_offset= pastes as offset=) @@ -4526,7 +4561,7 @@ $ ./build/ripwire . --hotspots --json **Answers:** paginate a high-cardinality verb paginate a high-cardinality verb. -HONORED by: --deps --callers --callees --tree --lint --hotspots --clones --cochange --owners --communities --community --doc-drift --whereis --grep/--regex --match --pattern --impact --uses --exercises --seams --zoom --external-surface --dead-code --mentions --graph-query --stray-content --test-gate --biggest-first --ensemble --quality-panel --context-ratio --nonlocal-state --comment-coherence --naming-consistency --safe-delete --pr-context --edit-check --flags --situ --for. Emit at most N rows, skipping the first M; N overrides the verb's own display cap (40 hotspot files, 30 co-change pairs, 60 whereis hits, 100 grep/match hits, 40 impact rows, 20 seam pairs, 40 readability rows, 40 ensemble symbol rows, 40 context-ratio symbol rows, 40 nonlocal-state rows, 200 graph-query rows / --top-k, 40 unflagged --edit-check caller rows, 8 --flags read sites per gate, 25 --flip context rows per listing, 8 --situ blast-radius files and 8 co-change partners). A verb NEVER pages the rows that ARE its answer: --edit-check's flagged callers, --flip's and --situ's tests-to-run rows and --flags' gate rows ride every page in full, and every verdict/count attribute is computed over the full set first. With --offset alone (no --limit) the verb's own default page size applies and the root discloses limit="0" — on OUTPUT that 0 means 'no explicit --limit', never a zero-row page (the flag itself refuses --limit=0). A BARE run whose default cap cut rows (capped="1") carries the same limit="0" and the whole paging block below, so you can page from the first answer without guessing. Deterministic seams (rows are already sorted) so --offset=N is the exact continuation of the previous --limit=N page. On --for, --limit=N/--offset=M do not window the bundle: they select its FILE-GRAIN WIDENING PAGE instead (one row per positive-score file, ranked file-first; see --for), the answer to "the head missed it, show me more files". The root element then carries shown= capped= total= has_more= next_offset= offset= limit= — loop until has_more="0". capped= compares the PAGE to the total (1 ⇔ shown < total), so a page past the end reads shown="0" capped="1" has_more="0": nothing was cut, the offset skipped everything — EXCEPT the verbs with TWO INDEPENDENT listings, which carry the noun-prefixed form instead (one shown= could only describe one): --test-gate shown_tests=/tests_capped= + shown_untested=/untested_capped=, --communities shown_modules=/modules_capped= + shown_bridges=/bridges_capped=, --ensemble and --context-ratio shown_syms=/syms_capped= + shown_files=/files_capped=; the window takes the PRIMARY listing (--test-gate's rows; its rows repeat on every page, complete). --edit-check is the same shape for a different reason: its rows split into the ANSWER (callers flagged incompatible="1", with their complete sites_l=) and the CONTEXT (unflagged callers). Only the context pages — shown_unflagged=/unflagged_capped=, with total= the unflagged count — while the flagged rows and the overload census ride every page in full and status=/defs=/callers=/incompatible= are computed over the FULL caller set before any window, so a page can never make the verdict say less than it knows. Any verb NOT in that list REFUSES both flags (exit 1) rather than accepting and ignoring them: budget/top-k verbs (--recall/--pack-task/--from-trace/ --expand/--outline/--pack-signatures/--format=candidates) are shaped by --top-k/--max-tokens/--token-budget, not a page (--for's bare bundle is shaped by --token-budget the same way, and takes --limit/--offset only as its file page, where the budget flags are refused in turn); the rest (--path/--connect/ --around/--exemplar/--report/--mermaid/--map-diff/--metrics and the default map) answer with a single fixed-shape result that has no row list to window at all. +HONORED by: --deps --callers --callees --tree --lint --hotspots --clones --cochange --owners --communities --community --doc-drift --whereis --grep/--regex --match --pattern --impact --uses --exercises --seams --zoom --external-surface --dead-code --mentions --graph-query --stray-content --test-gate --biggest-first --ensemble --quality-panel --context-ratio --nonlocal-state --comment-coherence --naming-consistency --safe-delete --pr-context --edit-check --flags --situ --for. Emit at most N rows, skipping the first M; N overrides the verb's own display cap (40 hotspot files, 30 co-change pairs, 60 whereis hits, 100 grep/match hits, 40 impact rows, 20 seam pairs, 40 readability rows, 40 ensemble symbol rows, 40 context-ratio symbol rows, 40 nonlocal-state rows, 200 graph-query rows / --top-k, 40 unflagged --edit-check caller rows, 8 --flags read sites per gate, 25 --flip context rows per listing, 8 --situ blast-radius files and 8 co-change partners). A verb NEVER pages the rows that ARE its answer: --edit-check's flagged callers, --flip's and --situ's tests-to-run rows and --flags' gate rows ride every page in full, and every verdict/count attribute is computed over the full set first. With --offset alone (no --limit) the verb's own default page size applies and the root discloses limit="0" — on OUTPUT that 0 means 'no explicit --limit', never a zero-row page (the flag itself refuses --limit=0). A BARE run whose default cap cut rows (capped="1") carries the same limit="0" and the whole paging block below, so you can page from the first answer without guessing. Deterministic seams (rows are already sorted) so --offset=N is the exact continuation of the previous --limit=N page. On --for, --limit=N/--offset=M do not window the bundle: they select its FILE-GRAIN WIDENING PAGE instead (one row per positive-score file, ranked file-first; see --for), the answer to "the head missed it, show me more files". The root element then carries shown= capped= total= has_more= next_offset= offset= limit= — loop until has_more="0". capped= compares the PAGE to the total (1 ⇔ shown < total), so a page past the end reads shown="0" capped="1" has_more="0": nothing was cut, the offset skipped everything — EXCEPT the verbs with TWO INDEPENDENT listings, which carry the noun-prefixed form instead (one shown= could only describe one): --test-gate shown_tests=/tests_capped= + shown_untested=/untested_capped=, --communities shown_modules=/modules_capped= + shown_bridges=/bridges_capped=, --ensemble and --context-ratio shown_syms=/syms_capped= + shown_files=/files_capped=; the window takes the PRIMARY listing (--test-gate's rows; its rows repeat on every page, complete). --edit-check is the same shape for a different reason: its rows split into the ANSWER (callers flagged incompatible="1", with their complete sites_l=) and the CONTEXT (unflagged callers). Only the context pages — shown_unflagged=/unflagged_capped=, with total= the unflagged count — while the flagged rows and the overload census ride every page in full and status=/defs=/callers=/incompatible= are computed over the FULL caller set before any window, so a page can never make the verdict say less than it knows. Any verb NOT in that list REFUSES both flags (exit 1) rather than accepting and ignoring them: budget/top-k verbs (--recall/--pack-task/--from-trace/ --expand/--outline/--pack-signatures/--format=candidates) are shaped by --token-budget the same way; --for takes --limit/--offset as its FILE page, where the budget flags are refused in turn (bare --pack-task --limit refuses: it has no page without a budget), and --for/--pack-task beside --token-budget take the window as the BUDGETED-BUNDLE CANDIDATE PAGE — one window over the ranked candidates, signatures only (bodies come via next= or --expand), their resumable continuation; the rest (--path/--connect/ --around/--exemplar/--report/--mermaid/--map-diff/--metrics and the default map) answer with a single fixed-shape result that has no row list to window at all. **Try it** @@ -4535,18 +4570,18 @@ _The family join: per function, which of four orthogonal evidence families fire, ``` $ ./build/ripwire . --ensemble --limit=8 - + - + - + - + ... [17 more line(s); run it to see the whole thing] ``` @@ -4569,24 +4604,24 @@ _Drop matching paths (repeatable) before ranking._ ``` $ ./build/ripwire . --exclude=present --exclude=bench --top-k=5 - + - - + + - + - + -... [11 more line(s); run it to see the whole thing] +... [17 more line(s); run it to see the whole thing] ``` -**Shaped by:** `--skipped`, `--in`, `--index-out` +**Shaped by:** `--expand`, `--skipped`, `--in`, `--index-out` ### `--map-diff` @@ -4602,14 +4637,14 @@ _Full map re-ranked with teleport toward git-changed files — clean tree, so ch $ ./build/ripwire . --map-diff --top-k=5 - - + + - + - + @@ -4634,20 +4669,21 @@ _Explicit incremental cache at a path OUTSIDE the repo (first call writes it)._ ``` $ ./build/ripwire . --cache=/aux/warm2.ripwirecache --top-k=3 - + - - + + - + - + - + +... [17 more line(s); run it to see the whole thing] ``` **Shaped by:** `--doctor`, `--index-out` @@ -4683,20 +4719,21 @@ _Force a cold parse (bypass the warm TMPDIR cache) — shows the cold-vs-warm co ``` $ ./build/ripwire . --no-cache --top-k=3 - + - - + + - + - + - + +... [17 more line(s); run it to see the whole thing] ``` **Shaped by:** `--doctor` @@ -4713,22 +4750,25 @@ _Crawl paths the repo's own .gitignore covers (default honours it and discloses ``` $ ./build/ripwire . --no-ignore --top-k=3 - + - - + + - + - + - + +... [17 more line(s); run it to see the whole thing] ``` +**Shaped by:** `--expand` + **Caveats (stated by the binary):** - by default they are skipped crawl paths the repository's own .gitignore covers. @@ -4746,24 +4786,24 @@ _Skip files above a size bound before parsing (note the corpus shrink in the hea ``` $ ./build/ripwire . --max-file-size=8K --top-k=3 - + - - + + - + - + - -... [3 more line(s); run it to see the whole thing] + +... [17 more line(s); run it to see the whole thing] ``` -**Shaped by:** `--skipped` +**Shaped by:** `--expand`, `--skipped` **Caveats (stated by the binary):** @@ -4826,20 +4866,21 @@ _Eval-only: a per-call-site census of WHICH mechanism resolved each call, and th ``` $ ./build/ripwire . --pin-census=/aux/pin_census.tsv --top-k=3 - + - - + + - + - + - + +... [17 more line(s); run it to see the whole thing] ``` ### `--mcp` @@ -4881,10 +4922,10 @@ _--mcp-legend=inline: every answer of the stdio session carries its own legend ( ``` $ ./build/ripwire '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize"}' '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"whereis","arguments":{"path":".","symbol":"rankGraphTeleport"}}}' '{"jsonrpc":"2.0","id":3,"method":"tools/call","params":{"name":"whereis","arguments":{"path":".","symbol":"computeOnePairOverlap"}}}' | ./build/ripwire --mcp --mcp-legend=inline {"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-11-25","serverInfo":{"name":"ripwire","version":"1.0"},"capabilities":{"tools":{},"resources":{}},"instructions":"Map before reading files. Start a new task with explore; use from_trace for an error; use impact plus uses before changing a sym … [line truncated: 735 more bytes on this line] -{"jsonrpc":"2.0","id":2,"result":{"content":[{"type":"text","text":" | +| **The gate suite** | `test/regression.sh`, `test/pargates.py` | 681 gate scripts plus the determinism, cache-transparency and golden contracts. | | **`--quality-delta`** | `src/quality.h` | Ten measured code-quality failure modes, reported only where a change made them worse. | ### The labeling protocol (why the held-out eval is allowed to disagree with the ranker) @@ -5888,7 +5888,7 @@ copy here would be exactly the dialect divergence that gate exists to catch. Com tags, wrap, stable-order defaults), seven individually invoked standalone gates (`g1freshcheck`, `skillscan`, `htmlexport`, `compresscheck`, `handoffcheck`, `releaseinstallcheck`, `taskroutecheck`), and a single loop -naming **678 gate scripts**, all of which exist on disk. +naming **681 gate scripts**, all of which exist on disk. `python3 test/pargates.py . ./build/ripwire -j 6` runs the same scripts in parallel so a full verification fits in one sitting. It does not modify `regression.sh`. @@ -7005,7 +7005,7 @@ Listed because the reason is more useful than the silence. shipped**. See `bench/locbench/anchorhop_calib.json`. The mention anchor's reproducible numbers are the ablations in §4. - **A single round gate-count.** Two in-tree numbers disagree (`test/pargates.py`'s docstring says - ~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 678. The + ~210; `test/argvdiffcheck.sh` says 200+), while the loop in `test/regression.sh` names 681. The loop is the authority; the stale docstrings are a known drift. Since 2026-09-10 the number is not written by hand anywhere: `docs/gatecount_build.py` derives it from the loop and rewrites every published site, `test/gatecountcheck.sh` fails if any of them drifts, and `test/manifestcheck.sh` diff --git a/docs/LIMITS.md b/docs/LIMITS.md index 40041f310..b2c7bc988 100644 --- a/docs/LIMITS.md +++ b/docs/LIMITS.md @@ -17,10 +17,10 @@ it once, marked `×N`. | total caps | files | caps whose file discloses | caps whose file discloses NOTHING | | --- | --- | --- | --- | -| 240 | 96 | 140 | **100** | +| 245 | 97 | 145 | **100** | Plus 7 ranking and apportionment parameters, in their own table below: they are not caps, they -are not counted as caps, and 240 + 7 is the 247 constants this generator parses out of `src/`. +are not counted as caps, and 245 + 7 is the 252 constants this generator parses out of `src/`. ## INDEXING, OUTPUT or BOUNDARY — which half of the answer a cap bounds @@ -38,8 +38,8 @@ None of them truncates anything, so none can be judged by `shown=`/`total=` and a disclosure — labelling them OUTPUT would ask for a `capped="1"` that could never honestly fire. The distinction was named in review on #108 and the rows below now carry it. -The `class` column below carries that answer where it is known. **132 of 240 caps are classified -(47 INDEXING, 41 OUTPUT, 44 BOUNDARY); the remaining 108 render `—`, which means NOT YET +The `class` column below carries that answer where it is known. **134 of 245 caps are classified +(47 INDEXING, 43 OUTPUT, 44 BOUNDARY); the remaining 111 render `—`, which means NOT YET CLASSIFIED — never "neither".** Classifications live in `docs/limits_classes.tsv`, a sidecar with a known expiry: the tag belongs on the declaration itself, and this file exists only because the round that @@ -89,7 +89,7 @@ refuse to write, so the column cannot be satisfied by pointing at nothing. ## Caps, by file -One table for each of the 96 files that declare a cap — the 240 caps counted above, and no parameter. +One table for each of the 97 files that declare a cap — the 245 caps counted above, and no parameter. ### `src/abicheck.h` @@ -190,6 +190,7 @@ Discloses: **none** | `kMaxRenameProbeFiles` | `32` | INDEXING | HEAD copies one rename probe reads; past it the probe stops (a hint, never a claim) | | `kMaxWorktreePaths` | `8192` | INDEXING | — | | `kWhereisHits` | `60` | OUTPUT | — | +| `kWhereisParseMaxBlobs` | `256` | — | — | ### `src/darkflags.h` @@ -323,7 +324,7 @@ Discloses: `capped`, `hosts_capped` ### `src/forpage.h` -Discloses: **none** +Discloses: `capped` | constant | value | class | note | | --- | --- | --- | --- | @@ -583,8 +584,18 @@ Discloses: **none** | constant | value | class | note | | --- | --- | --- | --- | +| `kMaxUnservedSelectors` | `16` | — | — | | `kRecentRows` | `40` | — | F3: ~45 B a row; the file-level answer, not the file list | +### `src/mapinventory.h` + +Discloses: `entries_capped`, `names_capped` + +| constant | value | class | note | +| --- | --- | --- | --- | +| `kInventoryEntryCap` | `256` | OUTPUT | — | +| `kInventoryNameCeiling` | `2000` | OUTPUT | — | + ### `src/mcp.h` Discloses: `hits_capped` @@ -867,7 +878,7 @@ Discloses: **none** ### `src/serialize.h` -Discloses: `capped`, `calls_capped`, `inc_capped`, `sibs_capped` +Discloses: `capped`, `calls_capped`, `entries_capped`, `inc_capped`, `names_capped`, `sibs_capped` | constant | value | class | note | | --- | --- | --- | --- | @@ -984,6 +995,7 @@ Discloses: `capped`, `coboost_commits_capped`, `terms_capped` | constant | value | class | note | | --- | --- | --- | --- | +| `kForOwnerHopRows` | `3` | — | — | | `kJsonEnvelopeDigitsMax` | `10` | — | — | ### `src/verbs_lint.h` diff --git a/docs/TUNING.md b/docs/TUNING.md index 0dc297ff9..03caee092 100644 --- a/docs/TUNING.md +++ b/docs/TUNING.md @@ -14,26 +14,26 @@ to production at defaults; that control is what makes these numbers mean anythin | cap declarations | distinct names | tunable | must stay `constexpr` | move >= 1 invocation | move nothing measurable | | --- | --- | --- | --- | --- | --- | -| 150 | 149 | 112 | 12 | **37** | 75 | +| 154 | 153 | 112 | 12 | **37** | 75 | -The first two columns are not the same number, and the gap is not a rounding: `src/` holds **150 cap -declarations** under **149 distinct names** (`kRowCap` declared in more than one file). The sweep +The first two columns are not the same number, and the gap is not a rounding: `src/` holds **154 cap +declarations** under **153 distinct names** (`kRowCap` declared in more than one file). The sweep patches by NAME, so its own population is NAMES and not declarations — and the two columns beside those are frozen at the commit named under Provenance while this census is re-read from `src/` on -every run, which makes them a third population again. Enumerated over the 149 names `src/` declares -today: **111 tunable**, **12 must stay `constexpr`**, and **26 declared since the sweep was +every run, which makes them a third population again. Enumerated over the 153 names `src/` declares +today: **111 tunable**, **12 must stay `constexpr`**, and **30 declared since the sweep was prepared, which no measurement has touched** (`kChurnMergeBombMaxFiles`, -`kExtendedLengthThresholdUnits`, `kFieldIdCapacity`, `kForPageRowsDefault`, -`kForPageUnionSymbolCap`, `kMaxAstQueryNesting`, +20 more). 111 + 12 + 26 = 149, and `emit` refuses -to render a partition that does not add up. The `tunable` column above reads 112 rather than 111 -because one name the sweep classified is no longer declared in `src/` at all: `kSituTestRowsShown`. -Quoting "113 of 150" would be wrong in both halves at once, which is the shape of error a generated -table exists to prevent. +`kExtendedLengthThresholdUnits`, `kFieldIdCapacity`, `kForOwnerHopRows`, `kForPageRowsDefault`, +`kForPageUnionSymbolCap`, +24 more). 111 + 12 + 30 = 153, and `emit` refuses to render a partition +that does not add up. The `tunable` column above reads 112 rather than 111 because one name the +sweep classified is no longer declared in `src/` at all: `kSituTestRowsShown`. Quoting "113 of 154" +would be wrong in both halves at once, which is the shape of error a generated table exists to +prevent. ## Read this ratio before the tables **37 of 112 tunable caps move any invocation at all. 75 move nothing measurable.** That is the -finding, and it says what NOT to do: this is not a 150-cap audit. Most of these constants are +finding, and it says what NOT to do: this is not a 154-cap audit. Most of these constants are inert on real invocations and should be left alone. The work worth doing is the small set below, plus the caps that fire SILENTLY — a cap that bites without disclosing is a defect independent of whether its value is right, and that fix is both cheaper and larger than any retuning. @@ -61,7 +61,7 @@ moving. Re-run `prepare|screen|sweep` to refresh them. ### `kForLensDefaultTopN` = `40` -`src/serialize.h` — discloses: `calls_capped`, `inc_capped`, `sibs_capped` — probe value `320` — **12 verb(s) respond** +`src/serialize.h` — discloses: `calls_capped`, `entries_capped`, `inc_capped`, `names_capped`, `sibs_capped` — probe value `320` — **12 verb(s) respond** | invocation | default | at probe | delta | | --- | --- | --- | --- | @@ -117,7 +117,7 @@ moving. Re-run `prepare|screen|sweep` to refresh them. ### `kForFileTailShownCap` = `24` -`src/serialize.h` — discloses: `calls_capped`, `inc_capped`, `sibs_capped` — probe value `192` — **10 verb(s) respond** +`src/serialize.h` — discloses: `calls_capped`, `entries_capped`, `inc_capped`, `names_capped`, `sibs_capped` — probe value `192` — **10 verb(s) respond** | invocation | default | at probe | delta | | --- | --- | --- | --- | @@ -134,7 +134,7 @@ moving. Re-run `prepare|screen|sweep` to refresh them. ### `kForPayloadBudgetBytes` = `7500` -`src/serialize.h` — discloses: `calls_capped`, `inc_capped`, `sibs_capped` — probe value `60000` — **10 verb(s) respond** +`src/serialize.h` — discloses: `calls_capped`, `entries_capped`, `inc_capped`, `names_capped`, `sibs_capped` — probe value `60000` — **10 verb(s) respond** | invocation | default | at probe | delta | | --- | --- | --- | --- | @@ -151,7 +151,7 @@ moving. Re-run `prepare|screen|sweep` to refresh them. ### `kForCapTailSigBytes` = `96` -`src/serialize.h` — discloses: `calls_capped`, `inc_capped`, `sibs_capped` — probe value `768` — **8 verb(s) respond** +`src/serialize.h` — discloses: `calls_capped`, `entries_capped`, `inc_capped`, `names_capped`, `sibs_capped` — probe value `768` — **8 verb(s) respond** | invocation | default | at probe | delta | | --- | --- | --- | --- | @@ -166,7 +166,7 @@ moving. Re-run `prepare|screen|sweep` to refresh them. ### `kMaxExpandSibs` = `100` -`src/serialize.h` — discloses: `calls_capped`, `inc_capped`, `sibs_capped` — probe value `800` — **5 verb(s) respond** +`src/serialize.h` — discloses: `calls_capped`, `entries_capped`, `inc_capped`, `names_capped`, `sibs_capped` — probe value `800` — **5 verb(s) respond** | invocation | default | at probe | delta | | --- | --- | --- | --- | @@ -219,7 +219,7 @@ moving. Re-run `prepare|screen|sweep` to refresh them. ### `kForAutoBodyBudgetBytes` = `6000` -`src/serialize.h` — discloses: `calls_capped`, `inc_capped`, `sibs_capped` — probe value `48000` — **2 verb(s) respond** +`src/serialize.h` — discloses: `calls_capped`, `entries_capped`, `inc_capped`, `names_capped`, `sibs_capped` — probe value `48000` — **2 verb(s) respond** | invocation | default | at probe | delta | | --- | --- | --- | --- | @@ -373,7 +373,7 @@ moving. Re-run `prepare|screen|sweep` to refresh them. ### `kMaxExpandIncludes` = `24` -`src/serialize.h` — discloses: `calls_capped`, `inc_capped`, `sibs_capped` — probe value `192` — **1 verb(s) respond** +`src/serialize.h` — discloses: `calls_capped`, `entries_capped`, `inc_capped`, `names_capped`, `sibs_capped` — probe value `192` — **1 verb(s) respond** | invocation | default | at probe | delta | | --- | --- | --- | --- | @@ -429,7 +429,7 @@ moving. Re-run `prepare|screen|sweep` to refresh them. ### `kWithGraphNodeCap` = `8` -`src/serialize.h` — discloses: `calls_capped`, `inc_capped`, `sibs_capped` — probe value `64` — **1 verb(s) respond** +`src/serialize.h` — discloses: `calls_capped`, `entries_capped`, `inc_capped`, `names_capped`, `sibs_capped` — probe value `64` — **1 verb(s) respond** | invocation | default | at probe | delta | | --- | --- | --- | --- | diff --git a/docs/captures/COMMANDS_showcase_2026-09-14.md b/docs/captures/COMMANDS_showcase_2026-09-14.md index 23dc272e6..ac7f16979 100644 --- a/docs/captures/COMMANDS_showcase_2026-09-14.md +++ b/docs/captures/COMMANDS_showcase_2026-09-14.md @@ -3921,29 +3921,29 @@ ripwire 0.6.1 (dev, AppleClang 21.0.0.21000101, emit=std::print, built_from=f8d4 # navigate — seeds, claims, slices, shapes -## `./build/ripwire . --at=src/graph.h:7397` +## `./build/ripwire . --at=src/graph.h:7512` *Hold a LOCATION, not a name: the enclosing-definition chain at FILE:LINE (a compiler error, a diff hunk, a stack frame), outermost -> innermost.* ````` - - + + ````` -## `./build/ripwire . --callers=@src/graph.h:7397` +## `./build/ripwire . --callers=@src/graph.h:7512` *The same seed in a SELECTOR position: @FILE:LINE resolves to the innermost enclosing definition, then --callers runs on it.* ````` - + - - + + diff --git a/docs/captures/COMMANDS_showcase_2026-10-10.md b/docs/captures/COMMANDS_showcase_2026-10-10.md new file mode 100644 index 000000000..306dccace --- /dev/null +++ b/docs/captures/COMMANDS_showcase_2026-10-10.md @@ -0,0 +1,7643 @@ +# ripwire — every verb, run for real + +- **Date:** 2026-10-10 (regenerated capture; supersedes any older `docs/captures/COMMANDS_showcase_*.md`) +- **Lives in `docs/captures/`** — a directory the crawl/retrieval lenses SKIP (`kCrawlSkipDirs`, src/ingest.h): a generated doc that quotes every verb's output out-scores the source for any query about the tool and was measured at 77% of `--recall` on this repo when it sat at the root. `test/argvdiffcheck.sh` harvests its `## `-heading command lines as differential vectors — keep that format. +- **Version:** `ripwire 0.6.5 (dev, AppleClang 21.0.0.21000101, emit=std::print, built_from=65d62fe5c)` +- **Repo:** the ripwire repo @ `6009ca69` — **CLEAN — `git status --porcelain` is empty**. The diff-aware verbs (`--situ`/`--test-gate`/`--quality-delta`/`--pr-context`/`--map-diff`/`--edit-check`) answer a question about the WORKING TREE, so that condition is part of their answer and every one of their captions below states which tree it recorded against. A clean tree is the honest default for a showcase, so they appear TWICE: once here on the clean tree (their empty/exit-0 shape) and once in the final section against a throwaway `git clone --local` sandbox carrying one deliberate regression, so their real gating shapes are visible without writing a byte into the read-only repo. +- **Corpus:** the ripwire repo itself (dogfood), via `./build/ripwire` +- **Sandbox diff** (the last section only): `.ripwire_notes | 2 ++ + .ripwire_quality_acks | 1 + + src/infra/sortutil.h | 31 ++++++++++++++++++++++++++++--- + 3 files changed, 31 insertions(+), 3 deletions(-)` — one preexisting function made deeply nested, one function's arity changed 1 -> 2, one copy-paste duplicate helper, one new 8-parameter public function. + +**How to read the blocks:** ripwire's real XML output is minified — often ONE long line. For scanability, long minified lines are displayed re-wrapped with a line break at every tag seam (`><`). Header COMMENT lines (the legends) always appear in full — they are exempt from the per-line cut; any OTHER display line over 300 bytes is cut with a `… [line truncated: N more bytes]` marker, which can hit a long root element or row. `--plan-lanes` emits JSON and is re-wrapped at object seams the same way. Long outputs are cut to their first ~30 display lines with a `… [N more display lines; full output is M bytes]` marker giving the true size. Exit codes are recorded when non-zero; wall time when >1s. + +**Not run (and why):** `ripwire ` (network clone), `--listen` / `--mcp-token` / `--allow-remote-edits` (the HTTP-server posture; `--mcp` itself IS captured in its own section as a one-shot stdio JSON-RPC exchange, and `wrap claude` shows the wiring), `--arch --baseline[-update]` (state writer against the read-only repo — `--note-add` / `--quality-baseline` / `--quality-ack` / the three edit verbs / `--edit-plan` ARE shown, inside the throwaway sandbox clone; `--index-out` / `--pin-census` write to scratch), `--eval-mined` (needs a `minedpair.jsonl` artifact from `bench/mine_traces.py`; none present in the tree), `--refetch` (git-url only), `--lsp` (the editor transport — its one-shot dialogues are gated by `test/lspcheck.sh`, not by showcase blocks), `--max-memory` (the memory guard is silent on this repo by construction; its stops are gated by `test/memguardcheck.sh`), `--force` (wrap-only modifier), `--scan-skills` bare form (would sweep `~/.claude/skills`; the explicit-DIR form is shown instead), `--help` (209 lines — read it from the binary). + + +--- + +# understand a codebase cold + +## `./build/ripwire .` + +*The default ranked symbol map — start here when landing cold in a repo.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [815 more display lines; full output is 26837 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --top-k=5` + +*Same map, capped to the 5 highest-ranked symbols.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [67 more display lines; full output is 8262 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --top-k=5 --no-inventory` + +*The same capped map without its inventory tier: --top-k=5 leaves files out, so the default map appends (program entries, and the files the ranked rows left out) after its rows; --no-inventory drops that tier, and the ranked rows are the same.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --top-k=0 --expand=rankGraphTeleport` + +*NEW since the last capture: --top-k=0 means PAYLOAD-ONLY — no ranked map rides along with the body you asked for.* + +````` + + + + + + + + + + +````` + +## `./build/ripwire . --token-budget=100` + +*GATE form: exit 3 if the map's own est_tokens exceeds the budget (over-budget failure shape).* + +**exit code: 3** + +````` + + +````` + +stderr: + +````` +ripwire: --token-budget exceeded: withheld_est_tokens=10806 > budget=100 +````` + +## `./build/ripwire . --for="incremental cache invalidation when a file content hash changes"` + +*The task lens: ranked signatures + quality metrics framed for the task.* + +**wall time: 1.31s** + +````` + + + + +incremental cache (--cache): per-file content hash + raw facts so a re-run re-parses ONLY c…constexpr std::uint32_t kCacheMagic = 0x4b505443 + +and every file whose content hash differs is re-parsed — so a stale or foreign blob self-heals…constexpr std::uint32_t kHeadSnapCacheScheme = 1 + +persistent in-memory index (parse once, reuse across MCP calls) ---- The MCP server is long-live…struct McpIndex + +the cached index for `root`, rebuilt only when stale (otherwise returned as-is, no parse, no gra…inline const McpIndex& getIndex( const std::string& root ) +inline McpRebuildBaseline mcpRebuildBaseline( const McpIndex& ix, bool isIncrementalPass ) +inline std::uint64_t contentHash64( std::string_view s ) noexcept +struct CacheEntry +inline void prewarmTagsQueries( const std::vector<std::string>& files, const HashMap<std::string, FileFacts>& cache, long long cacheWriteNs, IngestFile … [line truncated: 20 more bytes on this line] +inline std::uint64_t blobChecksum( std::string_view s ) noexcept +inline ContentIdIndex contentIdsBySym( const IngestResult& ing, const Graph& g, std::string_view root ) +inline std::vector<CacheWriteRow> buildCacheWritePlan( const std::vector<std::uint32_t>& orderIn, const std::vector<std::uint64_t>& pathHashes … [line truncated: 25 more bytes on this line] +inline bool ingestCommitTree( const std::string& root, const std::string& sha, const std::vector<std::string>& excludes, std::size_t maxFileBytes, IngestResult& … [line truncated: 7 more bytes on this line] +inline std::string reAbsolutize( std::string_view rel, std::string_view root ) +inline CachePathKeys buildCachePathKeys( const std::vector<std::string>& files, std::string_view rootDir ) +inline std::vector<std::vector<NodeId>> legoImplementorsOnSurface( const IngestResult& ing, const std::vector<std::vector<NodeId>>& impl … [line truncated: 63 more bytes on this line] +inline std::uint64_t indexContentHash( const std::vector<std::string>& files, const std::vector<long long>& fileMtime, const std::vector<std::uint64_t> … [line truncated: 25 more bytes on this line] +inline std::string makeHandle( const std::string& canonId, const std::string& path, const std::string& name, std::uint64_t contentHash ) +struct IngestFileScan +inline std::string spanTierMemoPath( const std::string& diskPath ) +inline RawFacts runParsePool( IngestResult& result, const char* rootDir, std::string_view cacheFile, bool captureValueUses, HashMap<std::string, FileFacts>& cac … [line truncated: 99 more bytes on this line] +inline void collectExtractPartials( const IngestFileScan& scan, IngestResult& result ) +inline std::uint64_t byteHash( const char* data, std::size_t n ) noexcept +… [48 more display lines; full output is 10037 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --for="rankGraphTeleport"` + +*Name-shaped query: the router picks name-exact BM25 (header says which/why).* + +````` + + + + + +PageRank with an explicit teleport / personalization vector p (Σp = 1). The prior is name-quality-biased through biasPrior() so all rank modes share one weighting seam; the transition matrix (edgesinline RankedGraph rankGraphTeleport( const Graph& g, const std::vector<float>&am … [line truncated: 31 more bytes on this line] +#### The convergence disclosure contract +### Wave-2 adversarial verification (2026-08-19) — six probes against `aa97c9e` + + + + + +& p, float alpha = 0.85f ) +{ + PROFILE_SCOPE_DESCRIBE( "rankGraph: PageRank (power iteration)" ); + const std::vector pw = biasPrior( g, p ); + const std::size_t N = pw.size(); + std::vector teleport( pw.begin(), pw.end() ); + std::vector rankDouble( N, 0.0 ); + PageRankRun run{}; // an N == 0 graph never enters the kernel: { 0, converged } — see PageRankRun + if( N ) + { + double teleportMass = 0.0; + for( const double value : teleport ) + { + teleportMass += value; + } + if( teleportMass > 0.0 ) + { +… [12 more display lines; full output is 4505 bytes on 29 raw line(s)] +````` + +## `./build/ripwire . --for="rankGraphTeleport" --no-route` + +*Same query with routing forced OFF (plain subtoken+body BM25) — contrast with the routed run.* + +````` + + + + + +uniform-teleport PageRank (the defaultinline RankedGraph rankGraph( const Graph& g, float alpha = 0.85f ) + +PageRank with an explicit teleport / personalization vector p (Σp = 1). The prior is name-quali…inline RankedGraph rankGraphTeleport( const Graph& g, const std::vector<float>& p, float alpha = 0.85f ) +inline ChurnRanking churnRankedGraph( const MainDispatch& d ) + +L10 (2026-09-04): the old wording claimed "the same corpus ranked by pagerank orders differently…inline constexpr const char* kChurnRankLegend = "<!-- rank_by=churn: k= is PageRank re-run with the teleport BIASED by git CHANGE-FREQUENCY over window= " "(a c… +struct RankedGraph +inline std::vector<float> takeRank( RankedGraph ranked, RankDisclosure& disclosureOut ) +inline std::vector<float> biasPrior( const Graph& g, const std::vector<float>& p ) +inline std::vector<float> churnTeleportWorkspace( const std::vector<std::string>& rootDirs, const IngestResult& ing, const char* since = "18 months ago& … [line truncated: 20 more bytes on this line] +inline ChurnRanking churnDecayRanking( const MainDispatch& d, const rw::SinceScope& sinceScope, bool isScoped, const char* verbLabel ) +inline std::string rankByText( const std::string& root, std::string_view mode, int topK, bool stable = false ) +PageRankRun pageRankDouble( const sparseCsr<float>& inEdges, std::span<const double> weightedOutDegree, std::span<const double> teleport, std::span<double> rank… +inline std::vector<float> churnDecayTeleport( const std::string& root, const IngestResult& ing, const SinceScope* scope = nullptr, bool* outHasChurnEvidence = n… +inline std::string didYouMean( const IngestResult& ing, std::string_view name ) +PageRankRun pageRankDouble( const sparseCsr<float>& inEdges, std::span<const double> weightedOutDegree, std::span<const double> teleport, std::span< … [line truncated: 45 more bytes on this line] +inline const McpIndex& getIndex( const std::string& root ) +inline std::vector<float> churnDecayTeleportWorkspace( const std::vector<std::string>& rootDirs, const IngestResult& ing, bool* outHasChurnEvidence = n … [line truncated: 12 more bytes on this line] +struct DataSectionsCut +inline int runEval( const std::string& root, const IngestResult& ing, const Graph& g, const std::vector<char>& currentDiff ) +inline std::vector<float> anchoredLexicalRank( const Graph& g, const std::vector<float>& lex ) +### Fixed — `rank_by` over MCP no longer answers a different ranking than `--rank-by` on … [line truncated: 35 more bytes on this line] +### 2. Gates for A +inline constexpr std::size_t kChurnMergeBombMaxFiles = 100 +#### The convergence disclosure contract +… [105 more display lines; full output is 16395 bytes on 72 raw line(s)] +````` + +## `./build/ripwire . --for="rankGraphTeleport" --signatures-only` + +*T3 opt-out: the signatures-only lens (no auto bodies, no bundle="auto" attribute) — contrast with the terminal default above.* + +````` + + + + + +PageRank with an explicit teleport / personalization vector p (Σp = 1). The prior is name-quality-biased through biasPrior() so all rank modes share one weighting seam; the transition matrix (edgesinline RankedGraph rankGraphTeleport( const Graph& g, const std::vector<float>&am … [line truncated: 31 more bytes on this line] +#### The convergence disclosure contract +### Wave-2 adversarial verification (2026-08-19) — six probes against `aa97c9e` + + + + +````` + +## `./build/ripwire . --for="tree-sitter parse of a source file" --adaptive` + +*Cut the result at the relevance cliff (Adaptive-k) — on a flat ranking nothing is cut and the header says so ([adaptive: kept N of N]).* + +````` + + + + +inline bool topLevelEvidence( std::string_view source, const TSLanguage* language, const Predicate& predicate ) +The crawl's per-file byte ceiling. A text file larger than this is skipped: at this size it is o…constexpr std::size_t kDefaultMaxFileBytes = 4u * 1024u * 1024u +TSTree* parseTree( TSParser* parser, std::string_view src ) + +Exercise every registered grammar and its embedded query, reporting loaded and expected totalsinline DoctorGrammarProbe doctorProbeGrammars() +struct FileHealth +enum class AstWalk : std::uint8_t +inline RevSide sliceAtRev( const std::string& root, const std::string& sha, const std::string& rel, const Symbol& sym, SliceFam fam, const ::TSLanguage* grammar…constexpr std::array<LangEntry, 53> kLangTable = +struct SliceScan +inline bool astroFenceTailIsBlank( std::string_view src, std::size_t from, std::size_t to ) noexcept +constexpr std::uint32_t kMaxKotlinStringNestDepth = 128u +std::vector<LocalNameFact> collectGatedLocalNames( std::string_view defBytes, std::uint32_t defStartLine, Lang lang ) +inline bool hasPhantomScopeSeparator( TSNode qualified ) noexcept +inline ModuleScan scanModule( std::string_view source, std::string_view path ) +bool jsonNestsTooDeep( std::string_view bytes ) noexcept +constexpr std::uint32_t kMaxYamlNestDepth = 64u +inline bool hasMainGuard( std::string_view source ) +SpanTierBatch spanTiersOfFiles( std::span<const std::string> diskPaths, bool useMemo ) +inline void runParseWorker( ParsePoolShared& sh, unsigned t ) +inline SliceScan sliceScanDefinition( const std::string& src, const Symbol& sym, SliceFam fam, const ::TSLanguage* grammar, std::string_view varName ) +struct LintRule +enum class DisclosureWhy : std::uint8_t +inline std::string buildMemoryStopAttr( const IngestResult& ing, bool json ) +inline bool hasPytestTable( std::string_view source ) +… [55 more display lines; full output is 9999 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --for="why does src/lexical.h chooseForRanker pick name-exact BM25"` + +*Mention anchoring (default-on): a path and a Symbol literally named in the task get lifted; the header says what anchored.* + +````` + + + + +The name-exact ranker AS THE RETRIEVAL LENS SERVES IT: whole-name BM25 plus the definition-over-…inline std::vector<float> lexicalScoresNameExactRanked( const IngestResult& ing, std::string_view query, const std::vector<float>* symbolScoreMul ) + +P11.12: the interpretive footer for --eval's ranker table, pulled into its own function so the 9…inline void printEvalRankerNote() +inline int runEvalRetrieval( const IngestResult& ing, const Graph& g ) + +calling agent knows to reformulate rather than trust the ranking. Calibrated empirically (2026-0…inline constexpr float kWeakLexicalScoreThreshold = 1.0f +inline RouteChoice chooseForRanker( const IngestResult& ing, std::string_view query ) +inline std::string candidatesRootTag( std::size_t keep, std::size_t corpusCount, const CandidateProvenance& prov ) +inline std::vector<float> lexicalScoresNameExactTiered( const IngestResult& ing, std::string_view query, const std::vector<float>* symbolScoreMul ) +inline int runEvalMined( const std::string& root, const IngestResult& ing, const Graph& g, const std::string& path ) +inline int runEvalSkills( const std::string& root, const IngestResult& ing, const Graph& g, const std::string& labelsPath ) +struct NameCorpusStats +struct LensRanking +inline bool hasIdentifierShape( std::string_view w ) noexcept +struct Bm25Params +inline int runEval( const std::string& root, const IngestResult& ing, const Graph& g, const std::vector<char>& currentDiff ) +inline double bm25ImpactBound( double idf, double T, const Bm25Params& p ) noexcept +rw::LensRanking computeLensRanking( const MainDispatch& d, std::string_view task, bool compactCandidate = false, bool fullDistribution = false ) // deep-tail: the file-grain tail rea … [line truncated: 65 more bytes on this line] +struct CandidateProvenance +inline constexpr int kLexWeightDoc = 2 +std::optional<int> runEvalViews( const MainDispatch& d ) +inline std::string_view takeQualifiedIdent( std::string_view src, std::size_t& i ) +inline std::vector<float> lexicalScoresNameExact( const IngestResult& ing, std::string_view query ) +struct RecallSectionPick +inline bool canonicalIdMatches( std::string_view canonAbs, std::string_view spec ) noexcept +… [75 more display lines; full output is 10887 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --for="why does src/lexical.h chooseForRanker pick name-exact BM25" --no-mention-boost` + +*Same task with the anchor disabled — the contrast the flag exists for.* + +````` + + + + +The name-exact ranker AS THE RETRIEVAL LENS SERVES IT: whole-name BM25 plus the definition-over-…inline std::vector<float> lexicalScoresNameExactRanked( const IngestResult& ing, std::string_view query, const std::vector<float>* symbolScoreMul ) + +P11.12: the interpretive footer for --eval's ranker table, pulled into its own function so the 9…inline void printEvalRankerNote() +inline int runEvalRetrieval( const IngestResult& ing, const Graph& g ) + +calling agent knows to reformulate rather than trust the ranking. Calibrated empirically (2026-0…inline constexpr float kWeakLexicalScoreThreshold = 1.0f +inline std::string candidatesRootTag( std::size_t keep, std::size_t corpusCount, const CandidateProvenance& prov ) +inline std::vector<float> lexicalScoresNameExactTiered( const IngestResult& ing, std::string_view query, const std::vector<float>* symbolScoreMul ) +inline int runEvalMined( const std::string& root, const IngestResult& ing, const Graph& g, const std::string& path ) +inline int runEvalSkills( const std::string& root, const IngestResult& ing, const Graph& g, const std::string& labelsPath ) +struct NameCorpusStats +struct LensRanking +inline bool hasIdentifierShape( std::string_view w ) noexcept +inline RouteChoice chooseForRanker( const IngestResult& ing, std::string_view query ) +struct Bm25Params +inline int runEval( const std::string& root, const IngestResult& ing, const Graph& g, const std::vector<char>& currentDiff ) +inline double bm25ImpactBound( double idf, double T, const Bm25Params& p ) noexcept +rw::LensRanking computeLensRanking( const MainDispatch& d, std::string_view task, bool compactCandidate = false, bool fullDistribution = false ) // deep-tail: the file-grain tail rea … [line truncated: 65 more bytes on this line] +struct CandidateProvenance +inline constexpr int kLexWeightDoc = 2 +std::optional<int> runEvalViews( const MainDispatch& d ) +inline std::string_view takeQualifiedIdent( std::string_view src, std::size_t& i ) +inline std::vector<float> lexicalScoresNameExact( const IngestResult& ing, std::string_view query ) +struct RecallSectionPick +inline bool canonicalIdMatches( std::string_view canonAbs, std::string_view spec ) noexcept +… [77 more display lines; full output is 10743 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --lego=Vehicle` + +*Interface -> implementors view: every existing impl of the named interface; the method contract is extracted for the C-family/Java/TS/Python tiers — for a Rust trait (this fixture) it discloses caveat="not-extracted-for-lang" rather than an empty list.* + +````` + + + + + + + + + +````` + +## `./build/ripwire . --exemplar="format byte sizes for humans"` + +*The repo's best-in-class instance to imitate before writing new code (picked by ROLE).* + +````` + + + + +ripwire '.' --verify='calls(runDefaultMap, rankGraphTeleport)' + + +````` + +## `./build/ripwire . --help-task="write a cheerful release announcement"` + +*The honest half of the contract: a task with no ripwire-shaped evidence ABSTAINS with zero commands rather than guessing.* + +````` + + + + +````` + +## `./build/ripwire . --recall="quality delta gating exit codes"` + +*Most relevant DOCS' full bodies (markdown only) — recall what is already written down.* + +````` +ripwire recall — "quality delta gating exit codes" — 113 relevant of 218 document files, best-first — total=113 shown=8 capped=1 truncated=7 generated_demoted=3 max_tokens=8000 share_bytes=2161 est_tokens=5028 + +━━ README.md (relevance 8.080) ━━ [sections: 2 of 25 selected (60 in doc), section-granular; whole doc 217180 B; lines="2746-2758,2759-2766"; dropped_by_budget=23] +#### 6.2 Exit codes + +This is the part to wire into a script. + +| Code | Meaning | +| --- | --- | +| 0 | The command completed. | +| 1 | The command refused the request. A refusal names the reason on stderr. When `--callers`, `--callees`, `--uses`, `--impact` or `--path` refuse a selector that matches no indexed definition, they also print an answer on stdout: the verb's element with `found="0"` and the name to retry with. | +| 2 | A policy gate fired: `--arch` found a layering violation, `--scan-skill` found a CRITICAL, `--quality-delta` found new debt. | +| 3 | The output exceeded the token budget that you set. | +| 4 | `--test-gate` found an open obligation. | +| 5 | The memory guard stopped the run: over `--max-memory` (default 65% of the machine's memory) with no partial answer possible, or a verb other than the default map facing a partial index. One stderr line names the limit and the override. | + +#### 6.3 JSON output + +`--json` emits JSON instead of XML, with keys mirroring the XML attribute names one to one. It is an +allow-list: the default map, `--for`, `--pack-task`, `--callers`, `--callees`, `--impact`, +`--quality-delta`, `--test-gate`, `--metrics`, and `--plan-lanes`, which is JSON-native. Every other +verb refuses on stderr with exit 1 rather than silently falling back to XML, so a verb added tomorrow +refuses by default. + + + +━━ prompts/help-wanted/quality-delta-unchanged-tree-zero.md (relevance 7.429) ━━ [sections: 1 of 18 selected (21 in doc), section-granular; whole doc 25463 B; lines="133-158"; dropped_by_budget=17] +### Root spelling (maintainer-owned; reproduce it once so you recognise it) + +```bash +… [150 more lines, 12829 bytes total] +````` + +## `./build/ripwire /aux/kbcorpus --recall="field affinity cache line data layout which fields are read together" --top-k=3 --max-tokens=1200` + +*The directory-as-knowledge-base pattern: --recall pointed at a 1392154-byte scratch dir of DUMPED TOOL OUTPUT (a git log, this repo's own generated command reference, --help text, an architecture doc, a fabricated JSON access log) instead of a source repo — no index to build, no daemon.* + +````` +ripwire recall — "field affinity cache line data layout which fields are read together" — 2 relevant of 2 document files, best-first — total=2 shown=1 capped=1 truncated=1 max_tokens=1200 share_bytes=2040 est_tokens=957 + +━━ commands.md (relevance 10.116) ━━ [sections: 1 of 160 selected (189 in doc), section-granular; whole doc 569309 B; lines="3756-3760"; dropped_by_budget=159] [truncated: 1865 of 7738 bytes] +### `--field-affinity[=STRUCT]` + +**Answers:** find fields that are read together but declared far apart — the cache-locality lens the CACHE-LOCALITY lens: which fields are READ TOGETHER but declared FAR APART. + +Builds a static field CO-ACCESS affinity graph (one observation per indexed C-family function body) and diffs it against the DECLARED field order and 64-byte cache-line geometry, reusing --layout's LP64 offset model. Bare = every aggregate in the repo, ranked by separation cost; =STRUCT narrows the … [line truncated: 1355 more bytes on this line] + +(capped: 1 of 2 relevant document files omitted — raise --max-tokens/budget_tokens or narrow the query for 1 more (~2548-byte budget)) +````` + +stderr: + +````` +ripwire: redacted 1 secret from emitted context (keyword-gated-secret=1) — pass --no-redact to disable +````` + +## `./build/ripwire . --tree` + +*File-by-file orientation map (top symbols per file).* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [367 more display lines; full output is 11224 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --html=/aux/map2.html` + +*Self-contained HTML force-directed call graph.* + +**wall time: 1.51s** + +````` +(empty) +````` + +Artifact written: + +````` + 159465 /aux/map2.html +````` + +## `./build/ripwire . --order=stable --top-k=5` + +*Stable (path/id) emit order — provider KV-cache hits across re-runs.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [67 more display lines; full output is 8280 bytes on 1 raw line(s)] +````` + + +--- + +# navigate / answer a question + +## `./build/ripwire . --around=rankGraphTeleport` + +*Ego graph around one symbol — depth 1 BY DEFAULT now (the root's depth= says so): ~6 KB where the 2-hop neighbourhood is ~64 KB on this repo.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [198 more display lines; full output is 9131 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --around=rankGraphTeleport --around-depth=2` + +*The restoring knob: --around-depth=2 brings back the whole 2-hop neighbourhood (depth="2" on the root) — pay for it only when the 1-hop view was not enough.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [2701 more display lines; full output is 80092 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --around=rankGraphTeleport --around-fanout=4` + +*The other knob: --around-fanout=4 keeps only the 4 strongest edges per node (default 32) — the same 1-hop depth, a quarter of the rows.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [20 more display lines; full output is 4221 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --callers=rankGraphTeleport` + +*Who calls SYM (1-hop in-edges).* + +````` + + + + + + + + + + +````` + +## `./build/ripwire . --callers=rankGraphTeleport --legend=full` + +*The same rows under --legend=full: the default is the compact dialect (one compact comment plus schema="ripwire.callers/v1" on the root), and --legend=full restores the prose legend — every row byte and every completeness attribute (counts_floor=, graph_ambiguous=, next=) identical, ~3 KB more legend. Works on EVERY XML verb.* + +````` + + + + + + + + + + + +````` + +## `./build/ripwire . --callers=DoesNotExist` + +*Unknown-symbol REFUSAL shape (exit 1) with a did-you-mean from real edit distance.* + +**exit code: 1** + +````` + + +````` + +stderr: + +````` +ripwire: --callers symbol not found: DoesNotExist +````` + +## `./build/ripwire . --callees=rankGraphTeleport` + +*What SYM calls (1-hop out-edges).* + +````` + + + + + + + + + + + +````` + +## `./build/ripwire . --uses=rankGraphTeleport` + +*The resolvable use-sites (call/read/write/import/extends) with file:line; count= is a floor.* + +````` + + + + + + + + + + + + +````` + +## `./build/ripwire . --graph-query='and(callers(name("rankGraphTeleport"),2),kind(all,fn))'` + +*Composable node-set query: functions within 2 caller-hops of rankGraphTeleport.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [28 more display lines; full output is 3649 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --external-surface` + +*Names referenced but never defined in-corpus (stdlib/third-party surface). The root carries names/shown/capped; the default is a 100-row window now, so total= and a pasteable next= join them when it bites (the explicit --limit form carries the same quintet). The sh BUILTINS (cd/echo/set…) are dropped and COUNTED as builtins_excluded= — grep/sed/git stay, they ARE the surface.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [73 more display lines; full output is 5589 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --external-surface --include-builtins` + +*The restoring knob: --include-builtins keeps the shell builtins in the same 100-row window (they now compete for rows, so the window's tail differs).* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [73 more display lines; full output is 5449 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --path=main,rankGraphTeleport` + +*Shortest directed call-path SRC -> DST. CHANGED: now reports from_p/to_p/from_defs and resolves the right `main` (was reachable="0").* + +````` + + + + + + + + +````` + +## `./build/ripwire . --connect=rankGraphTeleport,runEval,getIndex` + +*Minimal connecting subgraph over 3 symbols (finds shared-caller joins).* + +````` + + + + + + + + + + +````` + +## `./build/ripwire . --impact=rankGraphTeleport` + +*Transitive blast radius — everything that reaches SYM. NOW carries shown/capped.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [50 more display lines; full output is 5372 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --mentions=rankGraphTeleport` + +*Markdown docs that name SYM in a backtick (doc<->code edges).* + +````` + + + + + + + + + + +````` + +## `./build/ripwire . --affected=src/graph.h` + +*Test files that transitively reach the changed file.* + +````` + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --situ` + +*Mid-task situational report for the current git diff — recorded against a CLEAN tree (contrast with the sandbox run below).* + +````` +ripwire situational-awareness — 0 changed file(s), 0 symbols in them +root: . +at: 6009ca693 + (0 changed files — working tree is clean, nothing to analyze) +````` + +## `./build/ripwire . --test-gate` + +*Pre-PR gate on a CLEAN tree: no obligations, exit 0.* + +**wall time: 2.25s** + +````` + + +````` + +## `./build/ripwire . --grep=DISCLOSE` + +*Literal trigram-indexed search. Each hit carries its MATCHED line as the element's own CDATA (the wrapper is gone), plus shown/capped/hits_capped and a pasteable next= on the root.* + +````` + +if data.get( "schema" ) != SCHEMA: raise SystemExit( f"{path}: unexpected schema {data.get('schema')!r} (expecte +if not str( data.get( "tasks_lock_content_sha256", "" ) ).startswith( "questions:" ): train_repos = select_tasks +if train_repos: raise SystemExit( f"{path}: records from repo(s) that re-derive to LocBench +if base and ctx and base["status"] == "ok" and ctx["status"] == "ok": paired.append( ( instance_id, base["re +if not repos: return 0.0, [] +if base["localization_hit"] is None or ctx["localization_hit"] is None: return 0.0 +if bv: ratios.append( cv / bv - 1 ) +if not ratios: return None, None +if rw is None or native is None: return None +if not paired: out["note"] = "zero complete paired (baseline,ripwire_cli) runs — nothing to analyze yet" +if contaminated: print( f" ** {contaminated} baseline run(s) invoked ripwire despite the no-ripwire contract " +if "note" in out: print( f" {out['note']}" ); return +if out["n_pairs"] != 27: failures.append( f"expected 27 paired runs, got {out['n_pairs']}" ) +if out["n_incomplete"] != 2: failures.append( f"expected 2 incomplete pairs (the orphan + the contaminated-basel +if out["n_repos"] != 3: failures.append( f"expected 3 repos, got {out['n_repos']}" ) +if out.get( "n_contaminated_baseline" ) != 1: failures.append( f"expected exactly 1 contaminated baseline run co +if not ( out["resolved_delta_mean"] > 0 ): failures.append( "expected a positive resolved-rate delta" ) +if not ( out["resolved_delta_bootstrap_95_lower"] > 0 ): failures.append( "expected a POSITIVE bootstrap 95% low +if out["tokens_out_ratio_p50"] is None or abs( out["tokens_out_ratio_p50"] - 0.08 ) > 1e-6: failures.append( f"e +if out.get( "n_resolved_pairs" ) != 27: failures.append( f"expected all 27 pairs resolution-scored, got {out.get +if out.get( "substitution_rate_baseline" ) != 0.0: failures.append( f"expected baseline substitution rate 0.0 (n +if out.get( "substitution_rate_ripwire" ) is None or abs( out["substitution_rate_ripwire"] - 0.75 ) > 1e-9: fail +if out.get( "n_substitution_ripwire" ) != 27: failures.append( f"expected 27 substitution-scored ripwire runs, g +if out3.get( "substitution_rate_ripwire" ) is not None or out3.get( "n_substitution_ripwire" ) != 0: failures.ap +if out2["n_pairs"] != 27: failures.append( f"evaluator-none: expected 27 pairs, got {out2['n_pairs']}" ) +if out2["n_resolved_pairs"] != 0: failures.append( f"evaluator-none: expected 0 resolution-scored pairs, got {ou +if out2["resolved_delta_mean"] is not None or out2["resolved_delta_bootstrap_95_lower"] is not None: failures.ap +if out2["tokens_out_ratio_p50"] is None or abs( out2["tokens_out_ratio_p50"] - 0.08 ) > 1e-6: failures.append( " +… [73 more display lines; full output is 16956 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --match='(if_statement) @i'` + +*The same shape query WITH an explicit capture — identical hits, no auto_captured= attribute.* + +**wall time: 1.26s** + +````` + +if data.get( "schema" ) != SCHEMA: raise SystemExit( f"{path}: unexpected schema {data.get('schema')!r} (expecte +if not str( data.get( "tasks_lock_content_sha256", "" ) ).startswith( "questions:" ): train_repos = select_tasks +if train_repos: raise SystemExit( f"{path}: records from repo(s) that re-derive to LocBench +if base and ctx and base["status"] == "ok" and ctx["status"] == "ok": paired.append( ( instance_id, base["re +if not repos: return 0.0, [] +if base["localization_hit"] is None or ctx["localization_hit"] is None: return 0.0 +if bv: ratios.append( cv / bv - 1 ) +if not ratios: return None, None +if rw is None or native is None: return None +if not paired: out["note"] = "zero complete paired (baseline,ripwire_cli) runs — nothing to analyze yet" +if contaminated: print( f" ** {contaminated} baseline run(s) invoked ripwire despite the no-ripwire contract " +if "note" in out: print( f" {out['note']}" ); return +if out["n_pairs"] != 27: failures.append( f"expected 27 paired runs, got {out['n_pairs']}" ) +if out["n_incomplete"] != 2: failures.append( f"expected 2 incomplete pairs (the orphan + the contaminated-basel +if out["n_repos"] != 3: failures.append( f"expected 3 repos, got {out['n_repos']}" ) +if out.get( "n_contaminated_baseline" ) != 1: failures.append( f"expected exactly 1 contaminated baseline run co +if not ( out["resolved_delta_mean"] > 0 ): failures.append( "expected a positive resolved-rate delta" ) +if not ( out["resolved_delta_bootstrap_95_lower"] > 0 ): failures.append( "expected a POSITIVE bootstrap 95% low +if out["tokens_out_ratio_p50"] is None or abs( out["tokens_out_ratio_p50"] - 0.08 ) > 1e-6: failures.append( f"e +if out.get( "n_resolved_pairs" ) != 27: failures.append( f"expected all 27 pairs resolution-scored, got {out.get +if out.get( "substitution_rate_baseline" ) != 0.0: failures.append( f"expected baseline substitution rate 0.0 (n +if out.get( "substitution_rate_ripwire" ) is None or abs( out["substitution_rate_ripwire"] - 0.75 ) > 1e-9: fail +if out.get( "n_substitution_ripwire" ) != 27: failures.append( f"expected 27 substitution-scored ripwire runs, g +if out3.get( "substitution_rate_ripwire" ) is not None or out3.get( "n_substitution_ripwire" ) != 0: failures.ap +if out2["n_pairs"] != 27: failures.append( f"evaluator-none: expected 27 pairs, got {out2['n_pairs']}" ) +if out2["n_resolved_pairs"] != 0: failures.append( f"evaluator-none: expected 0 resolution-scored pairs, got {ou +if out2["resolved_delta_mean"] is not None or out2["resolved_delta_bootstrap_95_lower"] is not None: failures.ap +if out2["tokens_out_ratio_p50"] is None or abs( out2["tokens_out_ratio_p50"] - 0.08 ) > 1e-6: failures.append( " +… [73 more display lines; full output is 16840 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --query="teleport pagerank" --top-k=5` + +*Raw BM25 ranking (debug lens; --for is the real verb).* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [24 more display lines; full output is 3484 bytes on 1 raw line(s)] +````` + + +--- + +# zoom the detail ladder + +## `./build/ripwire . --for="pagerank power iteration" --detail=2` + +*Importance-weighted detail: FULL bodies for top-2, signatures for the rest.* + +````` + + + + + +NodeId order. Renumbering an otherwise-identical graph (the exact "same probe file, different so…inline constexpr float kScoreTieAbsEps = 1e-6f + +Render one form of the disclosure. Empty string whenever there is nothing to say — no power it…inline std::string renderDisclosure( const RankDisclosure& d, DiscloseAs as ) + +The PageRank power iteration itself — the numeric kernel every ranked document's order comes f…PageRankRun pageRankDouble( const sparseCsr<float>& inEdges, std::span<const double> weightedOutDegree, std::span<const double> teleport, std::span<double> r … [line truncated: 10 more bytes on this line] + +What a ranked document discloses about the power iteration that ordered it. `isPageRank == false…struct RankDisclosure +inline std::string rankByText( const std::string& root, std::string_view mode, int topK, bool stable = false ) +struct RankedGraph +inline constexpr const char* kChurnRankLegend = "<!-- rank_by=churn: k= is PageRank re-run with the teleport BIASED by git CHANGE-FREQUENCY over window= " "(a c… +inline std::uint32_t sliceRdIterationCeiling() noexcept +struct PageRankRun +inline std::uint32_t navRelevanceWeight( const Graph& g, NodeId n ) noexcept +inline RankedGraph rankGraphTeleport( const Graph& g, const std::vector<float>& p, float alpha = 0.85f ) +inline constexpr std::string_view kPowerShellPathPrependScope = "in PowerShell, for this window +HOT_FILES = ( "src/pagerank.cpp", # the power-iteration loop — G2's no-allocation scope "src/infra/radixSort.h", # LSD radix entry points "src/infra/radixSort.inl", # the counting … [line truncated: 76 more bytes on this line] +inline std::string powerShellPathPrependHint( std::string_view programDir ) noexcept +inline std::string powerShellSingleQuote( std::string_view s ) noexcept +inline std::string path_prepend_hint( std::string_view dir ) +handle_t leftmost_leaf() const +COLD_FILES = ( # ── ingest sections that are not on the per-file / per-symbol default path ──────────────────── ( "src/ingest_astquery.h", "the --match / - … [line truncated: 48 more bytes on this line] +inline void discloseUniformChurnFallback( bool hasChurnEvidence, bool stubbed, const char* verbLabel, const std::string& windowStamp ) +inline std::string doctorPathPrependLine( const std::string& selfPath ) +### Wave-2 adversarial verification (2026-08-19) — six probes against `aa97c9e` +### Same answer, a fraction of the tokens — read this table first if your agent is on a budget +… [108 more display lines; full output is 14910 bytes on 72 raw line(s)] +````` + +## `./build/ripwire . --pack-signatures --top-k=10` + +*Body-elided decl skeletons — recounted on this corpus. Measured as element bytes: the signature+doc elements --pack-signatures emits, against the SAME symbols' full bodies from --expand, with the CORPUS-ROOT PREFIX SUBTRACTED FROM BOTH SIDES. That subtraction is the whole methodology and the figure is meaningless without it: the root repeats inside every element's id= and p=, it is not what this verb elides, and counting it makes the headline a function of how deep the checkout happens to sit on disk — on one corpus, three spellings of the same root read 18.6 points apart before the subtraction and agree exactly after it. Root-neutralised on THIS repo: 92.7% fewer bytes at top-10, 88.7% at top-50, 87.6% at top-100 (re-derived 2026-09-20 when the native Windows port landed: src/infra/os_win32.cpp and os_win32_logic.h are ~3,900 lines of new, heavily commented infra that the ranked top-10/50/100 now reach, and their bodies are this ratio's DENOMINATOR, so the figure rises without --pack-signatures eliding anything new, from top-50 87.1. Measured by test/showcasecapturecheck.sh's own recount arm on the combined tree, which is the gate that would otherwise report the drift; before that, re-derived 2026-09-17 at the self-check macro vocabulary rename: VERIFY/VERIFY_TEXT/VERIFY_DEBUG_ONLY/VERIFY_NOT_REACHED/VERIFY_SAME_THREAD/VERIFY_NO_ALIAS*/DYNMAP_VERIFY/DEGRADED_PATH_ALERT renamed to ASSUME/EXPECTS/ENSURES/DASSERT/UNREACHABLE/ASSUME_SAME_THREAD*/ASSUME_NO_ALIAS*/DYNMAP_ASSUME/DISCLOSE (plus new VALIDATE sites) across 839 identifier renames in 179 files, and a hand-written Diagnostics.h/diagnostics.cpp replacing the old ones outright: this moves both WHICH symbols the ranked top-10/50/100 hold and how large their bodies are, from top-50 85.2. A real re-derivation of a corpus that changed, not a tolerance edit; previously re-derived 2026-09-12 when the rows dropped the path-repeating id= for the short sc= scope: the signature side shrank, the bodies did not; before that, 89.5% / 81.8% / 84.3% re-derived 2026-09-10 at the sibs= cap raise: kMaxExpandSibs went 8 -> 100, so --expand's bodies now carry the file context the old cap hid — 89.3% of all sibling names — and the body side is this ratio's DENOMINATOR, so the figure rises without --pack-signatures eliding anything new. Measured on a fixed tree with the top-50 membership and the signature side unchanged: top-50 from 71.0. A real re-derivation of a corpus that changed, not a tolerance edit; previously re-derived 2026-09-09 at the printf-family -> std::print conversion: converting ~1,500 emitter call sites to rw::emitTo/emitRaw/formatTo across 93 files changes how large the ranked symbols' BODIES are, and the body side is this ratio's denominator — top-50 from 72.3. A real re-derivation of a corpus that changed, not a tolerance edit; previously re-derived 2026-09-08 at the confident-zero round, issues #62/#63/#66: that change adds symbols to src/graphlegend.h and the new src/preprocdead.h and re-homes two long comment blocks from call sites onto the helpers they explain, which moves both WHICH symbols the ranked top-50 holds and how large their bodies are — top-50 from 74.0. A real re-derivation of a corpus that changed, not a tolerance edit; previously re-derived 2026-09-06 at the stranger-audit fix round: the doctor, cache-sweep and html-provenance bodies grew this corpus's BODY side, moving top-50 from 75.6 — a real re-derivation, not a tolerance edit; before that, re-derived 2026-09-05 at the capture-audit close: lane L7's P16 caps --expand's sibs= at 8 names, which SHRINKS the body side of this ratio and moved the figure down from 84.5/80.2/80.6 — the V1 2026-08-15 re-center, when sibs=/inc= first grew the body side from 70.0/61.0/63.8, in reverse; both were real re-derivations, not tolerance edits). top-50 is the number to quote, because the sigs payload is top-50 regardless of --top-k and is therefore what THIS command emits. A single small/trivial body can still invert it (signature+doc bigger than the body), like the --format=columnar sibling below. test/showcasecapturecheck.sh (C) re-derives all three from this repo every run, in the same quantity, and fails if the caption and the recount drift apart.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [132 more display lines; full output is 11518 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --outline=rankGraphTeleport --top-k=0` + +*Control-flow skeleton of one symbol, payload-only via the new --top-k=0.* + +````` + + + + +& p, float alpha = 0.85f ) +{ + PROFILE_SCOPE_DESCRIBE( "rankGraph: PageRank (power iteration)" ); + const std::vector pw = biasPrior( g, p ); + const std::size_t N = pw.size(); + std::vector teleport( pw.begin(), pw.end() ); + std::vector rankDouble( N, 0.0 ); + PageRankRun run{}; // an N == 0 graph never enters the kernel: { 0, converged } — see PageRankRun + if( N ) + { + ... + } + std::vector r( N, 0.f ); + std::transform( rankDouble.begin(), rankDouble.end(), r.begin(), []( double value ) { return float( value ); } ); + return { std::move( r ), run.iterationCount, run.hasConverged }; +} +]]> +````` + +## `./build/ripwire . --outline=rankGraphTeleport:1-10 --top-k=0` + +*CHANGED: a line range on --outline is now STRIPPED with a stderr note (it used to refuse).* + +````` + + + + +& p, float alpha = 0.85f ) +{ + PROFILE_SCOPE_DESCRIBE( "rankGraph: PageRank (power iteration)" ); + const std::vector pw = biasPrior( g, p ); + const std::size_t N = pw.size(); + std::vector teleport( pw.begin(), pw.end() ); + std::vector rankDouble( N, 0.0 ); + PageRankRun run{}; // an N == 0 graph never enters the kernel: { 0, converged } — see PageRankRun + if( N ) + { + ... + } + std::vector r( N, 0.f ); + std::transform( rankDouble.begin(), rankDouble.end(), r.begin(), []( double value ) { return float( value ); } ); + return { std::move( r ), run.iterationCount, run.hasConverged }; +} +]]> +````` + +stderr: + +````` +ripwire: --outline=rankGraphTeleport:1-10: --outline has no line-range form — outlining the whole symbol (use --expand=rankGraphTeleport:1-10 for a body slice) +````` + +## `./build/ripwire . --expand=rankGraphTeleport --top-k=0` + +*Full body + inline callee signatures.* + +````` + + + + +, with a BRANCH-FREE size(). Both, not one or the other. +// +// ── THE DESIGN, AND WHY IT IS THIS ONE ─────────────────────────────────────────────────────────────── +// The shape the host tree leans on hardest is `Map>` — many tiny id-lists (byName / +// canonByName / shard maps, and ~100 more structures after the conversion wave): WRITE-ONCE during the +// parse/merge, then READ-HOT during resolve. +// +// Three things matter for that shape, and the layout below gets all three: +// • no per-list malloc — the N small lists that would each allocate are inline; +// • a BRANCH-FREE size() — `return sz_`, because the size lives in its own field; +// • 16 BYTES per instance — because `inl_` and `heap_` are never both live, so they share storage. +// +// The union is the whole trick. The previous revision of this file paid 8 extra bytes (24 B) for the +// explicit size field, on the reasoning that a branch-free size() was worth it. That was a false +// choice: union the inline array with the heap pointer and the struct reaches 16 B — ankerl's size — +// while the size stays in its own field and size() stays branch-free. +// +// THE TRADE THAT REMAINS, stated honestly. At 16 bytes you can have: +// • ankerl::svector's 3 inline slots, with a size() that branches on is_direct() (and, once spilled, +// dereferences into the heap block to read the size — a dependent load, not just a branch); or +// • this type's 2 inline slots, with size() branch-free. +// The measurement chose the second. See bench/SVECTORAB.md; the short version is below. +// +… [1086 more display lines; full output is 66262 bytes on 1112 raw line(s)] +````` + +stderr: + +````` +ripwire: --pack-top-n is deprecated — use --pack-task/--detail instead (unchanged behavior for now) +````` + + +--- + +# assess quality / structure + +## `./build/ripwire . --metrics --top-k=10` + +*Fan-in/out + complexity annotations on the map.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [91 more display lines; full output is 10820 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --deps` + +*File->file dependency graph (god-files, cycles).* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [898 more display lines; full output is 23488 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --hotspots` + +*Complexity x recent git churn (maintenance pain).* + +**wall time: 1.48s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [14 more display lines; full output is 5749 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --clones` + +*Token-normalized duplicate bodies.* + +**wall time: 2.10s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [306 more display lines; full output is 16116 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --cochange` + +*Files that change together in git (hidden coupling).* + +**wall time: 1.52s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --hotspots --since="2 weeks ago"` + +*Hotspots scoped to RECENT churn (the regression lens).* + +**wall time: 1.36s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [13 more display lines; full output is 5089 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --arch=test/archfix/rules.txt` + +*Enforce layering rules (exit 2 on violation) — run against the repo's own test fixture rules.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [678 more display lines; full output is 99226 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --lint` + +*Built-in AST checks (c-cast, goto, unsafe-c-fn, ...).* + +**wall time: 2.04s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [695 more display lines; full output is 70057 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --lint-rules=test/lintrulesfix/rules` + +*User lint rules (YAML, ast-grep style) from a directory.* + +````` + + + + +use LOG() instead of printf +use LOG() instead of printf +use LOG() instead of printf +use LOG() instead of printf +use LOG() instead of printf + +````` + +stderr: + +````` +ripwire: lint-rules: test/lintrulesfix/rules/malformed.yaml:5: expected 'key: value' — file skipped +[math degraded] lint-rules: malformed rule file skipped (lintrules.h:395, auto rw::parseLintRuleFile(const std::string &, std::string_view, std::vector &)::(anonymous class)::operator()(std::size_t, const char *) const — logged once per site) +ripwire: AST query did not compile for any grammar: (this_is_not_a_real_node @x @@@ ((( ) +````` + +## `./build/ripwire . --lint --with-profile=report.txt` + +*Join MEASURED heat onto --lint findings — runs in a tiny fabricated demo corpus (one cache-pointer-chase-loop finding under a PROFILE_SCOPE site) because a real report needs a RIPWIRE_PROFILE build; the finding inside the profiled scope gains heat_* columns from the report's #PROF_TSV row.* + +Input file: + +````` +#PROF_TSV_BEGIN one row per scope, aggregated across threads; counters are RAW integers +scope file line calls total_ms l1d_mpki +walk: chase pass x.cpp 9 12 48.500 7.250 +#PROF_TSV_END +````` + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [13 more display lines; full output is 3428 bytes on 1 raw line(s)] +````` + +The joined finding — past the display cut above, extracted so the join is visible: + +````` +p = p->next +````` + +## `./build/ripwire . --communities` + +*Cluster the call graph into cohesive modules.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [188 more display lines; full output is 17290 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --zoom` + +*Nested module hierarchy (multi-level Louvain) + cross-module bridges — levels_shown="2" of levels= BY DEFAULT over the 40 largest top modules (~8 KB, where the whole tree is ~220 KB); a module AT the cut carries children=.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [171 more display lines; full output is 8794 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --zoom --zoom-levels=3` + +*The restoring knob: --zoom-levels=N prints N levels (0 = the whole tree); here three, ~11 KB.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [707 more display lines; full output is 28629 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --report` + +*Architecture summary (modules, god-files, cycles) as markdown.* + +````` + + +# ripwire architecture report + +2787 files · 26903 symbols · 41775 edges · 2640 modules (11994 call-graph isolated) + +Root: `.` + +Call-graph isolate provenance: 3221 declaration, 1908 header, 2215 source, 4650 document; 0 connected Louvain singletons + +## Modules (call-graph clusters; showing 12 of 2640) +- **src::emitTo@infra/emit.h:53:2760 [write,run,resolve]** — 1714 symbols +- **src::ASSUME@infra/Diagnostics.h:386:25510 [resolve,collect,add]** — 934 symbols +- **src::append@elixir_resolve.h:111:4875 [resolve,scan,read]** — 644 symbols +- **src::kindIs@infra/nodekind.h:46:3271 [push,emit,collect]** — 366 symbols +- **src::escapeXml@serialize.h:155:10267 [run,emit,write]** — 201 symbols +- **docs::`--top-k=N`@COMMANDS.md:59:8950 [pack,allow,insert]** — 179 symbols +- **src/infra::fail@os_win32.cpp:63:2553 [close,open,release]** — 108 symbols +- **test/ownerhopfix/py/pkg::big_handler_001@broadcast.py:3:72** — 106 symbols +- **src::relForHash@model.h:1634:133594 [clone,compute,scan]** — 53 symbols +- **src/infra::DYNMAP_ASSUME@dynamic_map.hpp:90:4411 [erase,find,insert]** — 50 symbols +- **test/ownerhopfix/gain/pkg::target_01@fanout.py:74:1458 [push]** — 41 symbols +- **src::rowOffsets@infra/sparseCsr.h:248:8990 [connect,verify,make]** — 39 symbols + +## God files (most depended-on; showing 10 of 489) +- `test/lib/clean-env.sh` — 186 dependents +- `src/model.h` — 89 dependents +- `src/infra/emit.h` — 86 dependents +- `src/infra/Diagnostics.h` — 60 dependents +- `src/serialize.h` — 40 dependents +… [33 more lines, 3676 bytes total] +````` + +## `./build/ripwire . --seams` + +*Cross-module call seams no test reaches. NOW carries seam_pairs/shown/capped.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [90 more display lines; full output is 12235 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --mermaid` + +*Module (directory) dependency graph as a Mermaid diagram.* + +````` +%% ripwire --mermaid: module (directory) dependency graph — node = dir (symbol count), edge = inter-module calls (>= 3). Render at mermaid.live. +flowchart LR + subgraph sg0 ["src"] + n101["src
7915"] + n102["src/infra
1280"] + end + subgraph sg1 ["test"] + n103["test
5126"] + n392["test/legochargefix
546"] + n333["test/fixtures/recallpassage
314"] + n448["test/ownerhopfix/py/pkg
158"] + n280["test/expandmodefix
151"] + n282["test/expandsibsfix
149"] + n589["test/rubyschemafix
99"] + n571["test/rubyattrsfix
85"] + n424["test/massfix
77"] + end + n73["docs
836"] + n2[".github
606"] + n0["(root)
547"] + subgraph sg5 ["bench"] + n4["bench
545"] + n5["bench/agentloop
331"] + n49["bench/locbench/results/r5_pooling
237"] + n50["bench/locbench/results/r6_expansion
187"] + n59["bench/recalleval
113"] + n43["bench/locbench
107"] + n38["bench/headtohead/r3-headroom-2026-08-03
102"] + n65["bench/skillrater/results/2026-09-07/arms
100"] + n62["bench/shotgun
96"] +… [20 more lines, 1599 bytes total] +````` + +## `./build/ripwire . --owners` + +*Bus-factor: recency-weighted author ownership per file.* + +**wall time: 1.97s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [921 more display lines; full output is 101244 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --dead-code=src` + +*Internal functions with no caller found in the index — a name-based graph reading, not a confidence score. NOTE the filter is a path-COMPONENT match: 'src' matches any .../src/... segment; use ./src to pin the root directory.* + +````` + + + + +````` + +## `./build/ripwire . --exercises=test/regression.sh` + +*Which symbols a TEST FILE exercises — the reverse direction of --affected.* + +````` + + + +````` + +## `./build/ripwire . --community=0` + +*Drill into ONE call-graph community by id — the drill= the --communities output itself advertises.* + +````` + + + +````` + +## `./build/ripwire . --quality-delta` + +*On a CLEAN tree: nothing got worse, exit 0. The gating shape is in the sandbox section below.* + +**wall time: 12.88s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [120 more display lines; full output is 14810 bytes on 1 raw line(s)] +````` + +stderr: + +````` +ripwire: no ./.ripwire_quality_baseline — auto-comparing the working tree vs git HEAD (commit the baseline with --quality-baseline to pin it) +````` + +## `./build/ripwire . --edit-check=rankGraphTeleport` + +*Fast per-symbol post-edit contract check vs git HEAD (unchanged on a clean tree).* + +````` + + + + + + + + + +````` + +## `./build/ripwire . --pr-context` + +*No-LLM review-evidence bundle for the working-tree diff (clean tree = empty).* + +````` + + + +````` + +## `./build/ripwire . --pr-context=HEAD~1` + +*The BASEREF form: diffed against merge-base(BASEREF, HEAD), never the ref tip — here the previous commit on the current line (a ref with NO merge base falls back to a disclosed two-dot diff: anchor="ref-tip-two-dot").* + +**wall time: 2.41s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [258 more display lines; full output is 17245 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --merge-scout=HEAD~2,HEAD~1` + +*Pairwise cross-arm conflict sites + suggested landing order (any committish sharing a merge base with HEAD works as an arm; one that does not is reported ok="0", never compared).* + +**wall time: 11.92s** + +````` + + + + + + + + + + + +````` + +## `./build/ripwire . --stray-content=lane/` + +*Which refs of the `lane/` ref family still hold divergent authored work vs HEAD, with verdicts.* + +**wall time: 5.78s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [218 more display lines; full output is 28936 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --stray-content=arm/for-howitworks-067-f2add` + +*A second selection, the one branch `arm/for-howitworks-067-f2add` (no ref family left to select on this checkout), picked at capture time from the refs this checkout really has and never the selection above: merged refs are OMITTED from the rows and counted in merged=; refs sharing no merge base with HEAD (a shallow clone, or a pre-rewrite history) land in unknown= with ok="0" — the counters always reconcile against refs=.* + +````` + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --stray-content=lane/ --plan` + +*Select the genuinely-unmerged refs of the `lane/` ref family and feed them to merge-scout for a landing order (a fully merged selection yields an empty landing set — still a measurement, disclosed on the root).* + +**wall time: 37.51s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [38564 more display lines; full output is 5687492 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --stray-content=zzzz-no-such-ref --plan` + +*A --plan filter that selects NO ref REFUSES (exit 1) naming the substring — before the wave-3 close this fell through to the '>512 refs match' sentence, and --abi under the same filter answered an empty measurement at exit 0.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --plan: --stray-content=zzzz-no-such-ref matches no local ref — a zero here would be a failure, not a measurement + (the filter is a substring match against refs/heads names; run bare --stray-content to list them, e.g. --stray-content=feat/) +````` + +## `./build/ripwire . --stray-content=lane/ --abi` + +*Cross-branch ABI-break gate over the `lane/` ref family: struct byte-contract drift on each ref's AUTHORED paths — exit 2 when any drift row is found (the only kind that gates), 0 when the compared refs are clean, and exit 1 if the --stray-content filter matches no ref at all.* + +**exit code: 2** — **wall time: 18.79s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --whereis=rankGraphTeleport` + +*Which ref's tree defines or mentions SYM — HEAD first, then every local branch. The default lists the kind="def" rows (here the first 60, most on branches, each carrying tip= and date=) and counts the references in : on this symbol HEAD's references would fill the 60-row cap before the branch definitions arrive, so the whole list (below) shows fewer definitions, and the default serves the page that shows more of them, whatever its bytes.* + +**wall time: 4.93s** + +````` + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --whereis=rankGraphTeleport --whereis-listing=all` + +*The whole hit list asked for by name: under the same 60-row cap HEAD's references fill the list, so it shows far fewer definitions (here HEAD's one) than the default above (here 60). The default serves the all page only when it shows the same definitions as the defs page in strictly fewer bytes (a tie serves it too).* + +**wall time: 5.04s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [35 more display lines; full output is 14367 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --whereis=computeOnePairOverlap --with-history` + +*Same, plus a git-history row (never / removed-by-commit) for names no tree carries.* + +**wall time: 11.73s** + +````` + + + + + + + +````` + +## `./build/ripwire . --flags` + +*The dark-content dashboard: gates BUILT but OFF. CHANGED: no longer invents gates from comments/heredocs, so the count only reflects real ifndef/define, CMake option(), and getenv gates.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [378 more display lines; full output is 23271 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --flags --flip=RIPWIRE_ASAN` + +*Blast radius of turning ONE gate on: live code, symbols, transitive reach, covering tests.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --flags --flip=RIPWIRE_ASA` + +*Unknown-gate refusal (exit 1) with a did-you-mean from a real edit distance (one character off RIPWIRE_ASAN).* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --flip: no gate named 'RIPWIRE_ASA' in . (did you mean 'RIPWIRE_ASAN'?) +ripwire: run `ripwire . --flags` for the gate table +````` + +## `./build/ripwire . --plan-lanes=3 --task="add a --since filter to the doc-drift verb and cover it with tests"` + +*NEW VERB: pre-hoc lane plan — which of 3 parallel worktrees would COLLIDE, before a line is written. JSON on stdout.* + +**wall time: 1.95s** + +````` +{"v":1,"verb":"plan-lanes","at":"6009ca693","root":".","task":"add a --since filter to the doc-drift verb and cover it with tests","source":"partition","requested":3,"lane_count":3,"claim_key":"path+scope+name","on_conflict":"producing-lane-rebases","corpus":{"files":2787,"symbols":26903,"edges":417 … [line truncated: 339 more bytes on this line] +"symbols":[{"p":"./src/docdrift.h","n":"DriftResult","scope":"DriftResult","l":286,"id":"./src/docdrift.h::DriftResult::DriftResult"}, +{"p":"./src/docdrift.h","n":"computeDocDrift","scope":"docdrift","l":2441,"id":"./src/docdrift.h::docdrift::computeDocDrift"}, +{"p":"./src/docdrift.h","n":"kDocDriftLegend","scope":"docdrift","l":2739,"id":"./src/docdrift.h::docdrift::kDocDriftLegend"}, +{"p":"./src/docdrift.h","n":"writeDocDriftPage","scope":"docdrift","l":2909,"id":"./src/docdrift.h::docdrift::writeDocDriftPage"}, +{"p":"./src/mcpverbs.h","n":"docDriftText","scope":"rw","l":651,"id":"./src/mcpverbs.h::rw::docDriftText"}, +{"p":"./src/verbs_change.h","n":"runDocDrift","scope":"","l":1675,"id":null}]},"lanes":[{"id":"lane-0","task":"add a --since filter to the doc-drift verb and cover it with tests","claims":{"symbols":[{"p":"./src/verbs_change.h","n":"runAbiCheck","scope":"","key":"183da9aaf74a4a92","id":null,"id_addr … [line truncated: 113 more bytes on this line] +{"p":"./src/gitmine.h","n":"gitWindowBoundarySha","scope":"rw","key":"18627516699d7062","id":"./src/gitmine.h::rw::gitWindowBoundarySha","id_addressable":true,"id_collides_with":0,"l":1672,"ord":0,"overloads":1,"amb":3,"cx":3,"ccx":2,"churn":71,"tested":0}, +{"p":"./src/docdrift.h","n":"writeGateability","scope":"docdrift","key":"42c456dfb55faee4","id":"./src/docdrift.h::docdrift::writeGateability","id_addressable":true,"id_collides_with":0,"l":2685,"ord":0,"overloads":1,"amb":3,"cx":6,"ccx":6,"churn":36,"tested":0}, +{"p":"./src/cli.h","n":"inHostActive","scope":"rw","key":"4446994deb74226c","id":"./src/cli.h::rw::inHostActive","id_addressable":true,"id_collides_with":0,"l":4589,"ord":0,"overloads":1,"amb":0,"cx":1,"ccx":0,"churn":469,"tested":0}, +{"p":"./src/darkflags.h","n":"writeFlags","scope":"darkflags","key":"5c2c4a2b311b8dba","id":"./src/darkflags.h::darkflags::writeFlags","id_addressable":true,"id_collides_with":0,"l":1298,"ord":0,"overloads":1,"amb":2,"cx":8,"ccx":8,"churn":32,"tested":0}, +{"p":"./src/docdrift.h","n":"recordWeakDisclosures","scope":"docdrift","key":"972779210e49c0cf","id":"./src/docdrift.h::docdrift::recordWeakDisclosures","id_addressable":true,"id_collides_with":0,"l":2430,"ord":0,"overloads":1,"amb":2,"cx":2,"ccx":1,"churn":36,"tested":0}, +{"p":"./src/testmap.h","n":"scriptGatesUnmodelledCount","scope":"rw","key":"98c4487e2e9e08bc","id":"./src/testmap.h::rw::scriptGatesUnmodelledCount","id_addressable":true,"id_collides_with":0,"l":1344,"ord":0,"overloads":1,"amb":0,"cx":4,"ccx":4,"churn":48,"tested":0}, +{"p":"./src/main.cpp","n":"scanReportVerbPrecedence","scope":"","key":"9ab981e987e8108e","id":null,"id_addressable":false,"id_collides_with":0,"l":3029,"ord":0,"overloads":1,"amb":33,"cx":8,"ccx":11,"churn":439,"tested":0}, +{"p":"./src/gitmine.h","n":"resolveSinceScope","scope":"rw","key":"9caaa3dbeaa688d0","id":"./src/gitmine.h::rw::resolveSinceScope","id_addressable":true,"id_collides_with":0,"l":331,"ord":0,"overloads":1,"amb":3,"cx":7,"ccx":7,"churn":71,"tested":0}, +{"p":"./src/main.cpp","n":"emitNoteAddUnprovenDefs","scope":"","key":"a950332ccfce6b8d","id":null,"id_addressable":false,"id_collides_with":0,"l":689,"ord":0,"overloads":1,"amb":1,"cx":2,"ccx":1,"churn":439,"tested":0}, +{"p":"./src/taskroute.h","n":"testCoverageTaskChoice","scope":"rw::taskroute","key":"bf72991aa28355c7","id":"./src/taskroute.h::rw::taskroute::testCoverageTaskChoice","id_addressable":true,"id_collides_with":0,"l":1388,"ord":0,"overloads":1,"amb":1,"cx":3,"ccx":2,"churn":41,"tested":0}, +{"p":"./src/mcpverbs.h","n":"unknownSubVerbRefusal","scope":"rw","key":"e336d39b0a6addea","id":"./src/mcpverbs.h::rw::unknownSubVerbRefusal","id_addressable":true,"id_collides_with":0,"l":5469,"ord":0,"overloads":1,"amb":1,"cx":3,"ccx":2,"churn":359,"tested":0}, +{"p":"./src/cli.h","n":"validateModifierGuards","scope":"rw","key":"ec2848a4801493a2","id":"./src/cli.h::rw::validateModifierGuards","id_addressable":true,"id_collides_with":0,"l":4617,"ord":0,"overloads":1,"amb":33,"cx":73,"ccx":66,"churn":469,"tested":0}], +"files":[{"p":"./src/cli.h","symbols":2,"churn":469,"ccx":66,"hotspot_rank":6}, +{"p":"./src/darkflags.h","symbols":1,"churn":32,"ccx":8,"hotspot_rank":45}, +{"p":"./src/docdrift.h","symbols":2,"churn":36,"ccx":7,"hotspot_rank":32}, +{"p":"./src/gitmine.h","symbols":2,"churn":71,"ccx":9,"hotspot_rank":17}, +{"p":"./src/main.cpp","symbols":2,"churn":439,"ccx":12,"hotspot_rank":4}, +{"p":"./src/mcpverbs.h","symbols":1,"churn":359,"ccx":2,"hotspot_rank":5}, +{"p":"./src/taskroute.h","symbols":1,"churn":41,"ccx":2,"hotspot_rank":38}, +{"p":"./src/testmap.h","symbols":1,"churn":48,"ccx":4,"hotspot_rank":36}, +{"p":"./src/verbs_change.h","symbols":1,"churn":78,"ccx":9,"hotspot_rank":22}]},"blast_radius":{"reaches":47,"files_total":17,"capped":false,"files":["./src/cli.h","./src/docdrift.h","./src/editplan.h","./src/gitmine.h","./src/main.cpp","./src/mcp.h","./src/mcpedit.h","./src/mcpserver.h","./src/mcpv … [line truncated: 194 more bytes on this line] +"tests_total":0,"tests_capped":false,"tests_granularity":"claimed-symbols","untested":47,"module_span":1,"notes":[], +"execution":{"policy":"codex-lane/v1","model":"gpt-5.6-sol","reasoning":"xhigh","rule":"wide-contract-work","basis":"structural-only","signals":{"claims":13,"files":9,"module_span":1,"max_ccx":66,"sum_ccx":119,"ambiguous_calls":85,"blast_reaches":47,"contract_touches":1,"conflicts":0,"untested":47," … [line truncated: 140 more bytes on this line] +… [75 more display lines; full output is 20891 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --plan-lanes --brief=/aux/lanes_brief.txt` + +*NEW VERB, explicit form: one line per lane, lane boundaries are the ones you wrote (the defensible mode).* + +**wall time: 1.81s** + +Input file: + +````` +add a --since filter to the doc-drift verb +add the CLI parse arm and help text for the new filter +write regression tests for the new filter +````` + +````` +{"v":1,"verb":"plan-lanes","at":"6009ca693","root":".","task":null,"source":"brief","requested":3,"lane_count":3,"claim_key":"path+scope+name","on_conflict":"producing-lane-rebases","corpus":{"files":2787,"symbols":26903,"edges":41775,"ambiguous":12805,"unresolved":12845},"carve":null,"core":{"files … [line truncated: 5 more bytes on this line] +"symbols":[]},"lanes":[{"id":"lane-0","task":"add a --since filter to the doc-drift verb","claims":{"symbols":[{"p":"./src/docdrift.h","n":"recordUnchecked","scope":"docdrift","key":"12641dab14abc8fd","id":"./src/docdrift.h::docdrift::recordUnchecked","id_addressable":true,"id_collides_with":0,"l":2 … [line truncated: 72 more bytes on this line] +{"p":"./src/docdrift.h","n":"sortWeakGroupsByPath","scope":"docdrift","key":"1944ba506280ff80","id":"./src/docdrift.h::docdrift::sortWeakGroupsByPath","id_addressable":true,"id_collides_with":0,"l":2401,"ord":0,"overloads":1,"amb":0,"cx":1,"ccx":0,"churn":36,"tested":0}, +{"p":"./src/mcpverbs.h","n":"docDriftText","scope":"rw","key":"1fa68e8d93c05a59","id":"./src/mcpverbs.h::rw::docDriftText","id_addressable":true,"id_collides_with":0,"l":651,"ord":0,"overloads":1,"amb":0,"cx":1,"ccx":0,"churn":359,"tested":0}, +{"p":"./src/cli.h","n":"kViewFlags","scope":"rw","key":"2a2f2487082cc6d8","id":"./src/cli.h::rw::kViewFlags","id_addressable":true,"id_collides_with":0,"l":3255,"ord":0,"overloads":1,"amb":0,"cx":0,"ccx":0,"churn":469,"tested":0}, +{"p":"./src/recall.h","n":"docFileMask","scope":"rw","key":"3149a219f599664c","id":"./src/recall.h::rw::docFileMask","id_addressable":true,"id_collides_with":0,"l":110,"ord":0,"overloads":1,"amb":0,"cx":4,"ccx":4,"churn":27,"tested":0}, +{"p":"./src/docdrift.h","n":"writeDocDriftPage","scope":"docdrift","key":"380b7de5df1cfd73","id":"./src/docdrift.h::docdrift::writeDocDriftPage","id_addressable":true,"id_collides_with":0,"l":2909,"ord":0,"overloads":1,"amb":12,"cx":14,"ccx":15,"churn":36,"tested":0}, +{"p":"./src/docdrift.h","n":"computeDocDrift","scope":"docdrift","key":"3b19cc3d8996c3b2","id":"./src/docdrift.h::docdrift::computeDocDrift","id_addressable":true,"id_collides_with":0,"l":2441,"ord":0,"overloads":1,"amb":13,"cx":26,"ccx":38,"churn":36,"tested":0}, +{"p":"./src/docdrift.h","n":"DriftResult","scope":"DriftResult","key":"422ab39546b6e0bd","id":"./src/docdrift.h::DriftResult::DriftResult","id_addressable":true,"id_collides_with":0,"l":286,"ord":0,"overloads":1,"amb":0,"cx":0,"ccx":0,"churn":36,"tested":0}, +{"p":"./src/verbs_change.h","n":"runDocDrift","scope":"","key":"904831e5c0f66b39","id":null,"id_addressable":false,"id_collides_with":0,"l":1675,"ord":0,"overloads":1,"amb":1,"cx":5,"ccx":4,"churn":78,"tested":0}, +{"p":"./src/ensemble.h","n":"kEnsembleChurnSince","scope":"ensemble","key":"a34fc78c05bf56a8","id":"./src/ensemble.h::ensemble::kEnsembleChurnSince","id_addressable":true,"id_collides_with":0,"l":120,"ord":0,"overloads":1,"amb":0,"cx":0,"ccx":0,"churn":23,"tested":0}, +{"p":"./src/mcpverbs.h","n":"unknownSubVerbRefusal","scope":"rw","key":"e336d39b0a6addea","id":"./src/mcpverbs.h::rw::unknownSubVerbRefusal","id_addressable":true,"id_collides_with":0,"l":5469,"ord":0,"overloads":1,"amb":1,"cx":3,"ccx":2,"churn":359,"tested":0}, +{"p":"./src/docdrift.h","n":"kDocDriftLegend","scope":"docdrift","key":"ff2ba74637bbbebf","id":"./src/docdrift.h::docdrift::kDocDriftLegend","id_addressable":true,"id_collides_with":0,"l":2739,"ord":0,"overloads":1,"amb":0,"cx":0,"ccx":0,"churn":36,"tested":0}], +"files":[{"p":"./src/cli.h","symbols":1,"churn":469,"ccx":0,"hotspot_rank":6}, +{"p":"./src/docdrift.h","symbols":6,"churn":36,"ccx":54,"hotspot_rank":32}, +{"p":"./src/ensemble.h","symbols":1,"churn":23,"ccx":0,"hotspot_rank":93}, +{"p":"./src/mcpverbs.h","symbols":2,"churn":359,"ccx":2,"hotspot_rank":5}, +{"p":"./src/recall.h","symbols":1,"churn":27,"ccx":4,"hotspot_rank":56}, +{"p":"./src/verbs_change.h","symbols":1,"churn":78,"ccx":4,"hotspot_rank":22}], +"symbols_total":3231,"symbols_capped":true},"blast_radius":{"reaches":13,"files_total":7,"capped":false,"files":["./src/docdrift.h","./src/main.cpp","./src/mcp.h","./src/mcpserver.h","./src/mcpverbs.h","./src/recall.h","./src/verbs_for.h"]},"tests_to_run":[], +"tests_total":0,"tests_capped":false,"tests_granularity":"claimed-symbols","untested":13,"module_span":6,"notes":[], +"execution":{"policy":"codex-lane/v1","model":"gpt-5.6-sol","reasoning":"xhigh","rule":"wide-contract-work","basis":"structural-only","signals":{"claims":12,"files":6,"module_span":6,"max_ccx":38,"sum_ccx":64,"ambiguous_calls":28,"blast_reaches":13,"contract_touches":1,"conflicts":1,"untested":13,"t … [line truncated: 139 more bytes on this line] +{"id":"lane-1","task":"add the CLI parse arm and help text for the new filter","claims":{"symbols":[{"p":"./scripts/optremarks.py","n":"main","scope":"","key":"01b3b880f77d1512","id":null,"id_addressable":false,"id_collides_with":121,"l":271,"ord":0,"overloads":1,"amb":0,"cx":23,"ccx":31,"churn":10, … [line truncated: 12 more bytes on this line] +{"p":"./src/cli.h","n":"HelpLine","scope":"rw","key":"1489844dbc310b82","id":"./src/cli.h::rw::HelpLine","id_addressable":true,"id_collides_with":0,"l":2878,"ord":0,"overloads":1,"amb":0,"cx":0,"ccx":0,"churn":469,"tested":0}, +{"p":"./src/cli.h","n":"kViewFlags","scope":"rw","key":"2a2f2487082cc6d8","id":"./src/cli.h::rw::kViewFlags","id_addressable":true,"id_collides_with":0,"l":3255,"ord":0,"overloads":1,"amb":0,"cx":0,"ccx":0,"churn":469,"tested":0}, +{"p":"./src/cli.h","n":"HelpFilter","scope":"HelpFilter","key":"3103a5e8bfc33d38","id":"./src/cli.h::HelpFilter::HelpFilter","id_addressable":true,"id_collides_with":0,"l":2957,"ord":0,"overloads":1,"amb":0,"cx":0,"ccx":0,"churn":469,"tested":0}, +{"p":"./docs/docs_commands_build.py","n":"main","scope":"","key":"4015853681ded3bc","id":null,"id_addressable":false,"id_collides_with":121,"l":865,"ord":0,"overloads":1,"amb":0,"cx":19,"ccx":28,"churn":17,"tested":0}, +{"p":"./src/mcprefusal.h","n":"kMcpValueFields","scope":"rw::mcprefuse","key":"4a7106e488a2aa80","id":"./src/mcprefusal.h::rw::mcprefuse::kMcpValueFields","id_addressable":true,"id_collides_with":0,"l":304,"ord":0,"overloads":1,"amb":0,"cx":0,"ccx":0,"churn":43,"tested":0}, +{"p":"./src/cli.h","n":"serveHelpSelector","scope":"rw","key":"4b247ee064fc4e94","id":"./src/cli.h::rw::serveHelpSelector","id_addressable":true,"id_collides_with":0,"l":3022,"ord":0,"overloads":1,"amb":3,"cx":4,"ccx":3,"churn":469,"tested":0}, +{"p":"./src/query.h","n":"tryParsePredicateOnAll","scope":"query","key":"786acc4dbfec612b","id":"./src/query.h::query::tryParsePredicateOnAll","id_addressable":false,"id_collides_with":1,"l":113,"ord":0,"overloads":2,"amb":0,"cx":1,"ccx":0,"churn":20,"tested":0}, +… [61 more display lines; full output is 18818 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --plan-lanes=99 --task=x` + +*Out-of-range refusal shape for the lane count.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --plan-lanes=99 is out of range — N must be 2..16 (1 is not a fan-out) +````` + +## `./build/ripwire . --layout=Symbol` + +*CPU/GPU contract view of one struct: computed offsets/sizes/padding + mirror check.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --layout=Lang` + +*The honest refusal (exit 1): Lang is an `enum class`, not a struct — no offsets are fabricated.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --layout: 'Lang' is an enum, --layout models structs (a scoped/unscoped enum's underlying type is not a byte layout) +````` + +## `./build/ripwire . --doc-drift` + +*Which of this repo's doc claims are now false. CHANGED: row attribute at= renamed to tgt= (at= is now only the root sha stamp).* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [243 more display lines; full output is 25250 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --doc-drift --gateability` + +*The finishable to-do list: docs whose LIVE failing anchors a date-stamp would reclassify.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [269 more display lines; full output is 26449 bytes on 1 raw line(s)] +````` + +Tail of the same output — the `` section: + +````` + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --doc-drift --with-history` + +*Same report, with git history splitting stale mentions into deleted-by-commit vs never-existed.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [222 more display lines; full output is 25276 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --from-trace=-` + +*Map a pasted stack trace onto indexed symbols. CHANGED: in_corpus= now reports the real count (was 0).* + +Input file: + +````` +AddressSanitizer:DEADLYSIGNAL +================================================================= +==41337==ERROR: AddressSanitizer: SEGV on unknown address 0x000000000018 (pc 0x000102f4a1c8 bp 0x00016d2f1a40 sp 0x00016d2f19e0 T0) + #0 0x102f4a1c8 in rw::rankGraphTeleport(Graph const&, std::vector const&, float) src/graph.h:8051 + #1 0x102f3e884 in rw::rankGraph(Graph const&, float) src/graph.h:8092 + #2 0x102e11f30 in runDefaultMap(MainDispatch const&) src/main.cpp:1751 + #3 0x102e01a44 in main src/main.cpp:4169 + #4 0x1a2b3c0dc in start+0x9dc (dyld:arm64e+0x60dc) +==41337==ABORTING +````` + +````` + + + + + + + + + + + +PageRank with an explicit teleport / personalization vector p (Σp = 1). The prior is name-quality-biased through biasPrior() so all rank modes share one weighting seam; the transition matrix (edgesinline RankedGraph rankGraphTeleport( const Graph& g, const std::vector<float>&am … [line truncated: 31 more bytes on this line] + +uniform-teleport PageRank (the defaultinline RankedGraph rankGraph( const Graph& g, float alpha = 0.85f ) +int runDefaultMap( const MainDispatch& d ) +int main( int argc, char** argv ) + + + +& p, float alpha = 0.85f ) +{ + PROFILE_SCOPE_DESCRIBE( "rankGraph: PageRank (power iteration)" ); + const std::vector pw = biasPrior( g, p ); + const std::size_t N = pw.size(); + std::vector teleport( pw.begin(), pw.end() ); + std::vector rankDouble( N, 0.0 ); + PageRankRun run{}; // an N == 0 graph never enters the kernel: { 0, converged } — see PageRankRun + if( N ) + { + double teleportMass = 0.0; +… [18 more display lines; full output is 5251 bytes on 29 raw line(s)] +````` + +## `./build/ripwire . --notes` + +*List all field notes (write-side memory) — the committed .ripwire_notes at the repo root, each with the sha/branch it was recorded at.* + +````` + + + + + + + + + + + gate scripts' claim (~line 1305) is NOT enforced — the derived-vs-stated sibling loop here covers docs/EVALS.md only. It drifted 407→451 unnoticed (fixed 2026-08-23). To close: grep both files ('file:line:' parsing) in the gateCountClaims arm.]]> + + + + +````` + +## `./build/ripwire . --pack-task="add a new output format flag to the CLI"` + +*ONE budget-shared bundle: ranking + top bodies + caller sigs + notes + tests_to_run. CHANGED: rows now carry n=/id=.* + +````` + + + + +Print the authoritative CLI usage and flag catalog to the caller-provided output streaminline void printUsage( std::FILE* out ) noexcept + +claude's `mcp add … -- ripwire --mcp`: the flag goes on the end of that command line, ahead of its newlineinline std::string wrapCliAddPost( const AgentTarget& row, std::string_view toolsArg ) + +ONE recipe path for every agent that can shell out, and the point is that the RECOMMENDATION does not vary by agent — only the MCP alternative does. Before this there were four near-identical CLI-fiinline void wrapEmitCliFirst( const AgentTarget& row, const std::string& token, c … [line truncated: 83 more bytes on this line] + +mcp-tools pass-through. Written into the server command only where its argument shape is known to be one more plain argument: the JSON args arrays (cursor/windsurf/gemini), opencode's command arrayinline bool wrapWritesToolsArg( const AgentTarget& row ) noexcept + +F7: a hostile/garbled frame line number (e.g. a fuzzed or truncated trace) can exceed UINT32_MAX; unchecked `v*10+d` wraps mod 2^32 (4294967297 -> 1), which then confidently maps to a REAL line in theinline std::uint32_t toUint( std::string_view s, bool& overflowed ) noexcept +struct AgentTarget + + + + + + + + + +… [59 more display lines; full output is 10197 bytes on 51 raw line(s)] +````` + +## `./build/ripwire . --pack-task="add a new output format flag to the CLI" --partition=3` + +*Fan-out form: one shared core + 3 per-agent slices carved along call-graph communities.* + +````` + wraps a ; bundle role=core|partition i= symbols= modules= bytes= tokens=: one agent's ctx, symbols= ids assigned, bytes= its size; tokens= = est_tokens= = bytes/2.36 (flat, the densest rate; the inner ctx est_tokens= is language-weighted). Inner ctx task= root= budget_tokens= dropped_positive=: the task, p= base, the slice's token target, ranked candidates its budget cut. of_top=: rows ranked (far) / bodies (callers); s rel=caller|callee shared=: 1-hop edge, bodies reached (absent at 1). window: shown= total= capped= (capped=1 cut). run_unknown=1: no runner derivable (a guess would be worse). est_tokens=: price as emitted (an upper bound under compact). : that callee matched by name alone (receiver unproven); it does NOT mean the edge is false. : rank N in this ranking, rows in r= order. : cyclomatic/cognitive complexity. : N callers in the index (absent: not measured). route=: the ranker: name-exact(X) = the task names symbol X (anchors: its evidence), subtoken+body = conceptual BM25 (:broad = 1-2 plain words, plain rg may also win; :declined(...) = a name hit refused as a common name). sc=: enclosing scope; the full id is p::sc::n (p= of the row or its ) and selectors take it. next=: the one pasteable follow-up. pure=1: const/constexpr signature (Swift: non-mutating) and no transitive side effect found; a hint. partitions=/requested=: partitions carved / asked for. modules=/split=: call-graph groups found / cuts forced to split one. core_symbols=/surface=: ids in the shared core / core plus the assignable remainder. budget_per_agent_tokens=: one agent's budget, core_budget_tokens= plus partition_budget_tokens=. total_bytes=N: bytes of all bundles together. overlap_mean=/overlap_max=: pairwise Jaccard over the ids partitions name, before trimming. shared_symbols=/union_symbols=: ids two or more partitions name / ids any partition names. core_overlap=: share of the core surface a partition reaches anyway. --> + + + + + +Print the authoritative CLI usage and flag catalog to the caller-provided output streaminline void printUsage( std::FILE* out ) noexcept + +claude's `mcp add … -- ripwire --mcp`: the flag goes on the end of that command line, ahead of…inline std::string wrapCliAddPost( const AgentTarget& row, std::string_view toolsArg ) + +ONE recipe path for every agent that can shell out, and the point is that the RECOMMENDATION doe…inline void wrapEmitCliFirst( const AgentTarget& row, const std::string& token, const std::string_view executablePath, const WrapListing& listing ) noexcept + +mcp-tools pass-through. Written into the server command only where its argument shape is known t…inline bool wrapWritesToolsArg( const AgentTarget& row ) noexcept + + + + + +inline bool printUsageTier( std::FILE* out, HelpTier tier, std::string_view want ) noexcept + + + + + + + + +NodeId order. Renumbering an otherwise-identical graph (the exact "same probe file, different so…inline constexpr float kScoreTieAbsEps = 1e-6f + +Render one form of the disclosure. Empty string whenever there is nothing to say — no power it…inline std::string renderDisclosure( const RankDisclosure& d, DiscloseAs as ) + +The PageRank power iteration itself — the numeric kernel every ranked document's order comes f…PageRankRun pageRankDouble( const sparseCsr<float>& inEdges, std::span<const double> weightedOutDegree, std::span<const double> teleport, std::span<double> r … [line truncated: 10 more bytes on this line] + +What a ranked document discloses about the power iteration that ordered it. `isPageRank == false…struct RankDisclosure +inline std::string rankByText( const std::string& root, std::string_view mode, int topK, bool stable = false ) +struct RankedGraph +inline constexpr const char* kChurnRankLegend = "<!-- rank_by=churn: k= is PageRank re-run with the teleport BIASED by git CHANGE-FREQUENCY over window= " "(a c… +inline std::uint32_t sliceRdIterationCeiling() noexcept +struct PageRankRun +inline std::uint32_t navRelevanceWeight( const Graph& g, NodeId n ) noexcept +inline RankedGraph rankGraphTeleport( const Graph& g, const std::vector<float>& p, float alpha = 0.85f ) +inline constexpr std::string_view kPowerShellPathPrependScope = "in PowerShell, for this window +HOT_FILES = ( "src/pagerank.cpp", # the power-iteration loop — G2's no-allocation scope "src/infra/radixSort.h", # LSD radix entry points "src/infra/radixSort.inl", # the counting … [line truncated: 76 more bytes on this line] +inline std::string powerShellPathPrependHint( std::string_view programDir ) noexcept +inline std::string powerShellSingleQuote( std::string_view s ) noexcept +inline std::string path_prepend_hint( std::string_view dir ) +handle_t leftmost_leaf() const +COLD_FILES = ( # ── ingest sections that are not on the per-file / per-symbol default path ──────────────────── ( "src/ingest_astquery.h", "the --match / - … [line truncated: 48 more bytes on this line] +inline void discloseUniformChurnFallback( bool hasChurnEvidence, bool stubbed, const char* verbLabel, const std::string& windowStamp ) +### Wave-2 adversarial verification (2026-08-19) — six probes against `aa97c9e` +### Same answer, a fraction of the tokens — read this table first if your agent is on a budget +### Anchor-only auto bodies — T3 substitution round, PRE-REGISTERED 2026-08-22 (before any fix code)< … [line truncated: 3 more bytes on this line] +… [61 more display lines; full output is 10488 bytes on 10 raw line(s)] +````` + +## `./build/ripwire . --export=cc.json:/aux/ripwire2.cc.json` + +*Per-file metrics as CodeCharta cc.json.* + +**wall time: 1.71s** + +````` +(empty) +````` + +Artifact written: + +````` + 425433 /aux/ripwire2.cc.json +{"projectName":"project","apiVersion":"1.3","attributeDescriptors":{"loc":{"title":"Lines of Code","description":"Physical line count","direction":-1},"symbols":{"title":"Symbols","description":"Definitions in the file","direction":-1},"cx":{"title":"Cyclomatic Complexity","description":"Sum of per-symbol cyclomatic complexity","direction":-1},"cognitive_cx":{"title":"Cognitive Complexity","descri +````` + +## `./build/ripwire . --batch=/aux/batch2.txt` + +*One-turn sweep: 4 newline-delimited verb:arg sub-queries answered in ONE deduped .* + +**wall time: 1.21s** + +Input file: + +````` +for:incremental cache invalidation +callers:rankGraphTeleport +grep:DISCLOSE +lego:Vehicle +````` + +````` + + + + + + + + + +incremental cache (--cache): per-file content hash + raw facts so a re-run re-parses ONLY c…constexpr std::uint32_t kCacheMagic = 0x4b505443 + +Composed exactly the way every OTHER blob family is (quality.h): one fixed-width identity hex pe…inline std::string spanTierMemoPath( const std::string& diskPath ) + +Ingest the tree at `sha`, materialized out of `root`'s object store. The HEAD side reuses the SA…inline bool ingestCommitTree( const std::string& root, const std::string& sha, const std::vector<std::string>& excludes, std::size_t maxFileBytes, IngestResult&… … [line truncated: 4 more bytes on this line] + +P1-15 — the `_reingest` envelope field for a response whose handling ran an INCREMENTAL pass, …inline bool mcpRefreshedThisRequest( std::uint64_t passesAtEntry ) +inline McpRebuildBaseline mcpRebuildBaseline( const McpIndex& ix, bool isIncrementalPass ) +struct FileHealth +inline std::vector<std::vector<NodeId>> legoImplementorsOnSurface( const IngestResult& ing, const std::vector<std::vector<NodeId>>& implementors, const std::ve … [line truncated: 8 more bytes on this line] +inline const McpIndex& getIndex( const std::string& root ) +inline std::string mcpFreshFields( std::uint64_t passesAtEntry ) +struct McpIndex +inline std::pair<Snapshot, bool> computeHeadSnapshot( const std::string& root, const std::string_view* cacheNever = nullptr, std::size_t maxFileBytes = kDefault… +inline RawFacts runParsePool( IngestResult& result, const char* rootDir, std::string_view cacheFile, bool captureValueUses, HashMap<std::string, FileFacts>& cache, const CacheLoadStats& ca … [line truncated: 67 more bytes on this line] +inline std::string receiptNextFor( const std::string& fileIdentity, const std::string& symbolName, const std::string& foldJson, const std::string& firstTestRun ) +IngestResult ingest( const char* rootDir, const std::vector<std::string>& excludeSubstr, std::string_view cacheFile, std::size_t maxFileBytes, bool captureValueUses, std::string_view excludeLabel, bool … [line truncated: 37 more bytes on this line] +const char* cacheArtifactVerdict( const std::string& path, bool captureValueUses ) +static bool cachePathIsDirectory( const std::string& cachePath ) +inline constexpr std::array<std::string_view, 8> kCacheRuleNames = +inline std::string oracleCachePath( const std::string& root, const std::string& headSha ) +… [59 more display lines; full output is 42445 bytes on 1 raw line(s)] +````` + + +--- + +# self-diagnosis + +## `./build/ripwire . --doctor` + +*Environment self-check: binary staleness, grammars, cache dir, git, tracked-binary staleness — exit 1 when any check fails (here: the PATH install is older than ./build).* + +**exit code: 1** — **wall time: 3.66s** + +````` + + + + + + + + + + +````` + + +--- + +# security + +## `./build/ripwire --scan-skill=skills/ripwire-orient/SKILL.md` + +*Scan a single skill file for injection/exfiltration patterns before installing.* + +````` + + + +````` + +stderr: + +````` +ripwire scan: 0 finding(s) in skills/ripwire-orient/SKILL.md +````` + +## `./build/ripwire --scan-skills=skills` + +*Scan a whole skills directory (exit 2 = CRITICAL, 1 = WARN). Explicit-DIR form only.* + +````` + + + +````` + +stderr: + +````` +ripwire scan: 0 finding(s) total (26 skill file(s) scanned, 0 unscannable file(s) skipped, 0 denylisted subtree(s) not descended) +````` + + +--- + +# knobs / modes + +## `./build/ripwire . --rank-by=churn --top-k=5` + +*Rank by git change-frequency prior instead of PageRank.* + +**wall time: 1.39s** + +````` + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --rank-by=churn-decay --since=HEAD~7 --exclude=test --exclude=docs --exclude=skills --in=src --limit=3` + +*Scope the recent-changes answer to ONE directory. The global block stays byte-identical, a second page follows it — n=/of= are its counts (of= IS the total, so the paging half carries no total=), capped="1" has_more="1" next_offset= offset= limit= page it, and next= replays THIS run's own corpus flags (--since/--exclude) so the page it names is a page of the same answer. The symbol map collapses to a disclosed stub — the map was not asked for and was not ranked at all, which is why the header carries no pr_iters=. merge_bombs_skipped= stays on the global block: it counts the window's skipped commits, not the directory's.* + +````` + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --rank-by=bogus --top-k=5` + +*An unknown value REFUSES (exit 1), NAMED, with the supported set listed.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --rank-by: unknown value 'bogus' (supported: pagerank|authority|hub|rrf|churn|churn-decay) +````` + +## `./build/ripwire . --callers=rankGraphTeleport --format=columnar` + +*Columnar output: paths table + parallel arrays, ~15-60% fewer tokens on MANY-row lists — small results can be LARGER (the columnar legend is a fixed cost).* + +````` + + +0=src/mcpindex.h 1=src/graph.h 2=src/eval.h 3=src/main.cpp + +0,1,1,2,3,3,3 +getIndex,rankGraph,anchoredLexicalRank,runEval,churnDecayRanking,churnRankedGraph,runDefaultMap +1165,8090,8736,171,1423,1462,1749 +fn,fn,fn,fn,fn,fn,fn +0,0,0,0,0,0,0 + + +````` + +## `./build/ripwire . --for="cache invalidation" --format=candidates --top-k=5` + +*Flat top-K export for an external reranker.* + +````` + + + +inline std::string spanTierMemoPath( const std::string& diskPath ) + + +inline std::vector<std::vector<NodeId>> legoImplementorsOnSurface( const IngestResult& ing, const std::vector<std::vector<NodeId>>& implementors, const std::vector<NodeId>& surfaceIds ) + + +### See the map — not just the numbers + + +inline std::string receiptNextFor( const std::string& fileIdentity, const std::string& symbolName, const std::string& foldJson, const std::string& firstTestRun ) + + +const char* cacheArtifactVerdict( const std::string& path, bool captureValueUses ) + + +````` + +## `./build/ripwire . --callers=rankGraphTeleport --format=bogus` + +*An unknown --format value REFUSES (exit 1), named, with the supported set listed.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --format: unknown value 'bogus' (supported: xml|columnar|rows|candidates) +````` + +## `./build/ripwire . --callers=rankGraphTeleport --json` + +*Machine-parseable JSON, same content, keys mirror the XML attrs.* + +````` +{"of":"rankGraphTeleport","defs":1,"count":7,"root":".","hop_tested":0,"hop_untested":7,"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true,"callers":[{"t":"fn","n":"getIndex","p":"src/mcpindex.h:1165"}, +{"t":"fn","n":"rankGraph","p":"src/graph.h:8090"}, +{"t":"fn","n":"anchoredLexicalRank","p":"src/graph.h:8736"}, +{"t":"fn","n":"runEval","p":"src/eval.h:171"}, +{"t":"fn","n":"churnDecayRanking","p":"src/main.cpp:1423"}, +{"t":"fn","n":"churnRankedGraph","p":"src/main.cpp:1462"}, +{"t":"fn","n":"runDefaultMap","p":"src/main.cpp:1749"}]} +````` + +## `./build/ripwire . --hotspots --json` + +*JSON refusal shape: an unsupported verb refuses loudly instead of silently falling back to XML.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --json is not yet supported for --hotspots — supported: the default map, --for, --pack-task, --callers/--callees, --impact, --quality-delta, --test-gate, --metrics, and --plan-lanes which is JSON-native (e.g. ripwire

--callers=SYM --json) +````` + +## `./build/ripwire . --hotspots --limit=3 --offset=3` + +*Pagination: 3 items, skipping the first 3 (deterministic seams).* + +**wall time: 1.89s** + +````` + + + + + + + +````` + +## `./build/ripwire . --ignore-tests --top-k=5` + +*Drop test paths from the corpus before ranking.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [67 more display lines; full output is 8173 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --exclude=present --exclude=bench --top-k=5` + +*Drop matching paths (repeatable) before ranking.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [65 more display lines; full output is 8099 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --map-diff --top-k=5` + +*Full map re-ranked with teleport toward git-changed files — clean tree, so changed=0 and it degrades to the plain map.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --no-cache --top-k=3` + +*Force a cold parse (bypass the warm TMPDIR cache) — shows the cold-vs-warm cost.* + +**wall time: 2.16s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [56 more display lines; full output is 8005 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --cache=/aux/warm2.ripwirecache --top-k=3` + +*Explicit incremental cache at a path OUTSIDE the repo (first call writes it).* + +**wall time: 1.90s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [56 more display lines; full output is 8005 bytes on 1 raw line(s)] +````` + +Artifact written: + +````` + 23740310 /aux/warm2.ripwirecache +````` + +## `./build/ripwire . --max-file-size=8K --top-k=3` + +*Skip files above a size bound before parsing (note the corpus shrink in the header).* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [17 more display lines; full output is 4032 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --scip=does_not_exist.scip --callers=rankGraphTeleport` + +*SCIP overlay with a missing index REFUSES (exit 1) naming the file — never silently serves the name-based map you named a precision index to improve on (it used to degrade in silence).* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --scip=does_not_exist.scip: cannot open the index — refusing rather than serving the name-based map you named a precision index to improve on (generate one with scip-clang/scip-python, or drop --scip) +````` + +## `./build/ripwire src test --top-k=5` + +*Multi-root workspace: ONE merged graph over two roots, paths labeled /.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --eval` + +*Self-eval: co-change recall vs BM25.* + +**wall time: 16.98s** + +````` +ripwire --eval (co-change recovery, averaged over 80 historical commits) + ranker recall@5 recall@10 recall@20 + ripwire 0.2% 3.7% 8.4% + BM25 3.7% 5.7% 8.5% + BM25sub 9.4% 13.9% 25.9% + BM25body 6.7% 14.6% 26.6% + fused 7.7% 15.0% 24.6% + anchored 6.7% 14.6% 26.6% + same-dir 0.2% 0.8% 2.3% + random 0.2% 0.4% 0.7% <- floor (random ranking over F=2787 files) + note: `ripwire` here is the DEFAULT MAP's structural-only PageRank (importance, not + relatedness) — it is NOT what a --for/--query retrieval call ranks with. BM25 / + BM25sub / BM25body are QUERY-TIME lexical rankers (whole-name / subtoken / + subtoken+body); fused = RRF(ripwire, BM25sub); anchored = BM25body + anchored PPR + expansion (--for --anchor, EXPERIMENTAL). The SHIPPED default for --for/--query is + the subtoken+body lexical family (routed to name-exact only for an identifier-shaped + query — lexical.h chooseForRanker), so a gap between the ripwire and BM25* rows here + is structural-importance-vs-lexical-relatedness on a co-change task, not the shipped + retrieval path losing to an alternative it was never running. +````` + +## `./build/ripwire . --eval-retrieval` + +*Known-item retrieval eval: MRR + recall@k per ranker per query mode.* + +**wall time: 129.75s** + +````` +ripwire --eval-retrieval (known-item, 4000 doc-commented symbols; gold is in-corpus by construction) + sample: population=6641 scored=4000 rule=smallest-key CAPPED — a SUBSET, not the population + (smallest fnv1a64(scope::name) over the population, cut on the key so an identity is never split; + path- and order-independent, but a corpus this size is NOT graded exhaustively — say so when citing it) + ingest: lex=rich (persisted subtoken stats; no per-query corpus re-tokenize) + ranker query-mode MRR recall@1 recall@5 recall@10 + subtoken name 0.683 55.0% 84.6% 90.1% + subtoken doc-phrase 0.911 88.9% 93.3% 94.4% + name-exact name 0.893 81.5% 95.5% 97.0% + name-exact doc-phrase 0.013 0.4% 1.7% 2.7% + anchored name 0.689 56.6% 83.8% 88.4% + anchored doc-phrase 0.907 88.3% 93.3% 94.0% + routed name 0.895 81.8% 95.7% 97.2% + routed doc-phrase 0.910 88.9% 93.0% 94.1% + note: routing chose name-exact on 3986/4000 NAME queries (a NAME query is always identifier-shaped); + the confidence gate routes doc-phrase queries to name-exact ONLY when EVERY content word names a symbol + (or an explicit camel/snake token appears) AND every matched name is specific enough to anchor on — + a common name (many definitions, or a subtoken carried by many symbol names) declines the route — so + conceptual prose falls back to subtoken+body; routed tracks the better ranker on BOTH modes + (routed==name-exact on name, ~=subtoken+body on doc-phrase). +````` + +## `./build/ripwire . --eval-stray=/aux/stray_labels2.tsv` + +*Labelled verdict-accuracy eval for --stray-content — three labels over REAL local refs (names resolved at capture time); exit 3 when accuracy is under the floor. Read got= against v= in the stray-content run: a ref the verb could not analyse (unknown) must never be credited as a merged hit.* + +**exit code: 3** — **wall time: 5.26s** + +Input file: + +````` +# refverdict labels for --eval-stray (the first three local branches, resolved at capture time; a missing branch is padded with a nonexistent name on purpose) +arm/for-howitworks-067-f2add merged +arm/for-howitworks-067-placebo unmerged +arm/orient-map-067-narrow merged +````` + +````` + + + + + + +````` + +## `./build/ripwire skills --eval-skills=/aux/skills_labels2.tsv` + +*Labelled skill-ROUTING eval over the repo's own skills/ directory (4 hand-labelled prompts).* + +Input file: + +````` +orient in an unfamiliar codebase fast ripwire-orient judged +who calls this function and what is the blast radius ripwire-navigate judged +plan parallel worktrees so the lanes do not collide ripwire-change-check judged +what is the weather in Paris none neg +````` + +````` +ripwire --eval-skills (skill routing over K=16 candidate skills [ripwire-router excluded]; 3 positive + 1 negative prompts; corpus '/aux/skills_labels2.tsv'; split test=4 dev=0) + arm hit@1 hit@2 mrr sep-auc fire/abstain@ORACLE-th (upper bound) + overlap 66.7% 66.7% 0.689 0.000 50.0% (th=-1.000) + name 33.3% 66.7% 0.537 0.667 50.0% (th=0.000) + bm25-desc 66.7% 66.7% 0.689 0.667 75.0% (th=2.865) + bm25-full 33.3% 66.7% 0.611 1.000 50.0% (th=0.426) + for-routed 33.3% 33.3% 0.556 1.000 50.0% (th=0.669) + random 6.2% 12.5% 0.211 0.500 <- floor (uniform-random ranking; auc 0.5 by definition) + provenance hit@1 (bm25-desc): router 0/0, desc 0/0, judged 2/3 (desc rows quote the descriptions - expect them easiest; judged is the honest number) + judged-only hit@1 per arm: overlap 2/3, name 1/3, bm25-desc 2/3, bm25-full 1/3, for-routed 1/3 + router-magnet: with ripwire-router ADMITTED as a candidate it takes top-1 on 1/3 positive prompts (bm25-desc arm) - why it is excluded above + per-skill (bm25-desc): name / permitted-rows / won / pos-fires / false-fires / neg-fires + ripwire-before-you-build 0 0 1 1 0 + ripwire-change-check 1 0 0 0 0 + ripwire-find-bug 0 0 0 0 0 + ripwire-fresh-eyes 0 0 0 0 1 + ripwire-graph-query 0 0 0 0 0 + ripwire-handoff 0 0 0 0 0 + ripwire-layers 0 0 0 0 0 + ripwire-mcp 0 0 0 0 0 + ripwire-navigate 1 1 1 0 0 + ripwire-opt-remarks 0 0 0 0 0 + ripwire-orient 1 1 1 0 0 + ripwire-perf-target 0 0 0 0 0 + ripwire-quality-bar 0 0 0 0 0 + ripwire-reuse-first 0 0 0 0 0 + ripwire-security-scan 0 0 0 0 0 + ripwire-write-tests 0 0 0 0 0 + misses (overlap): + line 3 want=ripwire-change-check got=ripwire-before-you-build "plan parallel worktrees so the lanes do not collide" +… [19 more lines, 3665 bytes total] +````` + +## `./build/ripwire wrap claude` + +*Print the recipe to wire ripwire into Claude Code as an MCP server.* + +````` +# ripwire -> Claude Code (CLI-first) +# RECOMMENDED — this agent can run shell commands, so call the CLI directly. It costs +# nothing until you invoke it, and it reads CLAUDE.md, so the paste block below IS the wiring: +ripwire . --for="" +# +# ...every follow-up call answers with the compact legend by default (terse definitions of only the +# attributes the answer carries; the payload is byte-identical either way). Add --legend=full when a +# definition's reasoning is needed -- `ripwire --help` carries the measured saving (one place, +# gate-held) -- this line does not repeat it, because two copies of a number is one that goes stale: +# ripwire . --callers=SYM --legend=full +# +# ALTERNATIVE — register the MCP server instead, for a warm index across calls: +claude mcp add ripwire -- ripwire --mcp +# verbs the agent can then call mid-task (33 total): +# read: analyze, rank_by, find_symbol, find_referencing_symbols, grep, cochange, memory_recall, situational_awareness, mentions, for, lego, owners, fetch_body, batch, flags, doc_drift, slice +# flagship reflex: exemplar, quality_delta, quality_baseline, impact, uses, affected, path_between, connect, explore, from_trace, edit_check, whereis, stray_content +# edit: replace_symbol_body, insert_before_symbol, insert_after_symbol +bash skills/install.sh # deploy to ${CLAUDE_CONFIG_DIR:-~/.claude}/skills (drift-gated) +bash skills/install.sh --hook # RECOMMENDED: advisory Read/Grep -> ripwire CLI nudge + session primer (opt-in, never blocks) +# +# context wiring — a binary on PATH is invisible to an agent until its rules file says when +# to reach for it. Paste the block below into CLAUDE.md: +# --- paste into CLAUDE.md --- +## ripwire — deterministic codebase maps (on PATH as `ripwire`) +Reach for it BEFORE blind grep + whole-file reads. First call ~1s cold; after that warm, ~0.1s. +- Orient on a task: `ripwire --for=""` — ranked, quality-annotated + signatures. Paste symbol/file names from the issue verbatim; named mentions get anchored. +- One task: `--pack-task="" --legend=compact`; before parallel agents: `--plan-lanes=N --task=""`, then read `lanes[].execution`. +- Have a stack trace / build error: `ripwire --from-trace=FILE --legend=compact` (`-` = stdin) — + paste the error, don't paraphrase it into a query. +… [14 more lines, 4058 bytes total] +````` + +## `./build/ripwire --version` + +*Version + short build info.* + +````` +ripwire 0.6.5 (dev, AppleClang 21.0.0.21000101, emit=std::print, built_from=65d62fe5c) +````` + + +--- + +# navigate — seeds, claims, slices, shapes + +## `./build/ripwire . --at=src/graph.h:8051` + +*Hold a LOCATION, not a name: the enclosing-definition chain at FILE:LINE (a compiler error, a diff hunk, a stack frame), outermost -> innermost.* + +````` + + + + +````` + +## `./build/ripwire . --callers=@src/graph.h:8051` + +*The same seed in a SELECTOR position: @FILE:LINE resolves to the innermost enclosing definition, then --callers runs on it.* + +````` + + + + + + + + + + +````` + +## `./build/ripwire . --at=src/graph.h:999999` + +*A seed past the end of the file — the refusal shape for a faulted location.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: the at flag's seed 'src/graph.h:999999' named no location (./src/graph.h has only 11947 lines — the seed asked for line 999999) +````` + +## `./build/ripwire . --verify="calls(runDefaultMap, rankGraphTeleport)"` + +*VERIFY a closed claim in one call: three-valued verdict (confirmed / refuted / not-established) with the evidence rows inline.* + +````` + + + + + +````` + +## `./build/ripwire . --verify="unused(rankGraphTeleport)"` + +*A claim that is FALSE — the refuted shape, with the references that refute it.* + +````` + + + + + + + + + + + + +````` + +## `./build/ripwire . --verify="contains(src/graph.h, \"no such literal anywhere\")"` + +*A literal-scan absence: refuted only with complete= evidence, never on a partial scan.* + +````` + + + +````` + +## `./build/ripwire . --verify="frobnicate(x)"` + +*An unparseable claim — the refusal names the accepted shapes.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --verify claim not recognized: 'frobnicate(x)' — unknown shape word. The claim language is CLOSED; the shapes are: calls(A, B) does A transitively call B · uses(SYM) is SYM referenced anywhere · unused(SYM) is SYM referenced nowhere · contains(FILE, "LITERAL") do FILE's bytes contain t … [line truncated: 218 more bytes on this line] +````` + +## `./build/ripwire . --slice=rankGraphTeleport` + +*Bare --slice=SYM: the INVENTORY of sliceable locals (), so a caller can pick VAR.* + +````` + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --slice=rankGraphTeleport:teleport` + +*Intra-procedural def-use slice of ONE variable: one row per line touching it, k=def|use|both, reaching definitions flow-sensitive (reach=cfg).* + +````` + + + + + + + teleport( pw.begin(), pw.end() );]]> + + + + + + + + +````` + +## `./build/ripwire . --slice=rankGraphTeleport:teleport --slice-flow=back --slice-depth=3` + +*TRANSITIVE backward value-flow from the seed variable, bounded BFS (depth= disclosed; a cut frontier says flow_truncated=1).* + +````` + + + + + + + teleport( pw.begin(), pw.end() );]]> + + + + + + + + + pw = biasPrior( g, p );]]> + + +& p, float alpha = 0.85f )]]> + + +& p, float alpha = 0.85f )]]> + + +````` + +## `./build/ripwire . --slice=rankGraphTeleport:teleport --slice-flow=fwd` + +*Forward flow: which statements the seed's value reaches, at the default depth bound.* + +````` + + + + + + + teleport( pw.begin(), pw.end() );]]> + + + + + + + + + + + + + + + + + + + + + 0.0 )]]> + + + + + + + + +````` + +## `./build/ripwire . --slice=rankGraphTeleport:nosuchvar` + +*A variable the definition does not bind — the refusal shape, naming the inventory.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --slice: no occurrence of 'nosuchvar' in rankGraphTeleport — sliceable locals: alpha, g, p, pw, N, teleport, rankDouble, run, teleportMass, value, inverseMass, r (bare --slice=rankGraphTeleport lists them with first-def lines) +````` + +## `./build/ripwire . --slice-depth=3` + +*--slice-depth without --slice-flow is refused loudly rather than silently ignored.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --slice-depth bounds the --slice-flow BFS — pass both (e.g. ripwire --slice=parseArgs:argIndex --slice-flow=fwd --slice-depth=4) +````` + +## `./build/ripwire . --slice=rankGraphTeleport:teleport --legend=compact` + +*The compact legend posture: rows byte-identical, a versioned schema id replaces the repeated explanatory prose — for a many-small-calls loop.* + +````` + + + + + + + teleport( pw.begin(), pw.end() );]]> + + + + + + + + +````` + +## `./build/ripwire . --pattern='rankGraphTeleport($A, $B, $C)'` + +*Structural search written in CODE: $NAME binds one node; grammars=/shapes= disclose what the pattern became per grammar (a 3-argument call shape — the 2-argument spelling has no call site in this repo and correctly reports hits=0).* + +**wall time: 1.03s** + +````` + +rankGraphTeleport( g, std::vector<float>( N, N ? 1.0f / float( N ) : 0.f ), alpha ) + +````` + +## `./build/ripwire . --pattern='DISCLOSE(...)'` + +*The ellipsis form over a macro-shaped call site; unsupported= names the families this verb does not serve.* + +**wall time: 1.01s** + +````` + +DISCLOSE( result, AbiResult::DisclosureWhy::NoMergeBase, "abi: no merge-base for a ref (unrelated history?) — that ref +DISCLOSE( Diagnostics::answerRefused, "--arch exits 1 with path:line and the reason on stderr; no report is printed", +DISCLOSE( baseline, ArchBaselineRead::DisclosureWhy::SymlinkRefused, "arch: refusing to read the arch baseline sidecar t +DISCLOSE( Diagnostics::answerRefused, "--baseline exits 1: pathguard names the refused link on stderr and the verb says +DISCLOSE( ex, Exclusions::DisclosureWhy::ForHeaderSaturated, "atoms: the for-header exclusion stream spent its whole bud +DISCLOSE( ex, Exclusions::DisclosureWhy::StmtCrementSaturated, "atoms: the statement-crement exclusion stream spent its +DISCLOSE( ex, Exclusions::DisclosureWhy::StmtAssignSaturated, "atoms: the statement-assign exclusion stream spent its wh +DISCLOSE( ex, Exclusions::DisclosureWhy::InnerCommaSaturated, "atoms: the inner-comma exclusion stream spent its whole b +DISCLOSE( st, Type3Stats::DisclosureWhy::PairCapHit, "clones: Type-3 pair cap hit — first N compared (both-gate-surviv +DISCLOSE( scan, CommentCoherenceScan::DisclosureWhy::UnreadableFile, "comment-coherence: an indexed file could not be re +DISCLOSE( st, StreamBlobStats::DisclosureWhy::ListUnwritable, "crossref: cannot write the blob-batch list — cross-bran +DISCLOSE( st, StreamBlobStats::DisclosureWhy::BatchNotStarted, "crossref: git cat-file --batch failed to start — cross +DISCLOSE( st, StreamBlobStats::DisclosureWhy::StreamEndedMidBlob, "crossref: git cat-file stream e +DISCLOSE( dropSink, RefEnumeration::DisclosureWhy::TipNotObjectName, "crossref: for-each-ref yielded a ref whose tip is +DISCLOSE( "crossref: refusing a diff whose revision arguments are not resolved object names" ) +DISCLOSE( sweep, ParallelSweep::DisclosureWhy::WorkerThrew, "crossref: a git worker threw — this shard of the sweep is +DISCLOSE( stats, StreamBlobStats::DisclosureWhy::BatchNotStarted, "crossref: a blob-stream slice was never read — comp +DISCLOSE( plumb, RefPlumbing::DisclosureWhy::RevisionNotObjectName, "crossref: ref tip or HEAD is not a resolved object +DISCLOSE( plumb, RefPlumbing::DisclosureWhy::MergeBaseNotObjectName, "crossref: merge-base returned something that is no +DISCLOSE( plumb, RefPlumbing::DisclosureWhy::NoMergeBase, "crossref: no merge-base for ref (shallow clone or unrelated h +DISCLOSE( "crossref: refusing to list a tree whose revision argument is not a resolved object name" ) +DISCLOSE( "whereis: the index's def sites match no HEAD row (working tree drifted from HEAD?) — keeping the lexical la +DISCLOSE( scan, WorktreeScan::DisclosureWhy::ListingFailed, "whereis: git could not list the working tree's changes — +DISCLOSE( scan, WorktreeScan::DisclosureWhy::OverPathCap, "whereis: more changed paths than the overlay reads — the re +DISCLOSE( scan, WorktreeScan::DisclosureWhy::CopyUnread, "whereis: a changed path could not be read from the working tre +DISCLOSE( Diagnostics::answerUnchanged, "lsTree lists the same rows per ref: only slower", "crossref: +DISCLOSE( out, CMakeScan::DisclosureWhy::SymlinkEscapesRoot, "flags: a CMake file's symlink target leaves the root — f +DISCLOSE( out, CMakeScan::DisclosureWhy::RootWalkFailed, "flags: cannot walk root for CMake files — cmake gates omitte +… [73 more display lines; full output is 17835 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --pattern='x'` + +*A pattern that collapses to a bare token is REFUSED — never reported as hits=0.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --pattern: this pattern collapses to a single bare token, which is a TEXT search, not a structural one — a clean parse is not enough (a bare word parses fine in most grammars and means nothing structural). Give it a shape (foo($X), $A + $B, if ($C) { ... }) or use the grep flag for litera … [line truncated: 176 more bytes on this line] +````` + +## `./build/ripwire . --grep=DISCLOSE --and=cache` + +*Boolean grep: hits where BOTH literals share the matched line (--grep-scope=line is the default).* + +````` + + + +inline bool topLevelEvidence( std::string_view source, const TSLanguage* language, const Predicate& predicate ) +The crawl's per-file byte ceiling. A text file larger than this is skipped: at this size it is o…constexpr std::size_t kDefaultMaxFileBytes = 4u * 1024u * 1024u +TSTree* parseTree( TSParser* parser, std::string_view src ) + +Exercise every registered grammar and its embedded query, reporting loaded and expected totalsinline DoctorGrammarProbe doctorProbeGrammars() +struct FileHealth +enum class AstWalk : std::uint8_t +inline RevSide sliceAtRev( const std::string& root, const std::string& sha, const std::string& rel, const Symbol& sym, SliceFam fam, const ::TSLanguage* grammar…constexpr std::array<LangEntry, 53> kLangTable = +struct SliceScan +inline bool astroFenceTailIsBlank( std::string_view src, std::size_t from, std::size_t to ) noexcept +constexpr std::uint32_t kMaxKotlinStringNestDepth = 128u +std::vector<LocalNameFact> collectGatedLocalNames( std::string_view defBytes, std::uint32_t defStartLine, Lang lang ) +inline bool hasPhantomScopeSeparator( TSNode qualified ) noexcept +inline ModuleScan scanModule( std::string_view source, std::string_view path ) +bool jsonNestsTooDeep( std::string_view bytes ) noexcept +constexpr std::uint32_t kMaxYamlNestDepth = 64u +inline bool hasMainGuard( std::string_view source ) +SpanTierBatch spanTiersOfFiles( std::span<const std::string> diskPaths, bool useMemo ) +inline void runParseWorker( ParsePoolShared& sh, unsigned t ) +inline SliceScan sliceScanDefinition( const std::string& src, const Symbol& sym, SliceFam fam, const ::TSLanguage* grammar, std::string_view varName ) +struct LintRule +enum class DisclosureWhy : std::uint8_t +inline std::string buildMemoryStopAttr( const IngestResult& ing, bool json ) +inline bool hasPytestTable( std::string_view source ) +… [55 more display lines; full output is 9884 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --for="tree-sitter parse of a source file" --sections=lego,compose` + +*Restore the / sections the --for bundle collapses to a counted stub by default (this is the stub's own next= spelling).* + +````` + + + + +inline bool topLevelEvidence( std::string_view source, const TSLanguage* language, const Predicate& predicate ) +The crawl's per-file byte ceiling. A text file larger than this is skipped: at this size it is o…constexpr std::size_t kDefaultMaxFileBytes = 4u * 1024u * 1024u +TSTree* parseTree( TSParser* parser, std::string_view src ) + +Exercise every registered grammar and its embedded query, reporting loaded and expected totalsinline DoctorGrammarProbe doctorProbeGrammars() +struct FileHealth +enum class AstWalk : std::uint8_t +inline RevSide sliceAtRev( const std::string& root, const std::string& sha, const std::string& rel, const Symbol& sym, SliceFam fam, const ::TSLanguage* grammar…constexpr std::array<LangEntry, 53> kLangTable = +struct SliceScan +inline bool astroFenceTailIsBlank( std::string_view src, std::size_t from, std::size_t to ) noexcept +constexpr std::uint32_t kMaxKotlinStringNestDepth = 128u +std::vector<LocalNameFact> collectGatedLocalNames( std::string_view defBytes, std::uint32_t defStartLine, Lang lang ) +inline bool hasPhantomScopeSeparator( TSNode qualified ) noexcept +inline ModuleScan scanModule( std::string_view source, std::string_view path ) +bool jsonNestsTooDeep( std::string_view bytes ) noexcept +constexpr std::uint32_t kMaxYamlNestDepth = 64u +inline bool hasMainGuard( std::string_view source ) +SpanTierBatch spanTiersOfFiles( std::span<const std::string> diskPaths, bool useMemo ) +inline void runParseWorker( ParsePoolShared& sh, unsigned t ) +inline SliceScan sliceScanDefinition( const std::string& src, const Symbol& sym, SliceFam fam, const ::TSLanguage* grammar, std::string_view varName ) +struct LintRule +enum class DisclosureWhy : std::uint8_t +inline std::string buildMemoryStopAttr( const IngestResult& ing, bool json ) +inline bool hasPytestTable( std::string_view source ) +… [66 more display lines; full output is 10217 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --for="tree-sitter parse of a source file" --auto-bodies` + +*Opt OUT of compact conceptual serving: restore the rank-first auto walk (bundle="auto").* + +````` + + + + +inline bool topLevelEvidence( std::string_view source, const TSLanguage* language, const Predicate& predicate ) +The crawl's per-file byte ceiling. A text file larger than this is skipped: at this size it is o…constexpr std::size_t kDefaultMaxFileBytes = 4u * 1024u * 1024u +TSTree* parseTree( TSParser* parser, std::string_view src ) + +Exercise every registered grammar and its embedded query, reporting loaded and expected totalsinline DoctorGrammarProbe doctorProbeGrammars() +struct FileHealth +enum class AstWalk : std::uint8_t +inline RevSide sliceAtRev( const std::string& root, const std::string& sha, const std::string& rel, const Symbol& sym, SliceFam fam, const ::TSLanguage* grammar…constexpr std::array<LangEntry, 53> kLangTable = +struct SliceScan +inline bool astroFenceTailIsBlank( std::string_view src, std::size_t from, std::size_t to ) noexcept +constexpr std::uint32_t kMaxKotlinStringNestDepth = 128u +std::vector<LocalNameFact> collectGatedLocalNames( std::string_view defBytes, std::uint32_t defStartLine, Lang lang ) +inline bool hasPhantomScopeSeparator( TSNode qualified ) noexcept +inline ModuleScan scanModule( std::string_view source, std::string_view path ) +bool jsonNestsTooDeep( std::string_view bytes ) noexcept +constexpr std::uint32_t kMaxYamlNestDepth = 64u +inline bool hasMainGuard( std::string_view source ) +SpanTierBatch spanTiersOfFiles( std::span<const std::string> diskPaths, bool useMemo ) +inline void runParseWorker( ParsePoolShared& sh, unsigned t ) +inline SliceScan sliceScanDefinition( const std::string& src, const Symbol& sym, SliceFam fam, const ::TSLanguage* grammar, std::string_view varName ) +struct LintRule +enum class DisclosureWhy : std::uint8_t +inline std::string buildMemoryStopAttr( const IngestResult& ing, bool json ) +inline bool hasPytestTable( std::string_view source ) +… [93 more display lines; full output is 13498 bytes on 55 raw line(s)] +````` + +## `./build/ripwire . --for="quality delta acks ledger rubber stamp"` + +*Doc-mention surfacing (default ON): a markdown doc naming a top-resolved symbol in a backtick rides in below that symbol — the legend's [doc mentions: …] clause says it fired.* + +````` + + + + +inline QualityDeltaOutcome computeQualityDelta( const std::string& root ) +std::optional<int> refuseForeignAckSelection( const rw::Config& cfg, const rw::quality::Scope& scope, const std::vector<rw::quality::Regression>& outOfSc … [line truncated: 14 more bytes on this line] + +runQualityViews was NOT a dispatch chain — it held two branches, one of which was 298 lines. T…std::optional<int> runQualityDelta( const MainDispatch& d ) + +H10 (capture-audit 2026-09-04): the --quality-ack run that has NOTHING to accept. It used to re-…int ackNothingToAccept( const std::string& acksFile, const gtl::btree_map<std::string, rw::quality::AckRecord>& acks, const rw::quality::Scope& scope, std::size… +struct DeltaBasis +inline const char* kAcksFile = ".ripwire_quality_acks" +DirtyPinVerdict inspectDirtyBaselinePin( const MainDispatch& d, const std::string& acksFile ) +struct IdentityHealing +inline std::pair<std::string, std::string> qualityDeltaJson( const std::string& root ) +inline bool writeAckRecords( const std::string& path, const gtl::btree_map<std::string, AckRecord>& acks ) +std::optional<int> resolveDeltaBasis( const MainDispatch& d, const std::string& baselineFile, RefPairDelta& refs, DeltaBasis& out ) +inline std::string staleAcksXml( const std::vector<StaleAck>& staleAcks, EscapeFn esc ) +struct QualityDeltaOutcome +inline std::string qualityAcksPath( const std::string& root ) +inline bool atomicWriteFile( const std::string& path, const std::string& blob ) +inline constexpr const char* kAtStampLegend = "<!-- at= is the git commit these numbers were computed at +inline std::string gitHeadSha( const std::string& root ) +inline gtl::btree_map<std::string, AckRecord> readAckRecords( const std::string& path, std::size_t& badLines ) +inline std::string renderAckRecords( const gtl::btree_map<std::string, AckRecord>& acks ) +inline rw::pathguard::NoFollowRead openRegularFileStream( std::string_view what, const std::string& path ) +int runQualityBaselinePin( const MainDispatch& d, const std::string& baselineFile, const std::string& acksFile ) +inline Result computeDmm( const std::string& root, std::string_view spec, const IngestResult& workingIng, const std::vector<std::string>& excludes, std::size_t maxFileBytes … [line truncated: 6 more bytes on this line] +inline std::string qualityBaselinePath( const std::string& root ) +inline constexpr const char* kQdSchemeLegend = "A THIRD re-filing, git-independent: on 2026-08-25 the per-symbol quality key stopped being a " "canonical-id hash (which degraded to a BA … [line truncated: 62 more bytes on this line] +… [52 more display lines; full output is 9778 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --for="quality delta acks ledger rubber stamp" --no-doc-mention` + +*The same task with doc-mention surfacing OFF — the contrast the flag exists for (no [doc mentions] clause, one fewer row).* + +````` + + + + +inline QualityDeltaOutcome computeQualityDelta( const std::string& root ) +std::optional<int> refuseForeignAckSelection( const rw::Config& cfg, const rw::quality::Scope& scope, const std::vector<rw::quality::Regression>& outOfSc … [line truncated: 14 more bytes on this line] + +runQualityViews was NOT a dispatch chain — it held two branches, one of which was 298 lines. T…std::optional<int> runQualityDelta( const MainDispatch& d ) + +H10 (capture-audit 2026-09-04): the --quality-ack run that has NOTHING to accept. It used to re-…int ackNothingToAccept( const std::string& acksFile, const gtl::btree_map<std::string, rw::quality::AckRecord>& acks, const rw::quality::Scope& scope, std::size… +struct DeltaBasis +inline const char* kAcksFile = ".ripwire_quality_acks" +DirtyPinVerdict inspectDirtyBaselinePin( const MainDispatch& d, const std::string& acksFile ) +struct IdentityHealing +inline std::pair<std::string, std::string> qualityDeltaJson( const std::string& root ) +inline bool writeAckRecords( const std::string& path, const gtl::btree_map<std::string, AckRecord>& acks ) +std::optional<int> resolveDeltaBasis( const MainDispatch& d, const std::string& baselineFile, RefPairDelta& refs, DeltaBasis& out ) +inline std::string staleAcksXml( const std::vector<StaleAck>& staleAcks, EscapeFn esc ) +struct QualityDeltaOutcome +inline std::string qualityAcksPath( const std::string& root ) +inline bool atomicWriteFile( const std::string& path, const std::string& blob ) +inline constexpr const char* kAtStampLegend = "<!-- at= is the git commit these numbers were computed at +inline std::string gitHeadSha( const std::string& root ) +inline gtl::btree_map<std::string, AckRecord> readAckRecords( const std::string& path, std::size_t& badLines ) +inline std::string renderAckRecords( const gtl::btree_map<std::string, AckRecord>& acks ) +inline rw::pathguard::NoFollowRead openRegularFileStream( std::string_view what, const std::string& path ) +int runQualityBaselinePin( const MainDispatch& d, const std::string& baselineFile, const std::string& acksFile ) +inline Result computeDmm( const std::string& root, std::string_view spec, const IngestResult& workingIng, const std::vector<std::string>& excludes, std::size_t maxFileBytes … [line truncated: 6 more bytes on this line] +inline std::string qualityBaselinePath( const std::string& root ) +inline constexpr const char* kQdSchemeLegend = "A THIRD re-filing, git-independent: on 2026-08-25 the per-symbol quality key stopped being a " "canonical-id hash (which degraded to a BA … [line truncated: 62 more bytes on this line] +… [52 more display lines; full output is 9733 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --safe-delete=rankGraphTeleport` + +*"Can I delete this?" — callers + transitive impact + every use site + how much of the radius is tested, composed in ONE call; risk= names what was found, never a verdict.* + +````` + + + + + + + + + +````` + +## `./build/ripwire . --safe-delete=DoesNotExist` + +*Unknown-symbol refusal shape for --safe-delete.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --safe-delete symbol not found: DoesNotExist +````` + +## `./build/ripwire . --handoff` + +*The continuation packet for the NEXT session: disk truth (branch/sha, changed symbols, blast radius, tests) + labeled suggestions. Recorded against a CLEAN tree.* + +**wall time: 1.70s** + +````` + + + + + + + + + + + + + +````` + +## `./build/ripwire . --handoff --token-budget=1200` + +*The same packet under a hard ceiling: heuristic rows drop tail-first (withheld= disclosed), verified rows never drop.* + +**wall time: 1.70s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [417 more display lines; full output is 44640 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --no-ignore --top-k=3` + +*Crawl paths the repo's own .gitignore covers (default honours it and discloses ignored_files=/ignored_dirs= only when it dropped anything — this repo's crawl drops nothing, so the header is identical to the default map's; --skipped's ignore_mode= says which rule applied).* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [56 more display lines; full output is 8005 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --no-stable --top-k=3` + +*--no-stable outside --mcp: what the flag does (or says) when there is no stable-by-default ordering to opt out of.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [56 more display lines; full output is 8005 bytes on 1 raw line(s)] +````` + +stderr: + +````` +ripwire: --no-stable is read only by --mcp/--listen (it opts out of the stable ordering the server turns on) — it changed nothing here; the CLI map orders important-first unless you pass --order=stable +````` + +## `./build/ripwire . --run-trace="cat /aux/asan_trace_now.txt; exit 1"` + +*EXEC-MODE --from-trace: run a command, and on a non-zero exit map its captured output onto indexed symbols in the same call — the whole fix-loop entry. ripwire exits 4 here because the wrapped command failed, which is the signal, not an incident.* + +**exit code: 4** + +````` + + + + + + const&, float) src/graph.h:8051 + #1 0x102f3e884 in rw::rankGraph(Graph const&, float) src/graph.h:8092 + #2 0x102e11f30 in runDefaultMap(MainDispatch const&) src/main.cpp:1751 + #3 0x102e01a44 in main src/main.cpp:4169 + #4 0x1a2b3c0dc in start+0x9dc (dyld:arm64e+0x60dc)]]> pw = biasPrior( g, p ); + const std::size_t N = pw.size(); + std::vector teleport( pw.begin(), pw.end() ); + std::vector rankDouble( N, 0.0 ); + PageRankRun run{}; // an N == 0 graph never enters the kernel: { 0, converged } — see PageRankRun + if( N ) + { + double teleportMass = 0.0; + for( const double value : teleport ) + { + teleportMass += value; + } + if( teleportMass > 0.0 ) + { + const double inverseMass = 1.0 / teleportMass; + for( double& value : teleport ) + { + value *= inverseMass; + } + } + run = pageRankDouble( g.inEdges, g.wOutDeg, teleport, rankDouble, PageRankConfig{ .alpha = double( alpha ) } ); + } + std::vector r( N, 0.f ); + std::transform( rankDouble.begin(), rankDouble.end(), r.begin(), []( double value ) { return float( value ); } ); + return { std::move( r ), run.iterationCount, run.hasConverged }; +}]]>inline std::vector<float> biasPrior( const Graph& g, const std::vector<float>& p )#define PROFILE_SCOPE_DESCRIBE( desc )#define PROFILE_SCOPE_DESCR … [line truncated: 524 more bytes on this line] +````` + +## `./build/ripwire . --run-trace="true"` + +*A command that exits 0: a minimal success record (exit, measured duration, disclosed output tail) and NO bundle — nothing failed, nothing to map.* + +````` + + + + +````` + +## `./build/ripwire . --run-trace="sleep 30" --run-timeout=2` + +*A command still running at the cap: its process group is killed and the run reports timed_out=1 — an honest timeout, never an empty success.* + +**exit code: 4** — **wall time: 2.41s** + +````` + + + + +````` + +stderr: + +````` +ripwire: --run-trace: TIMEOUT — the command exceeded the 2 s cap; its process group was killed +````` + +## `./build/ripwire . --run-timeout=5` + +*--run-timeout alone is refused loudly (it only modifies --run-trace).* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --run-timeout=SECONDS modifies --run-trace — pass it too (e.g. ripwire --run-trace="make -j" --run-timeout=60) +````` + + +--- + +# assess quality — the wider lens family + +## `./build/ripwire . --quality-panel` + +*THE single wide-angle quality read: six families in one pass, an eligible/ranked shortlist rather than a firehose.* + +**wall time: 4.03s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [185 more display lines; full output is 15572 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --biggest-first --limit=8` + +*Per-function size ranking, largest Halstead volume/token entropy/lines first — a size proxy, not a readability order (withdrawn, docs/EVALS.md §8) — a RANKING lens, not a grade.* + +````` + + + + + + + + + + + +````` + +## `./build/ripwire . --comment-coherence --limit=8` + +*Functions WITH a doc comment, most name-restating first: c_coeff (high = the comment repeats the name) and cic (Jaccard of comment vs identifier vocabulary), both reported, never collapsed.* + +````` + + + + + + + + + + + +````` + +## `./build/ripwire . --context-ratio --limit=8` + +*The local-reasoning lens: to understand this symbol, how much must you know that is NOT in front of you (ent_ratio= edge share, read_ratio= token-weighted).* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [21 more display lines; full output is 11182 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --nonlocal-state --limit=8` + +*Per function, the non-local MUTABLE state it can reach (transitively), most writes first — unsound by construction, and the legend says where.* + +**wall time: 2.15s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [41 more display lines; full output is 7594 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --ensemble --limit=8` + +*The family join: per function, which of four orthogonal evidence families fire, ranked by how many agree.* + +**wall time: 2.09s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [35 more display lines; full output is 7393 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --field-affinity` + +*The cache-locality lens over every aggregate: fields READ TOGETHER but declared FAR APART (split-line / straddle findings, Chilimbi separation weight) — advice only, never a rewrite.* + +**wall time: 1.07s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [510 more display lines; full output is 45864 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --field-affinity=Symbol` + +*The same lens narrowed to ONE struct — the one --layout=Symbol shows the offsets for.* + +**wall time: 1.04s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --naming-consistency --limit=8` + +*The corpus's OWN case-convention vote per (language, kind) group; off-convention names get a mechanical propose= (a suggestion, never a blind rename).* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [13 more display lines; full output is 4349 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --naming-calibration` + +*Score the naming-* rules against this repo's own rename history: proxy=old/(old+new) per rule, 0.50 = chance; read pairs= (sample size) first.* + +**wall time: 8.91s** + +````` + + + + + + + + + + + +

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+

+… [135 more display lines; full output is 14384 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --lint --naming-locals` + +*The opt-in --lint modifier: naming predicates over LOCAL variable names too, C/C++ only, only inside functions already past a size/complexity gate.* + +**wall time: 1.99s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [643 more display lines; full output is 72478 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --lint-catalog` + +*The built-in rule registry — one row per rule with sev=/category=/rationale/lang=/since=; no corpus needed.* + +````` + + +a C-style cast — cppcoreguidelines-pro-type-cstyle-cast prefers the explicit static_cast/const_cast/reinterpret_cast +a goto statement — cppcoreguidelines-avoid-goto +a do/while loop shape (on Swift, its do/catch block shares the same grammar node and also matches) +a call to an unbounded C string function (strcpy/strcat/sprintf/gets) +a call to a broken hash or cipher (MD5/SHA1/MD4/RC4) +a doubly-parenthesized expression — readability-redundant-parentheses +an if-body that is just `;` — bugprone-suspicious-semicolon +a C-style typedef struct/union where `using` is preferred +a non-trivial numeric literal inside a function body, outside a const/constexpr init +a catch block with an empty body +x = x — almost always a copy-paste bug +a function body over 80 lines +brace nesting depth over 4 inside a function body +a bare `return;` mixed with a value-returning return in the same function +a statement after an unconditional return/break/continue/throw/raise in the same block +a 1-2 letter Function/Method/Var name — visible far beyond any tiny scope [Beniamini/Hofmeister] +more than 5 split tokens in one name [Butler; AlSuhaibani] +foo1/foo2/... digit-suffix siblings sharing a base name in one scope [Butler] +internal consecutive underscores, or a C-family reserved __x/_X form [Butler] +snake_case and camelCase mixed inside one name [Butler] +an is/has/can/should/was-prefixed name whose KNOWN return type is not bool-like [LAPD A2] +a set-prefixed name whose KNOWN return type is not void-like [LAPD A3] +a co-visible pair within edit distance <=2, reordered tokens, or a bare/digit-suffixed twin [Namesake] +every split subtoken is corpus-ubiquitous (BM25 idf) on a body past the size floor — fires only at the low end [Sparck Jones 1972] +the comma operator inside an expression (never a for-header comma) [Gopstein FSE 2017] +++/-- evaluated inside a larger expression (never a whole statement) [Gopstein FSE 2017] +an assignment whose VALUE is consumed (a condition, an argument, ...) [Gopstein FSE 2017] +a conditional expression inside a conditional expression [Gopstein FSE 2017] +… [12 more display lines; full output is 7764 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --legend-dict` + +*The session legend dictionary the MCP server serves as ripwire://legend-dict/full — one definition per line, headed by its dictv= version; no corpus needed. =roster lists the completeness attributes it defines.* + +````` +ripwire legend dictionary ripwire.dict/v1 dictv=74f30d013d02c2a2 entries=829 +: the answer's rows come first; its root keeps only task= changed= from= to=, and this LAST child carries every other root attribute unchanged (schema= included); legend="ref": a definition is sent once per session (this dictionary's core, or the first answer that ne … [line truncated: 83 more bytes on this line] +schema=ripwire.KEY/v1: the line ripwire.KEY/v1 below reads the answer's rows +window: shown= total= capped= has_more= next_offset= offset= limit= page a list (capped=1 cut; next_offset= pastes as offset=) +X_capped= (any attribute ending _capped): 1 = cut +under ref, est_tokens= and over_ceiling= price the answer with its inline legend: an upper bound +under ref, a group of n <= 8 runner-less rows prints as its n single rows, each run_unknown=1 +ripwire.map/v1 : ranked symbol map: groups rows (k= rank), resolved callees; the header comment is data +ripwire.map-diff/v1 : the ranked map anchored at at=: what the diff touched, the map's row vocabulary +ripwire.metrics/v1 : the ranked map with per-symbol metrics: in/out, cx/ccx, loc, params, nest, humps/deep, locals, cbo, amp, tested, ev +ripwire.around/v1 : call neighbourhood of of=: depth= hops, fanout= kept per hop; absent rows lie outside that boundary +ripwire.query/v1 : lexical-rank map for the query term, the map's row vocabulary +ripwire.pack-signatures/v1 : the ranked map plus signature rows +ripwire.pack-top-n/v1 : the ranked map plus bodies of the top-N symbols +ripwire.skipped/v1 : why the index lacks a file ; indexed but unvouched ; census +ripwire.notes/v1 : field notes by target: holds ; the kept count comment: notes= rows, targets= rows, dangling= targets matching nothing indexed (listed, surfaced nowhere) +ripwire.lego/v1 : ONE interface/base type: , its method contract, every implementor +ripwire.expand/v1 : full bodies: of ; resolved callees +ripwire.expand-file/v1 : the file's own text: ; per scoped symbol; full id = p::sc::n +ripwire.candidate-page/v1 : budgeted-bundle candidate page: a resumable window over the ranked candidate set the bundle cuts, for task= under budget_tokens=, one tier per page, ranks global across pages +ripwire.pack-task/v1 : one-call task bundle for task= under budget_tokens=: ranking, ranked but over 1 hop out (of_top= ranked rows) > with callees > 1-hop from the bodies (o … [line truncated: 105 more bytes on this line] +ripwire.from-trace/v1 : trace frames mapped to indexed symbols, innermost first; the innermost in-corpus body included +ripwire.exemplar/v1 : the best-in-class instance of kind= for the task, chosen by role: , to imitate +ripwire.pack-task/v1 : N minimally overlapping agent bundles carved along call-graph communities plus one shared core; each wraps a ; bundle role=core|partition i= symbols= modules= bytes= tokens=: one agent's ctx, symbols= ids assigned, bytes= its size; tokens= = est_t … [line truncated: 345 more bytes on this line] +ripwire.callers/v1 : 1-hop CALLERS of of= (defs= matched, count= distinct symbols): ; hop_tested=/hop_untested= +ripwire.callees/v1 : 1-hop CALLEES of of= (defs= matched, count= distinct symbols): +ripwire.uses/v1 : resolvable use-sites of of=: +ripwire.impact/v1 : transitive blast radius of of=: reach set, importers; defs= matched, reaches= their transitive callers, radius_tested= non-tests an indexed test reaches, radius_untested= the rest; importers= files that #include/import a def's file +ripwire.path/v1 : one DIRECTED call path from= to to=, each a hop; reachable=0 hops=0 when none +ripwire.connect/v1 : minimal joining subgraph: groups, terminals, joins, edges, +… [800 more display lines; full output is 84154 bytes on 830 raw line(s)] +````` + +## `./build/ripwire . --lint --lint-select=cache-` + +*Run ONLY one rule family; the root carries selected="K of N" so a filtered zero is never confusable with an unfiltered one.* + +**wall time: 1.54s** + +````` + + + + + + + + + + +vector<std::vector<double>> +nodes[ perm[i] ] +nodes[ perm[ i + 1 ] ] +p = p->next +map +byName[ symN[i] ] +vector<std::vector<std::vector<std::uint32_t>>> +vector<std::vector<std::uint32_t>> +m[ symN[id] ] +vector<std::vector<double>> +m[ in.keys[ i ] ] +map +vector<std::vector<std::vector<std::uint32_t>>> +vector<std::vector<std::uint32_t>> +m[ symN[id] ] +map +m[ g_pool[ g_keyOf[i] ] ] +g_pool[ g_readOf[i] ] +g_pool[ g_readOf[i] ] +m[ g_pool[i] ] +… [553 more display lines; full output is 66790 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --lint --lint-ignore=naming-,cache-` + +*DROP two families, applied after selection; the raw select=/ignore= you passed rides on the root.* + +**wall time: 1.51s** + +````` + + + + + + + + + + + + + + + + + + + + + + + +0.5 +1023u +main (90 lines) +20 +delta < 0.02 && delta > -0.02 +0.02 +… [727 more display lines; full output is 66126 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --lint --lint-select=goto --lint-max-per-rule=4` + +*A rule that spends its own match budget (here 4, set BELOW the default 5000 so the floor shows on this repo): goto carries count_capped="1", its count= is a FLOOR, and the root names the call that counts the rest — findings_next= re-runs only the floored rules under a 10x budget.* + +**wall time: 1.55s** + +````` + + + +goto done; +goto epilogue; +goto epilogue; +goto epilogue; + +````` + +## `./build/ripwire . --lint --lint-select=goto --lint-max-per-rule=40` + +*Pasting that findings_next=: the same rule under a 10x budget counts its true total — no count_capped=, no findings_next=.* + +**wall time: 1.51s** + +````` + + + +goto done; +goto epilogue; +goto epilogue; +goto epilogue; +goto epilogue; +goto epilogue; +goto epilogue; +goto epilogue; +goto epilogue; +goto epilogue; +goto epilogue; +goto epilogue; +goto epilogue; +goto done; +goto done; + +````` + +## `./build/ripwire . --lint --lint-select=cach-` + +*An unresolvable PREFIX refuses (exit 1) with a did-you-mean from a real edit distance (one character off cache-).* + +**exit code: 1** — **wall time: 1.55s** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --lint-select: 'cach-' matches no rule or family (did you mean 'cache-'?) — see --lint-catalog for the full registry +````` + +## `./build/ripwire . --lint --lint-select=nosuchfamily` + +*A PREFIX with no near miss at all: the refusal points at --lint-catalog instead of guessing.* + +**exit code: 1** — **wall time: 1.57s** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --lint-select: 'nosuchfamily' matches no rule or family — see --lint-catalog for the full registry +````` + +## `./build/ripwire . --lint --sarif` + +*The SAME findings as SARIF 2.1.0 (what github/codeql-action/upload-sarif consumes) — pure re-serialization, results count == the native run's.* + +**wall time: 1.56s** + +````` +{"version":"2.1.0","$schema":"https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json","runs":[{"tool":{"driver":{"name":"ripwire","rules":[{"id":"c-style-cast","shortDescription":{"text":"c-style-cast"},"properties":{"builtin":true,"capped":false,"applicable": … [line truncated: 7 more bytes on this line] +{"id":"goto","shortDescription":{"text":"goto"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"do-while","shortDescription":{"text":"do-while"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"unsafe-c-fn","shortDescription":{"text":"unsafe-c-fn"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"weak-crypto","shortDescription":{"text":"weak-crypto"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"redundant-parens","shortDescription":{"text":"redundant-parens"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"suspicious-semicolon","shortDescription":{"text":"suspicious-semicolon"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"typedef-over-using","shortDescription":{"text":"typedef-over-using"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"magic-number","shortDescription":{"text":"magic-number"},"properties":{"builtin":true,"capped":true,"applicable":true}}, +{"id":"empty-catch","shortDescription":{"text":"empty-catch"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"self-assign","shortDescription":{"text":"self-assign"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"large-function","shortDescription":{"text":"large-function"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"deep-nesting","shortDescription":{"text":"deep-nesting"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"inconsistent-return","shortDescription":{"text":"inconsistent-return"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"unreachable-code","shortDescription":{"text":"unreachable-code"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"naming-short","shortDescription":{"text":"naming-short"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"naming-wordy","shortDescription":{"text":"naming-wordy"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"naming-series","shortDescription":{"text":"naming-series"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"naming-underscore","shortDescription":{"text":"naming-underscore"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"naming-case","shortDescription":{"text":"naming-case"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"naming-predicate","shortDescription":{"text":"naming-predicate"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"naming-setter","shortDescription":{"text":"naming-setter"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"naming-confusable","shortDescription":{"text":"naming-confusable"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"naming-uninformative","shortDescription":{"text":"naming-uninformative"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"atom-comma-operator","shortDescription":{"text":"atom-comma-operator"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"atom-embedded-crement","shortDescription":{"text":"atom-embedded-crement"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"atom-assign-as-value","shortDescription":{"text":"atom-assign-as-value"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"atom-nested-ternary","shortDescription":{"text":"atom-nested-ternary"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"atom-implicit-predicate","shortDescription":{"text":"atom-implicit-predicate"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +{"id":"atom-octal-literal","shortDescription":{"text":"atom-octal-literal"},"properties":{"builtin":true,"capped":false,"applicable":true}}, +… [11013 more display lines; full output is 1551625 bytes on 1 raw line(s)] +````` + +Parsed summary of the same SARIF (past the display cut): + +````` +sarif 2.1.0 rules= 39 results= 5502 +````` + +## `./build/ripwire . --lint --sarif --limit=5` + +*SARIF is always the FULL result set: paging alongside it refuses loudly.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --sarif always emits the full result set — drop --limit=N/--offset=M +````` + +## `./build/ripwire . --dmm` + +*The Delta Maintainability Model scalar for the WORKING TREE vs HEAD — recorded against a CLEAN tree (the sandbox section shows a real delta). UNAVAILABLE is a stated reason, never 0 or 1.* + +**wall time: 2.73s** + +````` + + +

+

+ +````` + +## `./build/ripwire . --dmm=HEAD` + +*The per-commit scalar: HEAD vs its first parent, with the three separately actionable sub-scores.* + +**wall time: 6.32s** + +````` + + +

+

+ +````` + +## `./build/ripwire . --dmm=HEAD~3..HEAD` + +*The range form: tree HEAD vs tree HEAD~3.* + +**wall time: 6.25s** + +````` + + +

+

+ +````` + +## `./build/ripwire . --cochange --cochange-groups` + +*Modularity-violation GROUPS instead of pairs: "X co-changes with {A,B,C}, none of which it depends on" — a greedy cover, disclosed as greedy.* + +**wall time: 1.47s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [787 more display lines; full output is 51999 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --cochange --cochange-recur=2` + +*Only pairs whose co-change RECURS in 2+ sub-windows of the mined window (sub_windows= is the denominator) — a one-off sprint stops reading like a structural defect.* + +**wall time: 1.37s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --html=/aux/map2.html --color-by=community` + +*The HTML graph with the initial colour mode set to community (the page embeds all five modes and keeps a live selector).* + +**wall time: 1.53s** + +````` +(empty) +````` + +Artifact written: + +````` + 159470 /aux/map2.html +````` + +## `./build/ripwire . --index-out=/aux/ci_index` + +*CI generate-and-exit: cold-parse and write BOTH committable cache families (lean + rich), no map on stdout.* + +**wall time: 3.26s** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --index-out wrote /aux/ci_index.lean.ripwirecache (23740310 bytes, lean family) +ripwire: --index-out wrote /aux/ci_index.rich.ripwirecache (49997444 bytes, rich family) +````` + +Artifact written: + +````` + 23740310 /aux/ci_index.lean.ripwirecache + 49997444 /aux/ci_index.rich.ripwirecache + 73737754 total +````` + +## `./build/ripwire . --cache=/aux/ci_index.lean.ripwirecache --top-k=3` + +*Consume the lean artifact in a PR job: restore-equivalence, never blob-byte-identity.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [56 more display lines; full output is 8005 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --pin-census=/aux/pin_census.tsv --top-k=3` + +*Eval-only: a per-call-site census of WHICH mechanism resolved each call, and the canonical id of every surviving target.* + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [56 more display lines; full output is 8005 bytes on 1 raw line(s)] +````` + +Artifact written: + +````` +# ripwire pin-census v3 C=kind\tmech\tpre\tpost\tflags\tcaller_id\tcallee\ttargets(|-sep)\tline +# v3 field escape: every id and callee field is escaped, so no separator occurs inside one. A backslash is +# written \\, TAB \t, LF \n, CR \r, every other control byte 0x00-0x1f and the targets separator | as \xHH +# (lowercase hex). Split lines on LF, fields on TAB and targets on |, THEN decode. Columns are unchanged +# from v2 and an id without those bytes is spelled exactly as in v2; a C++ out-of-line template member +# whose template-argument list spans lines keeps its line break as \n instead of splitting the row. +# line is the 1-based call-site line in the caller's file (v2, appended LAST so v1 readers are unchanged): +# the key a SCIP occurrence joins on, so a coverage loss can be classified per site instead of guessed. + 126155 /aux/pin_census.tsv +````` + +## `./build/ripwire . --plan-lint=test/planlintfix/wave.md` + +*The house PLAN/DESIGN format's STRUCTURE check — never semantics; exit 2 when a card or ledger row gates.* + +**exit code: 2** + +````` + + + + + + + + + + + + + +````` + +## `./build/ripwire . --plan-lint=test/planlintfix/wave_ledger.md` + +*The ledger-shaped fixture through the same check (exit 2: one gating card).* + +**exit code: 2** + +````` + + + + + + + +````` + +## `./build/ripwire . --doctor --agent=claude` + +*--doctor plus a LIVE integration inspection for one agent: PATH binary, installed-skill manifest parity, hook executability, MCP wiring — read-only, fixed repair commands, never config contents; exit 1 when any check fails.* + +**exit code: 1** — **wall time: 3.44s** + +````` + + + + + + + + + + + + + +````` + +## `./build/ripwire . --doctor --agent=nosuch` + +*Other --agent values refuse.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: unsupported --agent value 'nosuch' (supported: codex, claude) +````` + + +--- + +# the dirty-tree verbs (throwaway clone, NOT the read-only repo) + +Everything below runs with `cwd` = the throwaway clone at `/dirty` (`git clone --local` of this repo, then one deliberate regression in `src/infra/sortutil.h`). The read-only repo is never touched. The binary is the same `build/ripwire`, addressed absolutely. + +## `./build/ripwire . --situ` + +*Situational report for a real diff: blast radius + tests + co-change + forgotten co-change partners.* + +**wall time: 4.89s** + +````` +ripwire situational-awareness — 1 changed file(s), 13 symbols in them +root: . +at: 6009ca693+dirty + [1] blast radius: 303 symbols across 72 files transitively depend on these changes (showing 8 of 72 files — shown=8 total=72 capped=1 prcontext_cap=20 (--pr-context's own per-file list is cut at 20 too); next: --situ --limit=72) + counts_floor=1 graph_ambiguous=12808 graph_unresolved=12845 graph_unindexed=256 (files no grammar in this build could read at all) (the map header's own gauges) — every count above is a FLOOR, never a total: call edges are name-based, so dynamic dispatch, callbacks and macros can be missin … [line truncated: 46 more bytes on this line] + src/mcpverbs.h (34 dependent symbols) + src/slice.h (30 dependent symbols) + src/crossref.h (13 dependent symbols) + src/verbs_for.h (12 dependent symbols) + src/quality.h (11 dependent symbols) + src/verbs_navigate.h (10 dependent symbols) + src/resolve.h (9 dependent symbols) + src/serialize.h (9 dependent symbols) + [2] tests to run (5) order=evidence: [changed] you edited it, [partner] named after a changed file, then hops asc (1 = direct); "(n): a, b" = n runner-less files sharing that evidence; a (run: …) is relative to root: + test/verify_radix.cpp [hops=1] (run: bash test/greptiercheck.sh) + test/adaptivecutshapefix/adaptive_cut_shape_test.cpp [hops=2] (run: bash test/adaptivecutshapecheck.sh) + test/includeprecise_unit.cpp [hops=2] (run: bash test/includeprecisecheck.sh) + test/verify_csr.cpp [hops=2] (run: bash test/a9disclosurecheck.sh) + test/rustimport_unit.cpp [hops=4] (run: bash test/rustimportprecisecheck.sh) + script_gates_unmodelled=746 — test/*.sh gates never appear above: script-to-binary edges are not call edges (a path count) + [3] co-change — usually edited with these but NOT in your diff (0) window="18mo@HEAD" commits="4365": + (none — 4365 commits were mined and none co-edited a file outside your diff) + next: --test-gate +````` + +## `./build/ripwire . --test-gate` + +*The pre-PR gate with real obligations — exit 4 when tests-to-run or untested blast radius is non-empty.* + +**exit code: 4** — **wall time: 3.85s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --quality-delta` + +*Every row carries p="file:line", the gating rows are marked gating="1" and now bar= (the threshold each numeric row is judged against), and the exit-2 refusal prints a naming line on stderr; a gating row carries a pasteable next=.* + +**exit code: 2** — **wall time: 20.42s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [129 more display lines; full output is 16690 bytes on 1 raw line(s)] +````` + +stderr: + +````` +ripwire: cache : over its 2048 MiB budget — evicted 266 blob(s) of other roots or other ripwire builds (this build's blobs for this root are kept) +ripwire: no ./.ripwire_quality_baseline — auto-comparing the working tree vs git HEAD (commit the baseline with --quality-baseline to pin it) +ripwire: --quality-delta gating: 5 preexisting-worse major finding(s); first: api-surface src/infra/sortutil.h::rw::sortutil::nonNegativeFloatDescKey at src/infra/sortutil.h:109 (was=1 now=2) +````` + +## `./build/ripwire . --quality-delta --legend=full` + +*The same gating report under --legend=full — same rows, same exit 2; the default compact comment (schema="ripwire.quality-delta/v1", the shape an agent's edit loop should run) becomes ~4 KB of prose legend.* + +**exit code: 2** — **wall time: 10.79s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [129 more display lines; full output is 21568 bytes on 1 raw line(s)] +````` + +stderr: + +````` +ripwire: no ./.ripwire_quality_baseline — auto-comparing the working tree vs git HEAD (commit the baseline with --quality-baseline to pin it) +ripwire: --quality-delta gating: 5 preexisting-worse major finding(s); first: api-surface src/infra/sortutil.h::rw::sortutil::nonNegativeFloatDescKey at src/infra/sortutil.h:109 (was=1 now=2) +````` + +## `./build/ripwire . --quality-delta --json` + +*The same findings as JSON (one of the CI/scripting verbs --json supports) — same exit 2 as the XML form.* + +**exit code: 2** — **wall time: 10.84s** + +````` +{"baseline":"git-HEAD","regressions":9,"minor":1,"acked":0,"stale":147,"preexisting-worse":6,"new-symbol":3,"gating":5,"register-macro-excluded":62,"api-new-surface":2,"at":"6009ca693+dirty","renames":60,"rename_window_commits":400,"acked_by_rename":0,"acked_by_content":0,"renames_window_truncated": … [line truncated: 211 more bytes on this line] +{"kind":"complexity","sym":"src/infra/sortutil.h::rw::sortutil::lessByScoreDescId","was":1,"now":67,"p":"src/infra/sortutil.h:49","gating":true}, +{"kind":"dead-code","sym":"src/infra/sortutil.h::rw::sortutil::nonNegativeFloatAscKeyCopy","p":"src/infra/sortutil.h:119","origin":"new-symbol"}, +{"kind":"dead-code","sym":"src/infra/sortutil.h::rw::sortutil::sortScoredIdsWithOptions","p":"src/infra/sortutil.h:129","origin":"new-symbol"}, +{"kind":"duplication","members":"src/infra/sortutil.h::rw::sortutil::nonNegativeFloatAscKeyCopy | src/infra/sortutil.h::rw::sortutil::nonNegativeFloatDescKey","tokens":59,"p":"src/infra/sortutil.h:119","gating":true}, +{"kind":"nesting","sym":"src/infra/sortutil.h::rw::sortutil::lessByScoreDescId","was":1,"now":6,"p":"src/infra/sortutil.h:49","gating":true}, +{"kind":"new-clone-of-reused-helper","sym":"src/infra/sortutil.h::rw::sortutil::nonNegativeFloatAscKeyCopy | src/infra/sortutil.h::rw::sortutil::nonNegativeFloatDescKey","was":0,"now":4,"p":"src/infra/sortutil.h:119","gating":true}, +{"kind":"params","sym":"src/infra/sortutil.h::rw::sortutil::sortScoredIdsWithOptions","was":0,"now":8,"p":"src/infra/sortutil.h:129","origin":"new-symbol"}, +{"kind":"verbosity","sym":"src/infra/sortutil.h::rw::sortutil::lessByScoreDescId","was":8,"now":47,"p":"src/infra/sortutil.h:49","sev":"minor"}], +"sa":[{"kind":"api-surface","key":"085e3d408c2c4a35","why":"target-gone"}, +{"kind":"api-surface","key":"155d74d341a49f3c","why":"target-gone"}, +{"kind":"api-surface","key":"1a15386c2d1e47af","why":"target-gone"}, +{"kind":"api-surface","key":"298e798c7f075715","why":"target-gone"}, +{"kind":"api-surface","key":"56acdf9b5c314a14","why":"target-gone"}, +{"kind":"api-surface","key":"5a07390012b46e06","why":"target-gone"}, +{"kind":"api-surface","key":"6eef859578c8c376","why":"target-gone"}, +{"kind":"api-surface","key":"6f2394762f82a855","why":"target-gone"}, +{"kind":"api-surface","key":"7f2c3eefdf6e512e","why":"target-gone"}, +{"kind":"api-surface","key":"802e513731103806","why":"target-gone"}, +{"kind":"api-surface","key":"b6a24afef32a68a8","why":"target-gone"}, +{"kind":"api-surface","key":"c923e661c197b265","why":"target-gone"}, +{"kind":"api-surface:new-symbol","key":"1926b0d9e94541a0","why":"target-gone"}, +{"kind":"api-surface:new-symbol","key":"1d3814ba687a4aef","why":"target-gone"}, +{"kind":"api-surface:new-symbol","key":"1db76028879244ef","why":"target-gone"}, +{"kind":"api-surface:new-symbol","key":"36e39c7ab0fc1686","why":"target-gone"}, +{"kind":"api-surface:new-symbol","key":"4bce64bd920de0c3","why":"target-gone"}, +{"kind":"api-surface:new-symbol","key":"61f0e361ef7dee27","why":"target-gone"}, +{"kind":"api-surface:new-symbol","key":"79625906f9f71ad0","why":"target-gone"}, +{"kind":"api-surface:new-symbol","key":"8ecb190954dce18a","why":"target-gone"}, +{"kind":"api-surface:new-symbol","key":"995375dfa4e63104","why":"target-gone"}, +… [126 more display lines; full output is 15341 bytes on 1 raw line(s)] +````` + +stderr: + +````` +ripwire: no ./.ripwire_quality_baseline — auto-comparing the working tree vs git HEAD (commit the baseline with --quality-baseline to pin it) +ripwire: --quality-delta gating: 5 preexisting-worse major finding(s); first: api-surface src/infra/sortutil.h::rw::sortutil::nonNegativeFloatDescKey at src/infra/sortutil.h:109 (was=1 now=2) +````` + +## `./build/ripwire . --quality-delta --quality-ack --ack-only=zzznope` + +*NEW FLAG: --ack-only matching nothing REFUSES rather than falling back to acking everything.* + +**exit code: 1** — **wall time: 7.15s** + +````` +(empty) +````` + +stderr: + +````` +ripwire: no ./.ripwire_quality_baseline — auto-comparing the working tree vs git HEAD (commit the baseline with --quality-baseline to pin it) +ripwire: ack provenance backfill — 0 clone row(s) reconstructed from the current tree, 217 re-derived, 16 left UNVERIFIED (their group was not found here — a floor, not proof it is gone), 57 left legacy (member set not found cloning here — a floor, not proof it is gone), 1315 ineligible (no cu … [line truncated: 200 more bytes on this line] +ripwire: --ack-only=zzznope matched none of the 9 finding(s) — nothing written +````` + +## `./build/ripwire . --quality-delta --quality-ack --ack-only=api-surface` + +*NEW FLAG: ack only the api-surface findings — a per-finding ratchet instead of a rubber stamp.* + +**wall time: 7.07s** + +````` +(empty) +````` + +stderr: + +````` +ripwire: no ./.ripwire_quality_baseline — auto-comparing the working tree vs git HEAD (commit the baseline with --quality-baseline to pin it) +ripwire: ack provenance backfill — 0 clone row(s) reconstructed from the current tree, 217 re-derived, 16 left UNVERIFIED (their group was not found here — a floor, not proof it is gone), 57 left legacy (member set not found cloning here — a floor, not proof it is gone), 1315 ineligible (no cu … [line truncated: 200 more bytes on this line] +ripwire: acknowledged 1 of 9 finding(s) (8 left UNACKED by --ack-only, 0 already acked) → ./.ripwire_quality_acks +````` + +## `./build/ripwire . --quality-delta` + +*Re-run after the partial ack: acked=3, the rest still gate (exit 2).* + +**exit code: 2** — **wall time: 10.84s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [128 more display lines; full output is 16393 bytes on 1 raw line(s)] +````` + +stderr: + +````` +ripwire: no ./.ripwire_quality_baseline — auto-comparing the working tree vs git HEAD (commit the baseline with --quality-baseline to pin it) +ripwire: --quality-delta gating: 4 preexisting-worse major finding(s); first: complexity src/infra/sortutil.h::rw::sortutil::lessByScoreDescId at src/infra/sortutil.h:49 (was=1 now=67) +````` + +## `./build/ripwire . --ack-only=gating` + +*--ack-only WITHOUT --quality-ack REFUSES loudly (exit 1, the pairing named) — it used to be silently ignored.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --ack-only=SUBSTR narrows --quality-ack — pass both (e.g. ripwire

--quality-delta --ack-only=contract-change --quality-ack="reason") +````` + +## `./build/ripwire . --edit-check=nonNegativeFloatDescKey` + +*A real contract-change: was=1 now=2 params, with the call sites that are now provably incompatible, and a pasteable next= (--uses=SYM) on the root.* + +**wall time: 1.47s** + +````` + + + + + + +````` + +## `./build/ripwire . --edit-check=nonNegativeFloatDescKey --legend=full` + +*The same verdict under --legend=full: the default's one compact comment (the post-edit reflex at its cheapest) becomes ~5.7 KB of prose legend, every row identical.* + +**wall time: 1.46s** + +````` + + + + + + +````` + +## `./build/ripwire . --pr-context` + +*The review-evidence bundle with an actual changed file.* + +**wall time: 3.97s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [89 more display lines; full output is 8590 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --map-diff --top-k=5` + +*The map re-ranked with a teleport toward the changed file (changed=1 here, not 0).* + +**wall time: 2.29s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --clones` + +*The duplicated helper the sandbox edit introduced shows up as a clone group.* + +**wall time: 1.95s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [306 more display lines; full output is 16116 bytes on 1 raw line(s)] +````` + +## `./build/ripwire . --stray-content=zz-orphan` + +*CHANGED: a ref with NO merge base with HEAD now reports v="unknown" ok="0" in its own bucket — the absence of an answer, never a claim it is merged. (The sandbox carries a deliberately parentless branch built with `git commit-tree`; a shallow CI clone puts every ref here.)* + +**wall time: 1.61s** + +````` + + + + + +````` + +stderr: + +````` +[math degraded] crossref: no merge-base for ref (shallow clone or unrelated history?) — verdict is unknown, not merged (crossref.h:1257, RefPlumbing rw::crossref::probeRefBase(const std::string &, const RefInfo &, const std::string &) — logged once per site) +````` + +## `./build/ripwire . --stray-content=zz-orphan --plan` + +*CHANGED: --plan surfaces those same refs as an row rather than silently dropping them.* + +**wall time: 2.59s** + +````` + + + + +````` + +stderr: + +````` +[math degraded] crossref: no merge-base for ref (shallow clone or unrelated history?) — verdict is unknown, not merged (crossref.h:1257, RefPlumbing rw::crossref::probeRefBase(const std::string &, const RefInfo &, const std::string &) — logged once per site) +````` + +## `./build/ripwire . --dmm` + +*The DMM scalar on a REAL delta: the sandbox edit grew one unit past the nesting/complexity thresholds and added an 8-parameter one, so dmm is low and the three sub-scores say which property moved.* + +**wall time: 3.53s** + +````` + + +

+

+ +````` + +## `./build/ripwire . --quality-delta --scope=src/graph.h` + +*OWNERSHIP partition for a shared tree: every regression here lives in src/infra/, so under a scope naming src/graph.h they ALL print under with a do-not-ack banner and never gate — scoped-out-gating= says how many would have.* + +**wall time: 10.81s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [131 more display lines; full output is 16660 bytes on 1 raw line(s)] +````` + +stderr: + +````` +ripwire: no ./.ripwire_quality_baseline — auto-comparing the working tree vs git HEAD (commit the baseline with --quality-baseline to pin it) +````` + +## `./build/ripwire . --quality-delta --quality-ack --scope=src/graph.h --ack-only=api-surface` + +*The rubber-stamp guard: an --ack-only that names an OUT-OF-SCOPE row refuses (exit 1) and writes nothing.* + +**exit code: 1** — **wall time: 6.86s** + +````` +(empty) +````` + +stderr: + +````` +ripwire: no ./.ripwire_quality_baseline — auto-comparing the working tree vs git HEAD (commit the baseline with --quality-baseline to pin it) +ripwire: --ack-only=api-surface selects 1 finding(s) OUT OF SCOPE for --scope=src/graph.h — refusing, and writing nothing at all: + api-surface src/infra/sortutil.h::rw::sortutil::nonNegativeFloatDescKey at src/infra/sortutil.h:109 + those rows belong to whoever is editing those paths. Acking them here writes their debt into a committed ledger under YOUR + reason string, which is how a per-finding ratchet becomes a rubber stamp. Narrow the pattern, or widen the scope if they really are yours. +````` + +## `./build/ripwire . --handoff` + +*The continuation packet with a REAL diff: verified changed symbols + blast radius + tests-to-run, then the heuristic rows.* + +**wall time: 3.36s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +````` + +## `./build/ripwire . --note-add="lessByScoreDescId: keep this branch-free — it sits inside the PageRank sort comparator"` + +*Pin a field note (write-side memory) to a symbol; committed to .ripwire_notes in the sandbox. The BARE name is resolved through the same resolver the read verbs use and stored as the canonical id — the rewrite is echoed on stderr, because a silent one is not a disclosure.* + +````` +src/infra/sortutil.h::rw::sortutil::lessByScoreDescId 2026-10-10 keep this branch-free — it sits inside the PageRank sort comparator 6009ca69336d0adceb8f7ce6019372f7fd413e5c integration/train-26c +````` + +stderr: + +````` +ripwire: --note-add: tip: notes that say "chose X over Y because Z" surface better — consider adding the why +ripwire: --note-add: target 'lessByScoreDescId' canonicalised to 'src/infra/sortutil.h::rw::sortutil::lessByScoreDescId' — that is the id --for/--expand key notes by +````` + +## `./build/ripwire . --notes` + +*The note is listed under the canonical id, dangling="0" — i.e. it will actually surface. (Before the H1 fix the bare name was stored verbatim and read dangling="1": recorded, and surfaced nowhere.)* + +````` + + + + + + + + + + + + + + + + gate scripts' claim (~line 1305) is NOT enforced — the derived-vs-stated sibling loop here covers docs/EVALS.md only. It drifted 407→451 unnoticed (fixed 2026-08-23). To close: grep both files ('file:line:' parsing) in the gateCountClaims arm.]]> + + + + +````` + +## `./build/ripwire . --note-add="gitOneLine: which one?"` + +*Two definitions carry this name, so the write REFUSES rather than pick one: a note keys ONE canonical id, and an ambiguous selector is refused, never silently narrowed. Every candidate is named, with a runnable retry.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --note-add: tip: notes that say "chose X over Y because Z" surface better — consider adding the why +ripwire: --note-add: target 'gitOneLine' is ambiguous — it matches 2 definitions in 2 distinct contracts, and a note keys ONE canonical id (it would surface on one of them and look absent on the rest). Qualify one: ./src/handoff.h:gitOneLine, ./src/quality.h:gitOneLine — e.g. --note-add="./src/h … [line truncated: 33 more bytes on this line] +````` + +## `./build/ripwire . --note-add="lessByScoreDescIdd: typo"` + +*A name that resolves to nothing is refused with the read verbs' own did-you-mean — never written as a dead note.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --note-add: tip: notes that say "chose X over Y because Z" surface better — consider adding the why +ripwire: --note-add: target not found: lessByScoreDescIdd (did you mean 'lessByScoreDescId'?) — a note keys the canonical id a read verb resolves; to note a FILE instead, pass a path (one with a '/' or an extension), which may name a file that does not exist yet +````` + +## `./build/ripwire . --note-add="src/infra/sortutil.h::rw::sortutil::lessByScoreDescId: chose the flat two-branch compare over the nested ladder because the comparator sits inside the PageRank sort"` + +*The same symbol addressed by its CANONICAL id: already canonical, so nothing is rewritten and both notes land on ONE target.* + +````` +src/infra/sortutil.h::rw::sortutil::lessByScoreDescId 2026-10-10 chose the flat two-branch compare over the nested ladder because the comparator sits inside the PageRank sort 6009ca69336d0adceb8f7ce6019372f7fd413e5c integration/train-26c +````` + +## `./build/ripwire . --expand=lessByScoreDescId --top-k=0` + +*Both notes riding along with the symbol's body — the elements follow the body, past the display cut, so they are extracted below.* + +````` + + + + + + + + + + + + + +````` + +## `./build/ripwire . --insert-after-symbol=lessByScoreDescId --edit-payload=/aux/payload_note.h` + +*Insert immediately AFTER one uniquely-resolved definition; replaced_bytes=0 because the insert verbs never overwrite. The receipt carries the folded post-edit verification (lines=, edit_check, tests_to_run) so the loop closes in one call.* + +**wall time: 3.45s** + +````` +{"applied":"insert_after_symbol","symbol":"lessByScoreDescId","file":"src/infra/sortutil.h","span":{"start":2375,"end":2463},"lines":{"start":56,"end":58},"replaced_bytes":0,"old_file_bytes":10824,"new_file_bytes":10912,"file_eol":"lf","eol_normalized":false,"trailing_newline_folded":true,"separator … [line truncated: 652 more bytes on this line] +"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true},"tests_to_run":[{"p":"test/verify_radix.cpp","hops":1,"run":"bash test/greptiercheck.sh"}, +{"p":"test/adaptivecutshapefix/adaptive_cut_shape_test.cpp","hops":2,"run":"bash test/adaptivecutshapecheck.sh"}, +{"p":"test/includeprecise_unit.cpp","hops":2,"run":"bash test/includeprecisecheck.sh"}, +{"p":"test/verify_csr.cpp","hops":2,"run":"bash test/a9disclosurecheck.sh"}, +{"p":"test/rustimport_unit.cpp","hops":4,"run":"bash test/rustimportprecisecheck.sh"}], +"order":"evidence","partners":0,"run_first":1,"tests":5,"script_gates_unmodelled":746,"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true,"next":"bash test/greptiercheck.sh"} +````` + +stderr: + +````` +ripwire edit: applied atomically; receipt carries region, blob_sha, edit_check, tests_to_run; next: bash test/greptiercheck.sh +````` + +## `./build/ripwire . --insert-after-symbol=lessByScoreDescId --edit-payload=/aux/payload_note.h --no-post-check` + +*The opt-out: the same insert with the folded verification skipped — lines= still rides (it is free), edit_check/tests_to_run do not, and the two pasteable commands stay on stderr.* + +**wall time: 1.60s** + +````` +{"applied":"insert_after_symbol","symbol":"lessByScoreDescId","file":"src/infra/sortutil.h","span":{"start":2375,"end":2463},"lines":{"start":56,"end":58},"replaced_bytes":0,"old_file_bytes":10912,"new_file_bytes":11000,"file_eol":"lf","eol_normalized":false,"trailing_newline_folded":true,"separator … [line truncated: 572 more bytes on this line] +````` + +stderr: + +````` +ripwire edit: applied atomically; receipt carries region, blob_sha (post-check skipped); next: --edit-check=src/infra/sortutil.h:lessByScoreDescId +````` + +## `./build/ripwire . --insert-before-symbol=nonNegativeFloatDescKey --edit-payload=/aux/payload_note.h --edit-target-file=src/infra/sortutil.h` + +*Insert BEFORE, with --edit-target-file pinning which same-named definition (here unambiguous — the disambiguator is simply honoured).* + +**wall time: 3.75s** + +````` +{"applied":"insert_before_symbol","symbol":"nonNegativeFloatDescKey","file":"src/infra/sortutil.h","span":{"start":2895,"end":2983},"lines":{"start":74,"end":75},"replaced_bytes":0,"old_file_bytes":11000,"new_file_bytes":11088,"file_eol":"lf","eol_normalized":false,"trailing_newline_folded":false,"s … [line truncated: 755 more bytes on this line] +{"n":"benchAdaptive","p":"bench/bench_radix_ab.cpp:157","l":[162]}, +{"n":"radixSortNonNegativeFloatsDesc","p":"src/infra/sortutil.h:105","l":[114]}, +{"n":"radixSortByScoreDescId","p":"src/infra/sortutil.h:120","l":[180]}], +"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true},"tests_to_run":[{"p":"test/verify_radix.cpp","hops":1,"run":"bash test/greptiercheck.sh"}, +{"p":"test/adaptivecutshapefix/adaptive_cut_shape_test.cpp","hops":2,"run":"bash test/adaptivecutshapecheck.sh"}, +{"p":"test/includeprecise_unit.cpp","hops":2,"run":"bash test/includeprecisecheck.sh"}, +{"p":"test/verify_csr.cpp","hops":2,"run":"bash test/a9disclosurecheck.sh"}, +{"p":"test/rustimport_unit.cpp","hops":4,"run":"bash test/rustimportprecisecheck.sh"}], +"order":"evidence","partners":0,"run_first":1,"tests":5,"script_gates_unmodelled":746,"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true,"next":"--uses=src/infra/sortutil.h:nonNegativeFloatDescKey"} +````` + +stderr: + +````` +ripwire edit: applied atomically; receipt carries region, blob_sha, edit_check, tests_to_run; next: --uses=src/infra/sortutil.h:nonNegativeFloatDescKey +````` + +## `./build/ripwire . --replace-symbol-body=DoesNotExist --edit-payload=/aux/payload_note.h` + +*An unknown TARGET refuses and leaves every file byte-identical.* + +**exit code: 1** — **wall time: 1.59s** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --replace-symbol-body: symbol 'DoesNotExist' not found +````` + +## `./build/ripwire . --edit-plan=/aux/edit_plan.json --dry-run` + +*A versioned multi-edit TRANSACTION preflighted without writing: the receipt shows what each op would read and touch.* + +**wall time: 1.60s** + +Input file: + +````` +{ + "version": 1, + "edits": [ + { + "op": "insert_before_symbol", + "target": "nonNegativeFloatDescKey", + "payload": "plan_note.h" + } + ] +} +````` + +````` +{"schema":"ripwire.edit-plan/v1","mode":"dry-run","edits":1,"files":1,"callers_union":4,"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true,"atomic_scope":"per-file","rollback_on_write_error":true,"recheck_before_each_write":true,"multifile_crash_atomic":false, … [line truncated: 354 more bytes on this line] +````` + +## `./build/ripwire . --edit-plan=/aux/edit_plan.json --apply` + +*The same plan committed: per-file locks, re-verify-before-write, atomic rename, rollback on a later failure.* + +**wall time: 3.36s** + +````` +{"schema":"ripwire.edit-plan/v1","mode":"apply","edits":1,"files":1,"callers_union":4,"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true,"applied":1,"atomic_files":1,"atomic_scope":"per-file","rollback_on_write_error":true,"recheck_before_each_write":true,"mul … [line truncated: 538 more bytes on this line] +{"n":"benchAdaptive","p":"bench/bench_radix_ab.cpp:157","l":[162]}, +{"n":"radixSortNonNegativeFloatsDesc","p":"src/infra/sortutil.h:107","l":[116]}, +{"n":"radixSortByScoreDescId","p":"src/infra/sortutil.h:122","l":[182]}], +"graph_ambiguous":12805,"graph_unresolved":12845,"graph_unindexed":256,"counts_floor":true}}]} +````` + +## `./build/ripwire . --edit-plan=/aux/edit_plan.json` + +*Neither --dry-run nor --apply: the mode is explicit, so this refuses.* + +**exit code: 1** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --edit-plan requires exactly one of --dry-run or --apply +````` + +## `./build/ripwire . --quality-delta` + +*After the agent's edits: the complexity/nesting rows on lessByScoreDescId are gone (the replace undid them), the rest still gate (exit 2).* + +**exit code: 2** — **wall time: 11.14s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [125 more display lines; full output is 15757 bytes on 1 raw line(s)] +````` + +stderr: + +````` +ripwire: no ./.ripwire_quality_baseline — auto-comparing the working tree vs git HEAD (commit the baseline with --quality-baseline to pin it) +ripwire: --quality-delta gating: 2 preexisting-worse major finding(s); first: duplication src/infra/sortutil.h::rw::sortutil::nonNegativeFloatAscKeyCopy | src/infra/sortutil.h::rw::sortutil::nonNegativeFloatDescKey at src/infra/sortutil.h:88 (was=0 now=59) +````` + +## `./build/ripwire . --quality-baseline` + +*REFUSES, exit 1: this sandbox tree is already regressed, and pinning here would swallow that debt into the floor so every later delta read clean. It names how many gating findings it would absorb, the first of them, and the way forward.* + +**exit code: 1** — **wall time: 7.17s** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --quality-baseline: this tree already holds 2 gating finding(s) against HEAD — pinning here would absorb + them into the floor, and every later --quality-delta would read clean. First: duplication src/infra/sortutil.h::rw::sortutil::nonNegativeFloatAscKeyCopy | src/infra/sortutil.h::rw::sortutil::nonNegativeFloatDescKey (was=0 now=59) + Commit the tree first, or pass --allow-dirty to pin anyway (the sidecar then records the 2 absorbed, and every + report against it carries baseline_absorbed="2"). +````` + +## `./build/ripwire . --quality-baseline --allow-dirty` + +*The consent form: pin anyway. The sidecar is stamped with the dirty pin and the absorbed count, so the fact outlives the process that knew it.* + +**wall time: 10.05s** + +````` +(empty) +````` + +stderr: + +````` +ripwire: --quality-baseline --allow-dirty: pinned with 2 gating finding(s) ABSORBED into the floor (stamped in the sidecar; every --quality-delta against it carries baseline_absorbed="2") +ripwire: wrote ./.ripwire_quality_baseline (snapshot of 26062 per-symbol rows, 26905 indexed symbols total) +````` + +Artifact written: + +````` + 4419391 .ripwire_quality_baseline +# ripwire quality baseline v6 — regenerate with --quality-baseline; do not hand-edit +producer 01b0d59d63f2a402c75ee73a7c3b9e942e6094ebf92838a6833a4d824cfec943 +head 6009ca69336d0adceb8f7ce6019372f7fd413e5c +dirty 1 +absorbed 2 +ccx d19d7c13cbc7 2 +ccx 1f7edbff929e5 10 +ccx 6a732f6699bdb 0 +ccx 7d8862be4e +````` + +## `./build/ripwire . --quality-delta` + +*Against that sidecar the same tree reads regressions=0 — but baseline_absorbed= is on the root, so this green means clean SINCE THE PIN, never clean. A baseline is a floor YOU chose, and it belongs BEFORE the change.* + +**wall time: 10.00s** + +````` + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +… [120 more display lines; full output is 14641 bytes on 1 raw line(s)] +````` + + +--- + +# the MCP dialect — the same verbs over stdio JSON-RPC (one-shot exchange, not a persistent server) + +## `printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize"}' '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | ./build/ripwire --mcp` + +*initialize + tools/list: the manifest an agent host loads at session start — every verb's name, description and input schema.* + +````` +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-11-25","serverInfo":{"name":"ripwire","version":"1.0"},"capabilities":{"tools":{},"resources":{}},"instructions":"Map before reading files. Start a new task with explore; use from_trace for an error; use impact plus uses before changing a sym … [line truncated: 738 more bytes on this line] +{"jsonrpc":"2.0","id":2,"result":{"tools":[{"name":"analyze","description":"Architecture map for a directory: signatures and the call graph for the top symbols. Use when landing cold in a repo or subdir, before reading files; for a task-scoped inventory use 'for', for one symbol's neighborhood find_ … [line truncated: 742 more bytes on this line] +{"name":"rank_by","description":"The SAME architecture map 'analyze' serves, ranked by a different signal instead of plain PageRank. rank_by = pagerank (default, omit it — the CLI's own unbiased --rank-by=pagerank; 'analyze' can rank differently on a tree with uncommitted changes, where it biases … [line truncated: 1077 more bytes on this line] +{"name":"find_symbol","description":"A symbol's 1-hop neighborhood: the symbol (with a fetch_body handle) plus direct callers (calledBy) and callees (calls). Full transitive reach: 'impact'. Read/write/import sites, not just calls: 'uses'. JSON {symbol, calledBy, calls, defs, count, hop_tested, hop_ … [line truncated: 1181 more bytes on this line] +{"name":"find_referencing_symbols","description":"Direct (1-hop) callers of a symbol, each with a fetch_body handle. For the full transitive blast radius use 'impact', for read/write/import sites 'uses'. JSON {symbol, calledBy, defs, count, hop_tested, hop_untested, declined_calls, counts_floor}; ca … [line truncated: 945 more bytes on this line] +{"name":"grep","description":"Trigram literal search; each hit annotated with its enclosing symbol (which function/class it is in). Hits carry text; enclosing rows a fetch_body handle. pattern = the literal; in=any lifts the span tiering (default: tightest non-empty tier; rest rides as suppressed_co … [line truncated: 837 more bytes on this line] +{"name":"cochange","description":"Files that historically change together with this file — the co-edit partners. Fowler's Shotgun Surgery as change coupling. dep_capable=false means neither side could carry one (sh/md/json/binary), so surprising is undefined rather than informative. file = the fil … [line truncated: 700 more bytes on this line] +{"name":"memory_recall","description":"Most relevant memory notes / docs for a task, full text — the few that matter, not the whole corpus. path = docs/memory dir; task = what you're working on; top_k = docs to return, 1..1000 (default 8), refused outside that band, never clamped; budget_tokens = … [line truncated: 790 more bytes on this line] +{"name":"situational_awareness","description":"The 5 things to know about a diff, as JSON: blast_radius, tests_to_run, forgotten (usual co-change partners missing from this diff), hotspot_alert, modules_touched. forgotten = the Shotgun Surgery check. diff/files optional — defaults to 'git diff HEA … [line truncated: 1297 more bytes on this line] +{"name":"mentions","description":"Docs (markdown plans/designs) that name a code symbol in a backtick. symbol = the code symbol name; limit/offset page the files. An @FILE:LINE line-seed rebinds to the innermost definition enclosing that line and answers for it, disclosing the rebound name as 'sym' … [line truncated: 625 more bytes on this line] +{"name":"for","description":"Task-lens ranked, signatures-only inventory of the building blocks most relevant to a task (cx=complexity, in=reuse-count). task = the task in plain words; budget_tokens = an optional ceiling (the CLI --for --token-budget). The header carries route= (which ranker answere … [line truncated: 1053 more bytes on this line] +{"name":"lego","description":"Interface-to-impls view for ONE named interface/base: its method contract plus EVERY implementor (own-language only), each with file path. Use when implementing against a KNOWN interface (contrast 'for', which sprays top interfaces for a task). type = interface/base nam … [line truncated: 657 more bytes on this line] +{"name":"owners","description":"Bus-factor: recency-weighted (6-month half-life) author ownership per file. symbol = optional, restricts to the file that defines it; limit/offset page the rows. at= is the commit these numbers were computed at (+dirty = the working tree differed). An @FILE:LINE line- … [line truncated: 879 more bytes on this line] +{"name":"replace_symbol_body","description":"Replace a symbol's ENTIRE definition (signature through closing brace) with new_body — splices over the full def span, preserving every byte outside it verbatim; new_body must be a complete, well-formed definition. One trailing newline folds (trailing_n … [line truncated: 1323 more bytes on this line] +{"name":"insert_before_symbol","description":"Insert text immediately BEFORE a symbol's definition (its first byte); padded to the file's own blank-line seam (separator_padded=N). Same refusal contract as replace_symbol_body (not found / ambiguous / stale index / symlink / concurrent write, file unc … [line truncated: 877 more bytes on this line] +{"name":"insert_after_symbol","description":"Insert text immediately AFTER a symbol's definition (past its final byte, which is preserved exactly); padded to the file's own blank-line seam (separator_padded=N); one trailing newline folds (trailing_newline_folded). Same refusal contract as replace_sy … [line truncated: 878 more bytes on this line] +{"name":"fetch_body","description":"Full (or partial-range) source of a symbol's definition, addressed by the stable `handle` a read verb attached to it (bodies on request, not by default). start_line/end_line are optional, 1-based, INCLUSIVE and BODY-RELATIVE (line 1 = the def's first line), clampe … [line truncated: 1049 more bytes on this line] +{"name":"exemplar","description":"BEFORE writing a function / method / class / struct / interface / variable, get the repo's single best-in-class instance of that kind to imitate — signature AND full body. chosen by ROLE, NEVER by text similarity to your task: candidates are first filtered to cogn … [line truncated: 1375 more bytes on this line] +{"name":"quality_delta","description":"Your PR self-check, run every time you think a change is DONE — pairs with the CLI-only --test-gate (names the tests to run + the untested blast radius; not MCP-exposed) to form the two-step pre-PR gate. Reports ONLY what your working tree made WORSE vs basel … [line truncated: 867 more bytes on this line] +{"name":"quality_baseline","description":"PIN the quality floor: writes .ripwire_quality_baseline stamped with the current git HEAD sha, snapshotting complexity / duplication / dead-code / API surface. Call once at the start of non-trivial work, then quality_delta compares against this pinned floor … [line truncated: 445 more bytes on this line] +{"name":"impact","description":"IS IT SAFE TO CHANGE X? — the TRANSITIVE blast radius of a symbol via calls, nearest first (d= hop depth), then PageRank. Use before modifying or deleting a symbol; it beats find_referencing_symbols (direct callers only), and 'uses' catches the read/write/import sit … [line truncated: 1325 more bytes on this line] +{"name":"uses","description":"The STATICALLY RESOLVABLE use-sites of a symbol, not just calls: role (call | read | write | import | extends), file:line, and enclosing symbol. Use to see the footprint before renaming or changing a name — find_referencing_symbols and impact follow only calls. extern … [line truncated: 1146 more bytes on this line] +{"name":"affected","description":"WHICH TESTS TO RUN for a change — test files that transitively reach the changed files/symbols, ranked by evidence (edited > partner-named > hop distance). files = changed files and/or symbols, comma-separated: each item is tried as an indexed PATH pattern first, … [line truncated: 1238 more bytes on this line] +{"name":"path_between","description":"Does A REACH B, and HOW? — the shortest directed CALL path between two symbols, hop-by-hop. reachable=\"0\" hops=\"0\" is a valid 'not reachable' answer — call edges are name-based, so a missing dynamic/callback edge can hide a real path. Named path_between … [line truncated: 709 more bytes on this line] +{"name":"connect","description":"When a task touches 2..16 named symbols, returns the minimal subgraph RELATING them - terminals, the fewest joining intermediaries (with signatures), and call edges in true direction - finding the shared-caller joins a directed path_between cannot; unrelated symbols … [line truncated: 950 more bytes on this line] +{"name":"explore","description":"ONE-call task orientation: the routed+anchored ranking, full bodies of the top hits, their 1-hop callers, field notes, and tests_to_run — ALL under one deterministic byte budget, in a fixed section order (ranking > bodies > callers > notes > tests) that degrades gr … [line truncated: 2117 more bytes on this line] +{"name":"from_trace","description":"Paste a stack trace / sanitizer report / compiler error and get it mapped onto indexed symbols, ranked INNERMOST-first: the parsed frame map, the ranked suspects' signatures, and the innermost in-corpus symbol's FULL body. Out-of-corpus frames are listed a … [line truncated: 1226 more bytes on this line] +{"name":"edit_check","description":"Just edited a symbol? Did its CONTRACT (param count + publicness) change vs git HEAD, and which 1-hop callers are NOW INCOMPATIBLE with the new arity by fixed-arity evidence (not a guess — every folded definition disagrees)? This is call sites worth OPENING, not … [line truncated: 1908 more bytes on this line] +{"name":"whereis","description":"WHERE DOES THIS CONTENT LIVE? Which branch's tree defines or mentions a symbol, HEAD first, with on-head=0 naming the case this verb exists for: content that lives only on a branch (a finished fix stranded on 1 of 30 refs). Each distinct blob is read once (content-ad … [line truncated: 1385 more bytes on this line] +{"name":"stray_content","description":"Per branch: the lines its own divergent work AUTHORED (vs its merge-base with HEAD) that the live line does NOT have. Four verdicts (unmerged+superseded+merged+unknown=refs): v=unmerged is genuinely absent; v=superseded means the live line re-implemented the wo … [line truncated: 1123 more bytes on this line] +{"name":"flags","description":"WHAT IS BUILT BUT DARK here — the answer to 'why don't I see feature X?'. Harvests all three gate patterns (ifndef/define header gates, CMake option(), getenv reads) with each gate's kind, DEFAULT, the size of the code it guards, and its read sites. When a name is bo … [line truncated: 1434 more bytes on this line] +{"name":"doc_drift","description":"WHICH OF THIS REPO'S DOC CLAIMS ARE NOW FALSE. Verifies the CHECKABLE anchors in every markdown file against the live index and returns ONLY the ones that no longer hold: file:line refs (missing-file / past-eof / line-moved), backticked symbol mentions (undefined), … [line truncated: 1167 more bytes on this line] +{"name":"slice","description":"WHERE IS THIS VARIABLE DEFINED AND USED inside one function — NAME-BASED intra-procedural def-use rows of one variable inside ONE uniquely-resolved definition (the ARISE slicer, arXiv:2605.03117). symbol alone lists the sliceable locals to pick from; add var (or spel … [line truncated: 1722 more bytes on this line] +{"name":"batch","description":"ONE-TURN CONTEXT SWEEP: answer up to 16 heterogeneous READ sub-queries in a single call (the deterministic $0 counterpart of a parallel-search agent). queries = array over the SAME path, in EITHER grammar: {verb, ...args} objects, or the CLI --batch file's own \"verb:a … [line truncated: 2040 more bytes on this line] +````` + +The manifest, summarised (name / description bytes / required args) — what the host pays in context every session: + +````` +tools= 33 manifest_bytes= 47143 (~tokens at 4 bytes/token: 11785 ) +explore desc_bytes= 1322 schema_bytes= 958 required=['task'] +batch desc_bytes= 1235 schema_bytes= 962 required=['queries'] +edit_check desc_bytes= 1180 schema_bytes= 889 required=['symbol'] +slice desc_bytes= 953 schema_bytes= 932 required=['symbol'] +flags desc_bytes= 881 schema_bytes= 720 required=[] +whereis desc_bytes= 701 schema_bytes= 851 required=['symbol'] +exemplar desc_bytes= 954 schema_bytes= 591 required=[] +replace_symbol_body desc_bytes= 865 schema_bytes= 610 required=['symbol', 'new_body'] +impact desc_bytes= 821 schema_bytes= 666 required=['symbol'] +situational_awareness desc_bytes= 865 schema_bytes= 581 required=[] +affected desc_bytes= 790 schema_bytes= 610 required=['files'] +from_trace desc_bytes= 829 schema_bytes= 559 required=['trace'] +find_symbol desc_bytes= 752 schema_bytes= 590 required=['symbol'] +doc_drift desc_bytes= 728 schema_bytes= 604 required=[] +uses desc_bytes= 626 schema_bytes= 682 required=['symbol'] +stray_content desc_bytes= 678 schema_bytes= 604 required=[] +rank_by desc_bytes= 758 schema_bytes= 481 required=[] +for desc_bytes= 445 schema_bytes= 777 required=['task'] +fetch_body desc_bytes= 623 schema_bytes= 588 required=['handle'] +connect desc_bytes= 535 schema_bytes= 580 required=['symbols'] +find_referencing_symbols desc_bytes= 505 schema_bytes= 590 required=['symbol'] +insert_after_symbol desc_bytes= 458 schema_bytes= 577 required=['symbol', 'text'] +insert_before_symbol desc_bytes= 455 schema_bytes= 578 required=['symbol', 'text'] +owners desc_bytes= 425 schema_bytes= 625 required=[] +quality_delta desc_bytes= 833 schema_bytes= 193 required=[] +grep desc_bytes= 445 schema_bytes= 562 required=['pattern'] +memory_recall desc_bytes= 447 schema_bytes= 500 required=['task'] +path_between desc_bytes= 357 schema_bytes= 506 required=['from', 'to'] +cochange desc_bytes= 388 schema_bytes= 476 required=['file'] +analyze desc_bytes= 546 schema_bytes= 318 required=[] +lego desc_bytes= 360 schema_bytes= 467 required=['type'] +mentions desc_bytes= 305 schema_bytes= 486 required=['symbol'] +quality_baseline desc_bytes= 407 schema_bytes= 193 required=[] +````` + +## `printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize"}' '{"jsonrpc":"2.0","id":2,"method":"tools/list"}' | ./build/ripwire --mcp --mcp-tools=core` + +*--mcp-tools=core: the server lists only the core profile's tools (explore, batch, from_trace, impact, uses, fetch_body, edit_check, quality_delta), so a host that loads every schema at session start pays only for those.* + +````` +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-11-25","serverInfo":{"name":"ripwire","version":"1.0"},"capabilities":{"tools":{},"resources":{}},"instructions":"Map before reading files. Start a new task with explore; use from_trace for an error; use impact plus uses before changing a sym … [line truncated: 955 more bytes on this line] +{"jsonrpc":"2.0","id":2,"result":{"tools":[{"name":"fetch_body","description":"Full (or partial-range) source of a symbol's definition, addressed by the stable `handle` a read verb attached to it (bodies on request, not by default). start_line/end_line are optional, 1-based, INCLUSIVE and BODY-RELAT … [line truncated: 1092 more bytes on this line] +{"name":"quality_delta","description":"Your PR self-check, run every time you think a change is DONE — pairs with the CLI-only --test-gate (names the tests to run + the untested blast radius; not MCP-exposed) to form the two-step pre-PR gate. Reports ONLY what your working tree made WORSE vs basel … [line truncated: 867 more bytes on this line] +{"name":"impact","description":"IS IT SAFE TO CHANGE X? — the TRANSITIVE blast radius of a symbol via calls, nearest first (d= hop depth), then PageRank. Use before modifying or deleting a symbol; it beats find_referencing_symbols (direct callers only), and 'uses' catches the read/write/import sit … [line truncated: 1325 more bytes on this line] +{"name":"uses","description":"The STATICALLY RESOLVABLE use-sites of a symbol, not just calls: role (call | read | write | import | extends), file:line, and enclosing symbol. Use to see the footprint before renaming or changing a name — find_referencing_symbols and impact follow only calls. extern … [line truncated: 1146 more bytes on this line] +{"name":"explore","description":"ONE-call task orientation: the routed+anchored ranking, full bodies of the top hits, their 1-hop callers, field notes, and tests_to_run — ALL under one deterministic byte budget, in a fixed section order (ranking > bodies > callers > notes > tests) that degrades gr … [line truncated: 2117 more bytes on this line] +{"name":"from_trace","description":"Paste a stack trace / sanitizer report / compiler error and get it mapped onto indexed symbols, ranked INNERMOST-first: the parsed frame map, the ranked suspects' signatures, and the innermost in-corpus symbol's FULL body. Out-of-corpus frames are listed a … [line truncated: 1226 more bytes on this line] +{"name":"edit_check","description":"Just edited a symbol? Did its CONTRACT (param count + publicness) change vs git HEAD, and which 1-hop callers are NOW INCOMPATIBLE with the new arity by fixed-arity evidence (not a guess — every folded definition disagrees)? This is call sites worth OPENING, not … [line truncated: 1908 more bytes on this line] +{"name":"batch","description":"ONE-TURN CONTEXT SWEEP: answer up to 16 heterogeneous READ sub-queries in a single call (the deterministic $0 counterpart of a parallel-search agent). queries = array over the SAME path, in EITHER grammar: {verb, ...args} objects, or the CLI --batch file's own \"verb:a … [line truncated: 2040 more bytes on this line] +````` + +The core manifest, summarised (name / description bytes / required args): + +````` +tools= 8 manifest_bytes= 14166 (~tokens at 4 bytes/token: 3541 ) +explore desc_bytes= 1322 schema_bytes= 958 required=['task'] +batch desc_bytes= 1235 schema_bytes= 962 required=['queries'] +edit_check desc_bytes= 1180 schema_bytes= 889 required=['symbol'] +impact desc_bytes= 821 schema_bytes= 666 required=['symbol'] +from_trace desc_bytes= 829 schema_bytes= 559 required=['trace'] +uses desc_bytes= 626 schema_bytes= 682 required=['symbol'] +fetch_body desc_bytes= 623 schema_bytes= 588 required=['handle'] +quality_delta desc_bytes= 833 schema_bytes= 193 required=[] +````` + +## `printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize"}' '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"for","arguments":{"path":".","task":"pagerank power iteration"}}}' | ./build/ripwire --mcp` + +*MCP `for`: always bundle=sigs (never the CLI's compact route), the same ranked signatures as --for.* + +````` +{"jsonrpc":"2.0","id":1,"result":{"protocolVersion":"2025-11-25","serverInfo":{"name":"ripwire","version":"1.0"},"capabilities":{"tools":{},"resources":{}},"instructions":"Map before reading files. Start a new task with explore; use from_trace for an error; use impact plus uses before changing a sym … [line truncated: 738 more bytes on this line] +{"jsonrpc":"2.0","id":2,"result":{"content":[{"type":"text","text":". window: shown= capped= (capped=1 cut). at=: commit+dirty+shallow. next= … [line truncated: 6526 more bytes on this line] +"_index":"[index: files=2787 symbols=26903 hash=6fb46779]","_fresh":"ok"}} +{"jsonrpc":"2.0","id":3,"result":{"content":[{"type":"text","text":"", out ); const std::string headDate = whereisHeadDate( res ); @@ -3138,6 +4224,7 @@ inline void writeWhereisListedPage( std::FILE* out, const WhereResult& res, std: // text hit), which is a claim strong enough to override the line-removal oracle outright. So the fate // row is printed unless the index itself already proved the symbol is defined on HEAD. writeWhereisFate( out, res, ex ); + writeWhereisUnparsed( out, res, listed.asked ); writeWhereisRows( out, res, listed, hitPage, headDate, ex ); rw::emitRaw( out, "" ); @@ -3157,7 +4244,9 @@ inline ListedHits whereisServedListing( const WhereResult& res, std::size_t maxH { if( listing != WhereisListing::ShorterOfDefsAll ) { - return listedHits( res, listing ); + ListedHits named = listedHits( res, listing ); + named.asked = listing; // an explicit listing rides the (writeWhereisUnparsed) + return named; } ListedHits defs = listedHits( res, WhereisListing::Defs ); if( defs.attr.empty() ) @@ -3170,11 +4259,22 @@ inline ListedHits whereisServedListing( const WhereResult& res, std::size_t maxH // lists are in hit-list order and Defs is the def subsequence, so the definitions the All page shows are a PREFIX // of the Defs page's: comparing the counts compares the sets. More definitions wins whatever its bytes; only equal // sets fall to the bytes rule below. + // Counted as DEFINITION ROWS OF THE WHOLE LIST: on the Defs page a representative row stands for defRefs rows + // (foldSharedBranchDefs), on the All page every row is its own. The prefix argument still holds: each def row + // the All page shows is either a representative (on the Defs page no later than its own def-subsequence + // position, so within the first n) or folds into one that precedes it, and every (representative, ref) pair + // is counted once in defRefs — so the Defs page's weighted count is at least the All page's. const auto defsShown = [ & ]( const ListedHits& l ) { - const std::size_t n = std::min( l.rows.size(), maxHits ); - return std::size_t( std::count_if( l.rows.begin(), l.rows.begin() + std::ptrdiff_t( n ), - [ & ]( std::size_t i ) { return res.hits[ i ].isDef; } ) ); + const std::size_t n = std::min( l.rows.size(), maxHits ); + const bool weighted = l.attr == "defs"; + std::size_t shown = 0; + for( std::size_t k = 0; k < n; ++k ) + { + const WhereHit& h = res.hits[ l.rows[ k ] ]; + shown += !h.isDef ? 0 : ( weighted ? std::max( h.defRefs, 1 ) : 1 ); + } + return shown; }; const std::size_t defsOnDefsPage = defsShown( defs ); const std::size_t defsOnAllPage = defsShown( all ); diff --git a/src/docparse.h b/src/docparse.h index b90ff7a1e..8bd9f5e11 100644 --- a/src/docparse.h +++ b/src/docparse.h @@ -616,6 +616,26 @@ inline std::string shellQuote( const std::string& s ) } // namespace detail +// Could `markitdown ` start at all? The bridge runs through the shell, and a shell that finds no command answers +// rc 127, which runMarkitdown reads as "" — so a machine WITHOUT markitdown paid one /bin/sh start per binary doc on +// every warm run, for an answer known before the fork (cli-floor 2026-10-08: ~25 ms of a warm call on this repository, +// two present/ decks). This asks the same question the shell would, with no process: os::which walks PATH exactly as +// sh does (an empty entry is the current directory; a directory or a non-executable file named markitdown is not a +// program). It is asked per bridge doc, just before the popen it replaces, and never memoized, so installing +// markitdown under a running MCP server is seen on its next ingest. It answers TRUE whenever it cannot rule the shell out — PATH unset +// (sh then searches its own default path), an executable `./markitdown` (a Windows shell searches the current +// directory first), or an exported shell function of that name — because a false "absent" would drop a Section the +// shell would have produced, while a false "present" only costs the popen it always cost. Gated by +// test/docmdcachecheck.sh arms 6-9. +inline bool markitdownMayRun() +{ + if( std::getenv( "PATH" ) == nullptr || std::getenv( "BASH_FUNC_markitdown%%" ) != nullptr || std::getenv( "BASH_FUNC_markitdown()" ) != nullptr ) + { + return true; + } + return !os::which( "markitdown" ).empty() || !os::which( "./markitdown" ).empty(); +} + // Run `markitdown ` and return its stdout (Markdown), or "" if markitdown is not on PATH / it failed. // NOTE: this performs a subprocess call — used only for binary doc formats, only when they are ingested. inline std::string runMarkitdown( const std::string& path ) diff --git a/src/forpage.h b/src/forpage.h index 859149442..72496fc76 100644 --- a/src/forpage.h +++ b/src/forpage.h @@ -156,6 +156,316 @@ inline std::string forWidenNext( std::string_view task ) return forPageInvocation( task, kForPageRowsDefault, 0 ); } +// ── THE BUDGETED-BUNDLE CANDIDATE PAGE (issue #294; the ruling + the #362 review round) ─────────── +// What the caller passes, in one struct (the #362 review's request-struct ask: forCandidatePageDoc's +// parameter list was 12 deep; this is also the runPackTask quality gate's remedy — one struct, not +// eight loose scalars threaded through two callers). +struct ForCandidatePageReq +{ + std::string_view task; + std::string_view verbName; // "--for=" / "--pack-task=": next= names THEIR verb (#294 review) + std::string_view rankFlags; // B2 (#362 review round 4): the argv fragment of every accepted flag + // that changed THIS ranking — a pasted next= without them re-ranks + // differently and walks a different list. CLI: --no-route, + // --no-mention-boost, --no-doc-mention, --cochange-boost; the MCP + // twins expose only no_route (their other lifts are env ablations, + // not request surface). The bundle-shaping set is refused beside + // a window instead (cli.h), so it never needs an echo. + bool pasteHandle = true; // B1 (#362 review round 4): every surface carries the pasteable + // handle now — the five page paths rank one list (a CLI window + // forces fullDistribution, exactly what the twins always scored + // with), so a CLI continuation of an MCP page walks the same + // candidates. The guard stays so a future caller cannot + // silently drop the handle. + bool compactLegend = false; // emit in the compact dialect (the default posture): the page is + // compacted IN-DOC by the central layer (applyCompactDialect, + // candidate-page) so the measured est_tokens=/over_ceiling= + // describe the bytes actually emitted (#362 review, item 4 — + // the CLI-side compaction used to reprice a page the trim loop + // had measured against the full legend, and the stderr line + // contradicted the page). Pre-compacted here, the outer layers + // see schema= and pass it through (AlreadyCompact). + std::string_view routeNote; + std::string_view rootArg; + rw::RedactCounts* redactPtr = nullptr; + std::string_view atStamp; // gitstamp::stampAt: the index this page answered from + std::size_t tokenBudget = 0; + int pageLimit = 0; + int pageOffset = 0; +}; + +inline std::string forCandidatePageDoc( const rw::IngestResult& ing, const std::vector& lensRank, + const rw::AdaptiveCut& forCut, const ForCandidatePageReq& p ) +{ + using namespace rw; // the file's idiom: function-scoped + const std::string_view task = p.task; + const std::string_view verbName = p.verbName; + const std::string_view rankFlags = p.rankFlags; + const bool pasteHandle = p.pasteHandle; + const bool compactLegend = p.compactLegend; + const std::string_view routeNote = p.routeNote; + const std::string_view rootArg = p.rootArg; + rw::RedactCounts* const redactPtr = p.redactPtr; + const std::string_view atStamp = p.atStamp; + const std::size_t tokenBudget = p.tokenBudget; + const int pageLimit = p.pageLimit; + const int pageOffset = p.pageOffset; + + // the ranked candidate order: ALL ids sorted (score desc, id asc) — the same total order + // packSignatures selects with — truncated to the positive-scored candidates (the set the + // bundle's cut was against; total= counts it on EVERY page, so a re-decided denominator is + // impossible by construction). candIdxOf maps an id back to its candidate index: the TRUE + // resume point comes from the last PRINTED row's index + 1 (#362 review, item 2), not from + // a served-count arithmetic that pseudo-symbols (a candidate slot with no printed row) skew. + std::vector candOrder; + std::vector candIdxOf( ing.symbols.size(), kNoNode ); + { + const std::size_t S = ing.symbols.size(); + candOrder.resize( S ); + for( NodeId i = 0; i < S; ++i ) { candOrder[ i ] = i; } + rw::sortutil::radixSortByScoreDescId( candOrder, lensRank ); + candOrder.resize( std::min( forCut.positiveHits, S ) ); + for( std::size_t k = 0; k < candOrder.size(); ++k ) { candIdxOf[ candOrder[ k ] ] = std::uint32_t( k ); } + } + const std::size_t candidateTotal = candOrder.size(); + PageWindow win = pageWindow( candidateTotal, pageLimit, pageOffset ); + // SINGLE-TIER WINDOWS (#362 review, item 4): a head window never crosses the cliff — it ends + // AT the cliff, so every page is one tier and tier= never labels below-cliff rows as head. + // (cliffRank is the head count itself: "a cut at rank i keeps ranks [1,i]" — lexical.h.) + const bool below = forCut.cliffRank > 0 && win.begin >= forCut.cliffRank; + if( forCut.cliffRank > 0 && win.begin < forCut.cliffRank && win.end > forCut.cliffRank ) + { + win.end = forCut.cliffRank; + } + + // window the rank copy: candidates inside [win.begin, win.end) keep their score, everything + // else sorts below them (-1), so packSignatures' top-N selection serves exactly the window in + // rank order — no new selection logic, and the H1 ladder still trims within the page. + std::vector pageRank( ing.symbols.size(), -1.0f ); + for( std::size_t i = win.begin; i < win.end; ++i ) { pageRank[ candOrder[ i ] ] = lensRank[ candOrder[ i ] ]; } + + // the sigs block's byte budget: the page's shell (head+tail+legend) is RESERVED ahead of it + // (kForPageSpliceReserve, #362 review's named-constant ask), and the block never gets less + // than kForPageSigsFloorBytes — the ONE-ROW floor (#362 review, item 3): a budget that cannot + // fit a row serves the top row with over_ceiling="1" rather than an empty page whose next= + // points back at itself (a stuck loop). + constexpr std::size_t kForPageSpliceReserve = 600; + constexpr std::size_t kForPageSigsFloorBytes = 512; + const std::size_t sigsBytesBudget = tokenBudget > 0 + ? std::max( std::size_t( double( tokenBudget ) * kMinBytesPerToken ) > kForPageSpliceReserve + ? std::size_t( double( tokenBudget ) * kMinBytesPerToken ) - kForPageSpliceReserve : 0, + kForPageSigsFloorBytes ) + : kForPageSigsFloorBytes; + + // SINK FORM (Diagnostics.h §4b — the contract selfcheckcheck (R) pins): the page document is the + // sink. The degrade field is what the emitter reads and surfaces as reason= on the page itself, + // so a degraded render tells the release reader IN THE ANSWER — never silently-empty stdout, + // never a sink-less stderr-only note (selfcheckcheck: the pin goes DOWN, never up). + struct ForPageRender + { + enum class DisclosureWhy : std::uint8_t { ChargeStreamRefused, ChargeStreamTorn }; + DisclosureWhy why = DisclosureWhy::ChargeStreamRefused; + void disclose( DisclosureWhy w ) noexcept { why = w; } + } forPageRender; + + // the printed rows: packSignatures reports the ids it EMITTED (shownIdsOut) — the honest shown + // (rows printed) and the honest resume (the last printed row's candidate index + 1). + std::vector shownIds; + std::string sigsDoc; + bool rendered = false; + { + const std::optional redactBefore = redactPtr != nullptr ? std::optional( *redactPtr ) : std::nullopt; + const auto restoreRedact = [ & ]() noexcept { if( redactBefore ) { *redactPtr = *redactBefore; } }; + rw::MemoryStream stream; + std::FILE* const buffer = rw::openChargeStream( stream ); + if( buffer == nullptr ) + { + DISCLOSE( forPageRender, ForPageRender::DisclosureWhy::ChargeStreamRefused ); + restoreRedact(); + } + else + { + packSignatures( buffer, ing, pageRank, int( win.end - win.begin ), sigsBytesBudget, /*metrics=*/true, + nullptr, nullptr, redactPtr, + nullptr, nullptr, nullptr, nullptr, // Q3 enrichments: page rows are the core sig form + /*rankAdaptivePayload=*/true, + sigsBytesBudget, // H1 ladder over the page + nullptr, rootArg, + rw::SigLensRules::RelevanceFloor, nullptr, // the PR's hasRelevanceFloor=true under 26b's SigLensRules (train-26c seam; the + // page rows take no e=/docs-after-code rule — open train decision, as reviewed) + /*shownIdsOut=*/&shownIds, /*cappedOut=*/nullptr, /*topRowNext=*/{}, /*cutOut=*/nullptr, /*spelling=*/{}, + std::uint32_t( win.begin ) ); // rankBase: continuation rows carry their GLOBAL r= (#294 review) + const rw::MemoryStreamBytes block = stream.finish(); + if( !block.isWhole ) + { + DISCLOSE( forPageRender, ForPageRender::DisclosureWhy::ChargeStreamTorn ); + restoreRedact(); + } + else + { + sigsDoc.assign( block.bytes ); + rendered = true; + } + } + } + + // ── the assembly: one set of lambdas, reused by the degrade page AND the trim loop AND the ── + // ── final assembly, so the three cannot drift (#362 review's reuse ask) ─────────────────── + // one scratch buffer PER attribute: the escape helper appends to the buffer and returns a view + // of it, so two calls sharing one buffer inside one argument list race on realloc and mangle + // each other (the bug this page's first CI round caught on route=). + std::vector escTask; const std::string_view taskAttr = escapeXml( task, escTask ); + std::vector escRoute; const std::string_view routeAttr = escapeXml( routeNote, escRoute ); + std::vector escRoot; const std::string_view rootAttr = escapeXml( rootArg, escRoot ); + std::vector escAt; // the at= scratch (one buffer per attribute) + const auto buildHead = [ & ]( std::size_t shownNow, std::size_t nextOff, std::string_view extra ) { + return " 0 ? pageLimit : 0 ) + + "\" tier=\"" + ( below ? "below-cliff" : "head" ) + "\"" + + std::string( extra ); + }; + // THE PASTEABLE CONTINUATION HANDLE — the file page's next= contract: the argv that walks to + // next_offset. Dropped entirely past kNextAttrMaxBytes (a hint that pastes wrong is worse than + // no hint) and dropped when has_more="0" (#362 review, item 3: a last page handing back a next + // that returns an empty page carrying the same next= is a loop with no exit). + const auto buildNextAttr = [ & ]( std::size_t nextOff ) { + if( !pasteHandle ) { return std::string(); } // no surface passes false today (B1); the guard stays honest + if( nextOff >= candidateTotal ) { return std::string(); } + std::string inv = nextFlag( verbName, task ); // THEIR verb: pasting continues the same verb (#294 review) + inv += rankFlags; // B2: the ranking's own flags ride the handle, or the paste re-ranks a different list + inv += " --token-budget=" + std::to_string( tokenBudget ); + if( pageLimit > 0 ) { inv += " --limit=" + std::to_string( pageLimit ); } + inv += " --offset=" + std::to_string( nextOff ); + if( inv.size() > kNextAttrMaxBytes ) { return std::string(); } + return nextAttrXml( inv ); + }; + // next_tier (the ruling's suggestion 2): the tier the CONTINUATION page will carry — only + // beside a next= (nothing further means no tier to advertise). + const auto buildNextTierAttr = [ & ]( std::size_t nextOff ) { + if( !pasteHandle ) { return std::string(); } // rides beside next= only + if( nextOff >= candidateTotal ) { return std::string(); } + return std::string( " next_tier=\"" ) + ( ( forCut.cliffRank > 0 && nextOff >= forCut.cliffRank ) ? "below-cliff" : "head" ) + "\""; + }; + // est_tokens is MEASURED from the page's own final bytes (legend included); the tail's digit + // count feeds back into the total, so one re-derivation closes the loop (it converges immediately). + const auto buildTail = [ & ]( const std::string& h, std::string_view legend, std::string_view r, + bool overCeiling, std::size_t nextOff ) { + std::size_t est = std::size_t( double( h.size() + legend.size() + r.size() + 96 ) / kMinBytesPerToken ); + std::string t; + for( int pass = 0; pass < 3; ++pass ) + { + t = " est_tokens=\"" + std::to_string( est ) + + "\" budget_tokens=\"" + std::to_string( tokenBudget ) + + "\" root=\"" + std::string( rootAttr ) + "\"" + buildNextAttr( nextOff ) + buildNextTierAttr( nextOff ) + + ( overCeiling ? " over_ceiling=\"1\"" : std::string() ) + + ( atStamp.empty() ? std::string() : " at=\"" + std::string( escapeXml( atStamp, escAt ) ) + "\"" ) + ">"; + const std::size_t fin = std::size_t( double( h.size() + t.size() + legend.size() + r.size() ) / kMinBytesPerToken ); + if( fin == est ) { break; } + est = fin; + } + return std::pair( t, est ); + }; + // THE LEGEND — the page's own vocabulary, defined the way every other answer defines its own: + // a leading comment after the root tag, full dialect emitted here, compact dialect rewritten by + // the central table (compactlegend.h: applyCompactDialect with the "for"/"pack-task" hint), so + // --legend=full|compact works through the SAME machinery on both dialects (#362 review, item 5). + // the legend text: NO double-hyphen anywhere (G4 — it rides inside an XML comment), so the + // verbs are named dashless, the file page's own convention ("the two flags are named without + // their dashes"). + const std::string dashlessVerb = verbName == "--pack-task=" ? std::string( "pack-task" ) + : std::string( "for" ); + const std::string legend = + ""; + + // the honest degrade page: the emitter READS the sink — the full quintet with zero rows and + // reason= naming what was lost, built by the SAME lambdas so it cannot drift (#362 review's ask). + if( !rendered ) + { + const std::string reason = " reason=\"" + std::string( forPageRender.why == ForPageRender::DisclosureWhy::ChargeStreamTorn + ? "sigs-charge-stream-torn" : "sigs-charge-stream-refused" ) + "\""; + const std::string h = buildHead( 0, win.end, reason ); + const auto [ t, est ] = buildTail( h, legend, "", /*overCeiling=*/false, win.end ); + return h + t + legend + ""; + } + // rows: packSignatures' block minus its own open tag (kept verbatim with its close). + const std::size_t openBeg = sigsDoc.find( "' ); + const std::size_t keepFrom = ( openBeg != std::string::npos && openEnd != std::string::npos ) ? openEnd + 1 : 0; + std::string rowsNow = sigsDoc.substr( keepFrom ); + + // THE TRUE RESUME (#362 review, item 2): the last PRINTED row's candidate index + 1 — NOT + // win.begin + a served count: a pseudo-symbol (a candidate slot packSignatures serves without + // printing a row) skews the arithmetic one short, and the next page re-serves the last row + // (the duplicate seam). shownIds is packSignatures' own emitted-ids report, in rank order. + std::size_t kept = shownIds.size(); + std::size_t nextNow = ( kept > 0 ) ? std::size_t( candIdxOf[ shownIds.back() ] ) + 1 : win.end; + // the forward guarantee (#362 review, item 3): next_offset ALWAYS moves past this page's + // window start when anything was printed — a next= that points back at the page it came + // from is a loop with no exit. + if( kept > 0 && nextNow <= win.begin ) { nextNow = win.begin + 1; } + + // THE BUDGET TRIM (#362 review, item 1): the page IS an answer under the budget. The H1 + // ladder already trimmed within the block; if the assembled page still exceeds, drop + // trailing WHOLE rows — cutting at the START of the last row (a cut at its close leaves + // the row open and the document malformed) — never below ONE row while the window is + // non-empty (item 3: a budget that cannot fit a row serves the top row with over_ceiling="1" + // rather than an empty page whose next= points back at itself), and re-derive the page's own + // arithmetic (shown/next_offset/has_more/next=) from what shipped. + std::string h = buildHead( kept, nextNow, {} ); + auto [ t, est ] = buildTail( h, legend, rowsNow, /*overCeiling=*/false, nextNow ); + if( tokenBudget > 0 && est > tokenBudget ) + { + const std::size_t rowFloor = ( win.end > win.begin ) ? std::size_t( 1 ) : std::size_t( 0 ); + while( kept > rowFloor && est > tokenBudget ) + { + const std::size_t lastClose = rowsNow.rfind( "" ); + const std::size_t rowStart = rowsNow.rfind( ""; + --kept; + nextNow = ( kept > 0 ) ? std::size_t( candIdxOf[ shownIds[ kept - 1 ] ] ) + 1 : win.end; + h = buildHead( kept, nextNow, {} ); + std::tie( t, est ) = buildTail( h, legend, rowsNow, /*overCeiling=*/false, nextNow ); + if( est <= tokenBudget ) { break; } + } + if( est > tokenBudget ) + { + std::tie( t, est ) = buildTail( h, legend, rowsNow, /*overCeiling=*/true, nextNow ); + } + } + std::string pageOut = h + t + legend + rowsNow; + // (#362 review, item 4) the dialect actually emitted: compact first (the central layer — it + // reprices est_tokens= and settles over_ceiling= on the bytes it writes), so the page's own + // numbers describe what ships; the outer layers see schema= and pass it through untouched. + if( compactLegend ) + { + applyCompactDialect( pageOut, "candidate-page" ); + } + // the stderr notice reads the FINAL page: est_tokens straight off the emitted root, so the + // line can never contradict the document (the full-dialect trim numbers used to). + if( tokenBudget > 0 ) + { + const std::size_t finalEst = rootUnsignedAttr( std::string_view( pageOut ).substr( pageOut.find( '<' ), pageOut.find( '>' ) + 1 ), "est_tokens" ); + if( finalEst > tokenBudget && pageOut.find( " over_ceiling=\"1\"" ) != std::string::npos ) + { + rw::emitTo( stderr, "ripwire: for-page est_tokens={} exceeds the stated budget={} — the window cannot fit; " + "the top row is served with over_ceiling=1\n", finalEst, tokenBudget ); + } + } + return pageOut; +} + + struct ForFileRow { std::uint32_t fileId = 0; diff --git a/src/gitmine.h b/src/gitmine.h index cee6a5b88..b11fb4073 100644 --- a/src/gitmine.h +++ b/src/gitmine.h @@ -19,6 +19,7 @@ // gitmine.h no longer carries its own copy; see jsonesc.h for the dedup rationale #include +#include // popenTrimmed's 64 KB read block #include // the join's once-per-process disclosure flags #include // std::popcount — the recurrence mask's set-bit count (Clio sub-windows) #include @@ -238,23 +239,27 @@ inline std::string sinceUnresolvedRefusal( std::string_view value, std::string_v // popen-trim shape in the tool: the quality.h git one-liners, the doctor probes, crossref.h and binstale.h all // reach it as quality::popenTrimmed (a using-declaration of this). It lives HERE — the lower header, which // quality.h includes — because resolveSinceScope needed the same reader (N4, 2026-09-04) and a second copy in -// this file was a --quality-delta new-clone-of-reused-helper row. Built on readByteSafeLine (infra/stdinline.h), -// the G3 byte-safe line reader every git pipe in this header uses: no fixed buffer to split a long path on -// (churnjoincheck's G3 arm forbids fgets here), a high byte never sign-changed. Lines are re-joined with the -// '\n' the reader consumed, so multi-line callers (crossref.h, the rename map) read exactly what fgets gave them. +// this file was a --quality-delta new-clone-of-reused-helper row. It reads the WHOLE stream in blocks and splits +// nothing: the answer is every byte the command wrote, then the trailing whitespace trim. That is byte-for-byte +// what the line reader it replaces produced (each line re-joined with the '\n' it consumed, an unterminated last +// line given one, which the trim then removes), with no fixed buffer to split a long path on (churnjoincheck's +// G3 arm) and no high byte ever sign-changed — but one fread per 64 KB instead of one locked fgetc per byte. The +// per-byte read was 55% of a --whereis on a 137-branch checkout (one `git ls-tree -r` per branch, cli-floor +// 2026-10-08), and every git one-liner in the tool reads through here. inline std::string popenTrimmed( const std::string& cmd ) { + PROFILE_SCOPE_DESCRIBE( "gitmine: popenTrimmed (one shell + git child, block read)" ); std::FILE* pipe = os::popen( cmd.c_str(), "r" ); if( !pipe ) { return {}; } - std::string out; - std::string line; - while( readByteSafeLine( pipe, line ) ) + std::string out; + std::array block; + for( std::size_t got = std::fread( block.data(), 1, block.size(), pipe ); got > 0; got = std::fread( block.data(), 1, block.size(), pipe ) ) { - out += line; - out += '\n'; + ASSUME( got <= block.size(), "fread returns at most the count it was asked for" ); + out.append( block.data(), got ); } os::pclose( pipe ); while( !out.empty() && ( out.back() == '\n' || out.back() == '\r' || out.back() == ' ' ) ) @@ -669,8 +674,24 @@ struct GitCommandLines std::vector lines; bool isStarted = false; int status = 0; + bool abandoned = false; // the read stopped early (abandonGitWalks): lines is a prefix, never an answer }; +// A git walk this run no longer needs: --for / --metrics / --exemplar start the 18-month history walk beside the ingest +// (main.cpp startHistoryWalks), and a memory-guard refusal used to join that walk's future before exiting — the refusal +// line was on stderr, the exit code waited for the end of the log (CodeRabbit on #383, main.cpp:3721). Set once, +// process-wide, only on an exit path: the walk's reader (gitLogNameOnlyRaw, the `--name-only` stream that IS the 431 ms +// walk) and the shared line reader (gitCommandLines) stop at their next line and pclose, git gets SIGPIPE on its next +// write, and the join ends there. A block reader (popenTrimmed: the walk's sha-list freshness probe, milliseconds on a +// real repository) runs to its end. A line reader that stopped early says so (`abandoned`), and the stream cache +// (quality.h gitRawCommitStreamCached) never writes a prefix. test/memguardcheck.sh (B21). +inline std::atomic gGitWalksAbandoned{ false }; +inline void abandonGitWalks() noexcept { gGitWalksAbandoned.store( true, std::memory_order_relaxed ); } +inline bool gitWalksAbandoned() noexcept { return gGitWalksAbandoned.load( std::memory_order_relaxed ); } +// the walk readers' line read: false at the end of the stream, or as soon as the walks are abandoned (the lines read so +// far are then a prefix no caller serves or caches — both are guarded by the same flag) +inline bool readWalkLine( std::FILE* pipe, std::string& line ) { return !gitWalksAbandoned() && readByteSafeLine( pipe, line ); } + inline GitCommandLines gitCommandLines( const std::string& cmd ) { GitCommandLines out; @@ -696,7 +717,7 @@ inline GitCommandLines gitCommandLines( const std::string& cmd ) // is a silently DROPPED commit rather than a fabricated one — still a number the reader would trust. // It leaves a trailing '\r' in place by contract, which is why the strip below still runs. std::string line; - while( readByteSafeLine( pipe, line ) ) + while( readWalkLine( pipe, line ) ) { while( !line.empty() && ( line.back() == '\n' || line.back() == '\r' ) ) { @@ -707,7 +728,8 @@ inline GitCommandLines gitCommandLines( const std::string& cmd ) out.lines.push_back( line ); } } - out.status = os::pclose( pipe ); + out.abandoned = gitWalksAbandoned(); // the rest of the stream was never read; pclose ends the child at its next write + out.status = os::pclose( pipe ); return out; } @@ -1560,8 +1582,41 @@ struct RawCommitStream { struct Commit { std::int64_t epoch = 0; std::vector paths; }; std::vector commits; + // K51: the walk COULD NOT BE READ (git would not start, or exited non-zero part-way) — as against "read, and the + // window holds no commit". `commits` is then empty or a prefix, never a measurement, and the memoizing caller + // must not store it: an empty stream cached under the (repo, HEAD, window) key silenced churn=/amp= until HEAD moved. + bool unread = false; + // Why, when `unread`: git never ran (no popen, or the shell could not find/execute it: exit 127/126) — `commits` is + // empty — as against git ran and stopped part-way, `commits` the newest-first prefix it printed before it died. + bool notRun = false; + // git's decoded exit code (WEXITSTATUS), -1 when it did not exit normally or never started; 0 on a clean read. + int exitCode = 0; }; +// K51: mark a walk unread. `exitCode` is git's decoded exit (-1: it never started, or did not exit normally); the shell's 127 +// (not on PATH) and 126 (not executable) mean git never ran either, so `commits` is empty, not a prefix. +inline void markWalkUnread( RawCommitStream& out, int exitCode, bool neverStarted ) noexcept +{ + out.unread = true; + out.exitCode = exitCode; + out.notRun = neverStarted || exitCode == 127 || exitCode == 126; +} + +// A pclose()/wait status as an exit code: WEXITSTATUS on a normal exit, -1 for a signal or an unreadable status. +inline int decodedExitCode( int status ) noexcept +{ + return WIFEXITED( status ) ? WEXITSTATUS( status ) : -1; +} + +// K51: fold the walk's pclose() status into `out` — 0 is a clean read and changes nothing. +inline void noteWalkExit( RawCommitStream& out, int status ) noexcept +{ + if( status != 0 ) + { + markWalkUnread( out, decodedExitCode( status ), false ); + } +} + inline RawCommitStream gitLogNameOnlyRaw( const std::string& root, const std::string& coSince ) { PROFILE_SCOPE_DESCRIBE( "gitmine: gitLogNameOnlyRaw (git log --name-only popen)" ); @@ -1572,11 +1627,12 @@ inline RawCommitStream gitLogNameOnlyRaw( const std::string& root, const std::st std::FILE* pipe = os::popen( cmd.c_str(), "r" ); if( !pipe ) { + markWalkUnread( out, -1, true ); return out; } std::string s; - while( readByteSafeLine( pipe, s ) ) // F6: THE line reader, not a char[4096] a long path can be split across + while( readWalkLine( pipe, s ) ) // F6: THE line reader, not a char[4096] a long path can be split across; stops when abandoned { while( !s.empty() && ( s.back() == '\n' || s.back() == '\r' ) ) { @@ -1598,7 +1654,9 @@ inline RawCommitStream gitLogNameOnlyRaw( const std::string& root, const std::st } out.commits.back().paths.push_back( std::move( s ) ); // readByteSafeLine clear()s its buffer first, so moving out of it is safe } - os::pclose( pipe ); + // K51: git died (a missing or corrupt object, a bad ref) — the lines above are a prefix, not the window — or the shell + // never ran it (127: not on PATH, 126: not executable), and there are no lines at all. + noteWalkExit( out, os::pclose( pipe ) ); return out; } diff --git a/src/graph.h b/src/graph.h index e8eb7b1bc..681635140 100644 --- a/src/graph.h +++ b/src/graph.h @@ -5603,6 +5603,15 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null PROFILE_SCOPE_DESCRIBE( "buildGraph/1d: byName (name -> def ids)" ); for( const Symbol& s : ing.symbols ) { + if( s.kind == SymKind::Section && s.lang == Lang::Ruby ) + { + continue; // definitions-only (#339): a Ruby schema column answers --uses/--grep/--whereis as + // a def, but admits NO call edge — the byName table feeds both call binding and the + // common-name damping, so an untyped `response.code` must not become a column's + // caller, and 200 `name` columns must not damp a model's real `def name`. + // Mirrors contextratio.h's isMeasurableKind treatment of Sections. Edges may come + // later behind an evidence rule (receiver proven to be the owning model). + } byName[ s.name ].push_back( s.id ); } } @@ -6054,6 +6063,88 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null }; rw::SmallVec rubyReachable; // reused per-call buffer: the candidates a Ruby call to self can reach (RubySelfReach) + // ---- the resolve loop's dense per-candidate views (cli-perf-068) --------------------------------------------------- + // The loop below walks a name's WHOLE candidate list several times per call — the language filter, the namespace gate, + // the same-file and same-directory tiers — and each step read one field out of a Symbol at a random id. That is cheap + // for a name with two definitions and was the resolve loop's cost for a name with hundreds: since every module-scope + // call got an owner (24bb6e5d), the test scripts' top-level `ok "…"` / `no "…"` calls (~650 same-named shell + // definitions each, ~28k calls on this repository) reached those walks, and the loop went 22 -> 79 ms on that commit + // alone. symLang/symKind are copies of the two Symbol fields those walks read (symFileId, above, is the third), so + // each walk reads a dense array in id order instead: same values, same order, same answer. `nameHasFnLocalDef` is + // the one precomputed fact reachableByName needs to say "nothing to set aside": a name none of whose definitions is + // function-local cannot yield (localDefOutOfReach is false for every candidate, and every list the loop hands + // reachableByName is a subset of the name's byName list), so its walk — a Symbol load plus a binary search per + // candidate — is skipped with the answer it would have returned. Keyed by the byName entry's address: nothing + // inserts into or erases from byName past its construction, so the address is stable for the loop. + std::vector symLang( N ); + std::vector symKind( N ); + ankerl::unordered_dense::set*> nameHasFnLocalDef; + { + PROFILE_SCOPE_DESCRIBE( "buildGraph/2l: dense candidate views + function-local name set" ); + for( std::size_t symIndex = 0; symIndex < N; ++symIndex ) + { + symLang[ symIndex ] = ing.symbols[ symIndex ].lang; + symKind[ symIndex ] = ing.symbols[ symIndex ].kind; + } + if( !ing.fnLocalScopes.empty() ) + { + for( const auto& [ name, ids ] : byName ) + { + if( std::any_of( ids.begin(), ids.end(), [ & ]( NodeId c ) { return ing.symbols[ c ].fnLocal != 0; } ) ) + { + nameHasFnLocalDef.insert( &ids ); + } + } + } + } + ENSURES( symLang.size() == N && symKind.size() == N && symFileId.size() == N, "the dense views cover every symbol id the loop can hold" ); + + // ---- the per-(name, language) candidate memo: work that is invariant across the calls to one name --------------- + // For a name with many definitions, three walks over its list decide the same thing for every call written in the + // same language: which candidates survive the language filter (the name-based fallback's admission loop), whether + // the namespace gate removes any of them for the call's role, and how the survivors fall by file. The resolve loop + // used to redo all three per call — 650 candidates for every one of ~28k module-scope `ok` / `no` calls in this + // repository's test scripts (the cand-build stage alone was the loop's largest). This memo computes them ONCE per + // (byName entry, caller language), on the first call that needs them, and every later call copies the survivors and + // finds its same-file tier by binary search over the file-sorted list. Only the unnarrowed whole-name path reads it + // (the list handed in IS the byName entry, single root, nothing in cand yet), only for lists of kLangCandMemoMin or + // more, and `candMemo` is dropped the moment anything but a removal could have touched cand (the Ruby side cut), or + // any removal did (its size no longer matches). Same survivors, same order, same tiers: the answer is unchanged. + struct LangCandidates + { + std::vector ids; // the name's candidates the language filter admits, in list order + std::uint32_t nsPassRoles = 0; // bit k: every one passes namespaceCompatible( RefRole( k ), … ) + bool sortedByFile = false; // ids is non-decreasing in symFileId, so one file's ids are one run + }; + struct NameLangCandidates + { + std::uint32_t filledLangs = 0; // bit k: byLang[ k ] is computed + std::array byLang; + }; + static_assert( kLangCount <= 32 && kRefRoleCount <= 32, "the memo's language and role sets are 32-bit masks" ); + constexpr std::size_t kLangCandMemoMin = 16; // below this the walks are cheaper than the memo lookup + HashMap*, NameLangCandidates> langCandMemo; + langCandMemo.reserve( 1024 ); + const auto fillLangCandidates = [ & ]( const rw::SmallVec& ids, Lang lang, LangCandidates& out ) + { + for( NodeId c : ids ) + { + if( langCompatible( symLang[ c ], lang ) ) + { + out.ids.push_back( c ); + } + } + for( std::size_t roleIndex = 0; roleIndex < kRefRoleCount; ++roleIndex ) + { + const RefRole role = static_cast( roleIndex ); + if( std::all_of( out.ids.begin(), out.ids.end(), [ & ]( NodeId c ) { return namespaceCompatible( role, symKind[ c ] ); } ) ) + { + out.nsPassRoles |= 1u << roleIndex; + } + } + out.sortedByFile = std::is_sorted( out.ids.begin(), out.ids.end(), [ & ]( NodeId a, NodeId b ) { return symFileId[ a ] < symFileId[ b ]; } ); + }; + // ---- census arming + the ORACLE side (eval-only; src/pincensus.h) ------------------------------ // The oracle rows are a straight transcription of the overlay's own (from, calleeName) → target table // into the census's id space — the same table graph.h consults below, so the two sides of the join @@ -6112,6 +6203,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null cand.clear(); tier.clear(); + const LangCandidates* candMemo = nullptr; // set when cand is exactly a memoized list (see langCandMemo) float tierConf = 1.0f; // tier 1: same file (default; overridden below) // census bookkeeping (inert unless armed): which narrowing stages actually FIRED on this site, and // how wide the tier was when it reached the locality tie-break. Read only at the emission point. @@ -6269,7 +6361,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null if( !scipPinned && r.lang != Lang::Elixir && !r.qualifier.empty() ) { appendCanonicalCandidates( cand, qkey, r, CanonicalScopes { canonByName, canonFamilyByName, specializationsWithBases, narrower, chaUp, ing.symbols }, - [ & ]( NodeId c ) { return langCompatible( ing.symbols[c].lang, r.lang ) && sameRoot( c, r.fileId ); } ); + [ & ]( NodeId c ) { return langCompatible( symLang[ c ], r.lang ) && sameRoot( c, r.fileId ); } ); canonical = !cand.empty(); } // ── L3 fn-pointer/callback binding resolve — BEFORE Rule 1, because a local variable shadows a @@ -6578,7 +6670,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null std::size_t found = 0; for( NodeId c : it->second ) { - if( langCompatible( ing.symbols[c].lang, r.lang ) && symFileId[c] == aliasFile ) + if( langCompatible( symLang[ c ], r.lang ) && symFileId[c] == aliasFile ) { only = c; if( ++found > 1 ) @@ -6627,7 +6719,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null // whichever same-named def competes, and yielding would move them from external= into declined=, so such a call // resolves exactly as it did before local defs had bodies. const bool localsYield = !scipPinned && !canonical && !narrowed && !builtinGated && it != byName.end() && !ing.fnLocalScopes.empty() - && reachableByName( ing, *baseIds, r, reachScratch ) + && nameHasFnLocalDef.contains( &it->second ) && reachableByName( ing, *baseIds, r, reachScratch ) && !( r.role == RefRole::Call && r.qualifier.empty() && bindingTier.empty() && externalVeto.isExternalBound( r ) ); const rw::SmallVec* nameIds = localsYield ? &reachScratch : baseIds; // FE-B: for a MEMBER call the include graph says which file the caller imports, never what its receiver is — the @@ -6675,7 +6767,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null std::vector& into = ( hedgeEligible && ( isMemberCallRef( r ) || implicitReceiverLang( r.lang ) ) ) ? memberRule3 : cand; for( NodeId c : rule3Out ) { - if( langCompatible( ing.symbols[c].lang, r.lang ) ) + if( langCompatible( symLang[ c ], r.lang ) ) { into.push_back( c ); } @@ -6709,11 +6801,26 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null continue; } } + else if( !multiRoot && nameIds == baseIds && baseIds == &it->second && cand.empty() && nameIds->size() >= kLangCandMemoMin ) + { + // the whole-name path for a name with many definitions: the language filter's survivors, memoized + // per (name, language) — sameRoot is constant-true on a single root, so they are this loop's own + NameLangCandidates& memo = langCandMemo[ nameIds ]; + const std::size_t langIdx = static_cast( r.lang ); + ASSUME( langIdx < kLangCount, "a reference's language is a Lang enumerator" ); + if( ( memo.filledLangs >> langIdx & 1u ) == 0 ) + { + fillLangCandidates( *nameIds, r.lang, memo.byLang[ langIdx ] ); + memo.filledLangs |= 1u << langIdx; + } + candMemo = &memo.byLang[ langIdx ]; + cand.assign( candMemo->ids.begin(), candMemo->ids.end() ); + } else { for( NodeId c : *nameIds ) { - if( langCompatible( ing.symbols[c].lang, r.lang ) && sameRoot( c, r.fileId ) ) + if( langCompatible( symLang[ c ], r.lang ) && sameRoot( c, r.fileId ) ) { cand.push_back( c ); // same lang, or ObjC↔C++ bridge; same ROOT (A10) } @@ -6721,7 +6828,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null for( std::size_t i = 0; localsYield && cand.empty() && i < baseIds->size(); ++i ) // nothing reachable survived: the whole list { const NodeId c = ( *baseIds )[ i ]; - if( langCompatible( ing.symbols[c].lang, r.lang ) && sameRoot( c, r.fileId ) ) + if( langCompatible( symLang[ c ], r.lang ) && sameRoot( c, r.fileId ) ) { cand.push_back( c ); } @@ -6735,7 +6842,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null { for( NodeId c : *baseIds ) { - if( !langCompatible( ing.symbols[c].lang, r.lang ) || sameRoot( c, r.fileId ) ) + if( !langCompatible( symLang[ c ], r.lang ) || sameRoot( c, r.fileId ) ) { continue; } @@ -6759,6 +6866,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null : RubyCut::Unchanged; cut != RubyCut::Unchanged ) { + candMemo = nullptr; // cand is the side cut's list now, not the memoized one cand.swap( filtScratch ); if( cut == RubyCut::Declined ) { @@ -6782,12 +6890,12 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null // uniquely a macro by construction); see namespaceCompatible's comment for the derivation and // test/nsfiltercheck.sh for the gate that keeps it that way. It is here because it is the seam a // future round would edit, and a silent edit is the failure mode the gate exists to catch. - if( !cand.empty() ) + if( !cand.empty() && !( candMemo != nullptr && ( candMemo->nsPassRoles >> static_cast( r.role ) & 1u ) != 0 ) ) { std::size_t keepCount = 0; for( std::size_t ci = 0; ci < cand.size(); ++ci ) { - if( namespaceCompatible( r.role, ing.symbols[ cand[ci] ].kind ) ) + if( namespaceCompatible( r.role, symKind[ cand[ci] ] ) ) { cand[ keepCount++ ] = cand[ci]; } @@ -6815,6 +6923,10 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null disposition = vetoExternal( r ); // nothing inside std answers → external=, counted External continue; } + if( candMemo != nullptr && cand.size() != candMemo->ids.size() ) + { + candMemo = nullptr; // a gate or guard above removed a candidate: cand is no longer the memoized list + } // ---- FE-B: PROVEN or NAME-ONLY (receiverevidence.h, point 2). A call no rule above answered binds by name; the // candidates the language's own lookup proves are its answer (the rest never were candidates of that lookup), and @@ -6856,6 +6968,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null if( !provedScratch.empty() ) { cand.swap( provedScratch ); + candMemo = nullptr; // cand is Rule 3's proven list now, not the memoized one: the ladder may not read it as that file-sorted run narrowed = true; } } @@ -6983,11 +7096,23 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null } else { - for( NodeId c : cand ) + if( candMemo != nullptr && candMemo->sortedByFile ) { - if( ing.symbols[c].fileId == r.fileId ) + // the memoized list is file-sorted, so this file's candidates are one run, in list order + const auto fileLess = [ & ]( NodeId c, std::uint32_t fileId ) noexcept { return symFileId[ c ] < fileId; }; + const auto lessFile = [ & ]( std::uint32_t fileId, NodeId c ) noexcept { return fileId < symFileId[ c ]; }; + const auto runBegin = std::lower_bound( cand.begin(), cand.end(), r.fileId, fileLess ); + const auto runEnd = std::upper_bound( runBegin, cand.end(), r.fileId, lessFile ); + tier.insert( tier.end(), runBegin, runEnd ); + } + else + { + for( NodeId c : cand ) { - tier.push_back( c ); // tier 1: same file + if( symFileId[ c ] == r.fileId ) + { + tier.push_back( c ); // tier 1: same file + } } } if( tier.empty() ) // tier 2: same directory @@ -6996,7 +7121,7 @@ inline Graph buildGraph( const IngestResult& ing, const ScipOverlay* scip = null const std::uint32_t rdir = fileDir[ r.fileId ]; for( NodeId c : cand ) { - if( fileDir[ing.symbols[c].fileId] == rdir ) + if( fileDir[ symFileId[ c ] ] == rdir ) { tier.push_back( c ); } diff --git a/src/historyread.h b/src/historyread.h new file mode 100644 index 000000000..aa75ef315 --- /dev/null +++ b/src/historyread.h @@ -0,0 +1,95 @@ +#pragma once + +// historyread.h — the ONE spelling of "the git history behind churn= and amp= could not be read in this run" (K51). +// +// The history walk (`git log` over the 18-month window) feeds two numbers every rich verb prints: churn= (--for, the +// commit count of a file) and amp= (a symbol's callers plus the files that share a commit with it). When git cannot +// finish that walk the stream is EMPTY (git never ran: not on PATH, not executable, no popen) or a newest-first PREFIX +// (git ran and stopped part-way: an unreadable object, a bad ref). Undisclosed, both numbers read as "this file never +// changed" / "no co-change partners", or as the whole window when they are only its newest part. This header is the +// disclosure for it, in the dialect of each host document, so the failure is a field of the answer, not a trace. +// +// THE ATTRIBUTE. history_unread="1" on the root of a document that prints churn= or amp=; absent when the walk was read +// (including a read that found no commit — an unborn branch, git's own exit 128 with no HEAD: that is a measurement, and +// it stays silent). It means: churn= and the co-change half of amp= count only the commits that WERE read, so both are +// FLOORS — none read when git did not run (churn= absent, amp= callers only), the newest part of the window when it +// stopped part-way. A prefix is kept rather than dropped because a floor is still information (a file with churn="3" from +// a prefix changed at least 3 times); the attribute is what stops it reading as the full count. An absent churn= under +// the attribute is not zero churn. It does NOT mean the repository is damaged beyond that walk, that the other numbers +// in the document are affected, or that a retry will fail: the failed walk is never cached, so the next call walks again. +// The comment names which cause this run hit. A verb that runs no history walk (a plain map, --callers) never carries it. +// +// NOT COVERED (pre-existing, deferred): the other git walks — --cochange and the MCP cochange tool, --rank-by=churn and +// --owners — do not yet tell a failed walk from an empty one (CHANGELOG, the K51 entry). +// +// ONE RENDERER (the prconverge.h rule): the forms differ only in the syntax of their host, never in what they say, so a +// new host adds a DiscloseAs case here, never a sibling function. XmlAttrs and the comment form (LegendComment) go on every +// XML root that prints the numbers; JsonKeys is the JSON twin. The comment is its own definition, so a first screen that +// carries the attribute defines it (legendcoveragecheck), and it holds no "--" digraph and no "<" (it lives in an XML comment). +// +// STATE. One process-wide report, written once by main.cpp's amp=/churn= block on the main thread, after the walks were +// joined and before any emitter runs; emitters only read it. The MCP server never runs that block, so it never sets it. +// Gate: test/qchurncheck.sh (the K51 block). + +#include +#include + +#include "prconverge.h" // DiscloseAs — the shared "which spelling does the host accept" enum + +namespace rw +{ + +// The DISCLOSE sink (CONTRIBUTING §3): the roots whose walk could not be read, by cause. +struct HistoryReadReport +{ + enum class DisclosureWhy : std::uint8_t + { + WalkNotRun, // git did not run (not on PATH, not executable, no popen): the root's stream is empty + WalkStopped, // git ran and exited non-zero part-way: the root's stream is a newest-first prefix (possibly empty) + }; + std::uint32_t unreadRoots = 0; + std::uint32_t notRunRoots = 0; + std::uint32_t stoppedRoots = 0; + void disclose( DisclosureWhy why ) noexcept + { + ++unreadRoots; + ++( why == DisclosureWhy::WalkNotRun ? notRunRoots : stoppedRoots ); + } +}; + +inline HistoryReadReport& historyRead() noexcept +{ + static HistoryReadReport report; + return report; +} + +// "" when every walk was read, so a caller that appends unconditionally keeps its exact bytes on the ordinary path. +inline std::string renderHistoryUnread( DiscloseAs as ) +{ + if( historyRead().unreadRoots == 0 ) + { + return {}; + } + if( as == DiscloseAs::XmlAttrs ) + { + return " history_unread=\"1\""; + } + if( as == DiscloseAs::JsonKeys ) + { + return ",\"history_unread\":true"; + } + std::string comment = ""; + return comment; +} + +} // namespace rw diff --git a/src/ingest.cpp b/src/ingest.cpp index 840d28e7f..b1c23fa9c 100644 --- a/src/ingest.cpp +++ b/src/ingest.cpp @@ -58,6 +58,7 @@ #include #include #include +#include // cli-perf-068: the git ignore probe on a worker (ingest_crawl.h IgnoreProbeSpeculation) #include #include #include @@ -315,10 +316,18 @@ IngestResult ingest( const char* rootDir, const std::vector& exclud // #350 layer 3: one memory watch for this ingest, shared by the crawl and the parse pool (memguard.h) memguard::Watch memWatch; - // 1) deterministic crawl -> sorted file list (this list IS result.files / the fileId space) + // cli-perf-068: the git ignore probe runs beside the walk and the cache load, on the previous call's answer, and is + // checked against this call's own answer before the parse pool (ingest_crawl.h IgnoreProbeSpeculation). No sidecar + // under --no-cache: the probe then runs inside the crawl, as it always did. + IgnoreProbeSpeculation ignoreSpeculation( !cacheFile.empty() ); + + // 1) deterministic crawl -> sorted file list (this list IS result.files / the fileId space). A lambda because a lost + // ignore speculation runs it a second time, on git's fresh answer (below, before the parse pool). + const auto crawlInto = [ & ]() { PROFILE_SCOPE_DESCRIBE( "ingest: crawl (collectSources)" ); - auto [ crawledPaths, oversizeSkipped, taxonomySkips ] = collectSources( rootDir, excludeSubstr, maxFileBytes, excludeLabel, respectGitignore, &memWatch ); + auto [ crawledPaths, oversizeSkipped, taxonomySkips ] = collectSources( rootDir, excludeSubstr, maxFileBytes, excludeLabel, respectGitignore, &memWatch, + &ignoreSpeculation ); result.files = std::move( crawledPaths ); // #228: record the root once, for rootRelPath (model.h). A directory crawl joins it onto every path; a // single-file root IS its one path, so the root-relative view anchors at that file's directory instead. @@ -344,7 +353,8 @@ IngestResult ingest( const char* rootDir, const std::vector& exclud DISCLOSE( result.memoryStop, MemoryStop::DisclosureWhy::CrawlOverLimit, "ingest: the memory guard stopped the crawl — files= is what it saw, a floor of the tree" ); } } - } + }; + crawlInto(); // Win 1 (PERF.md P1) — lazy grammar compilation: load the cache FIRST, then compile only the // grammars needed by cache-miss files (new or hash-changed). On a fully-warm zero-change run, @@ -373,10 +383,25 @@ IngestResult ingest( const char* rootDir, const std::vector& exclud HashMap cache = cacheFile.empty() ? HashMap{} : loadCache( std::string( cacheFile ), rootDir, captureValueUses, result.files, cacheStats ); - const long long cacheWriteNs = cacheStats.blobWriteNs; // per-fileId scan arrays (language classify + hash/stat-gate + health slots) — ingest_prewarm.h IngestFileScan scan = makeFileScan( result.files ); + // The ignore speculation is decided HERE: nothing past this point has run on the crawl's file list but the cache load + // and the scan arrays, and both are redone. A lost speculation (the previous call's ignore answer differs from git's + // answer for this call) re-walks on the fresh answer; the crawl's memory stop is re-armed and re-recorded with it. + if( !ignoreSpeculation.crawlStands() ) + { + result.memoryStop = MemoryStop{}; + memWatch.rearmForParse(); // clears the crawl's stop flags: the second walk is measured on its own + crawlInto(); + cacheStats = CacheLoadStats{}; + cache = cacheFile.empty() ? HashMap{} + : loadCache( std::string( cacheFile ), rootDir, captureValueUses, result.files, cacheStats ); + scan = makeFileScan( result.files ); + } + ENSURES( scan.lang.size() == result.files.size(), "the scan arrays describe the crawl the parse pool will read" ); + const long long cacheWriteNs = cacheStats.blobWriteNs; + // lazy tags.scm prewarm: detect the cache-miss set (prefilling scan.hash/stat), launch the async // grammar compiles, and hand the compile/ready state to the parse pool via `prewarm` (ingest_prewarm.h). QueryPrewarm prewarm; diff --git a/src/ingest.h b/src/ingest.h index 162938b60..8e454dc2a 100644 --- a/src/ingest.h +++ b/src/ingest.h @@ -881,4 +881,34 @@ inline bool astQueryNestsTooDeep( std::string_view query ) // local to the re-parsed substring. std::vector collectGatedLocalNames( std::string_view defBytes, std::uint32_t defStartLine, Lang lang ); +// ---- --whereis on other refs: where the TAGS PATH defines one name inside blobs the index never read ---- +// HEAD's rows take kind="def" from the index; a row on another ref comes from a git blob the index never saw. This +// runs the index's own extraction on such a blob — the crawl's extension → grammar rule (with the .h → ObjC content +// reroute), the nesting refusal, the member-macro re-parse, the `#if 0` filter, the language's tags query through +// captureTagsFacts, and the markdown section walk — so a branch definition is labelled by the same captures that +// label HEAD's. `lines` is the 1-based @name line of every definition named `name`, ascending (RawDef::line, which +// is Symbol::line: the line HEAD's relabel keys on). +struct BlobText +{ + std::string_view path; // the tree path: its extension picks the grammar, as in the crawl + std::string_view bytes; // the blob's content +}; +enum class BlobDefsStatus : std::uint8_t +{ + Parsed, // the tags path ran over the whole blob: `lines` is its complete answer for the name + NoGrammar, // no indexed language for this path (the index defines nothing in such a file either) + Unread, // not parsed: binary, over the crawl's default size ceiling, or a nesting refusal (the crawl skips these too) + Failed, // the parser or the tags query could not finish (a null tree, an unavailable query, a partial extract) +}; +struct BlobDefinitions +{ + std::vector lines; + BlobDefsStatus status = BlobDefsStatus::Failed; +}; +// Parallel over hardware_concurrency() workers, biggest blob first (spanTiersOfFiles' pool shape); each blob writes +// only its own slot, so the result is the serial one. The tags queries a batch needs are compiled and installed into +// the process-global query cache BEFORE any worker starts — the single-writer rule ingest() itself follows — so a +// long-lived server must call this under quality::headSnapshotIngestMutex, like every other ingest call site. +std::vector definitionLinesInBlobs( std::span blobs, std::string_view name ); + } // namespace rw diff --git a/src/ingest_astquery.h b/src/ingest_astquery.h index f1c0b8d7b..d3a38808d 100644 --- a/src/ingest_astquery.h +++ b/src/ingest_astquery.h @@ -2143,4 +2143,240 @@ std::vector collectGatedLocalNames( std::string_view defBytes, st return out; } +// ---- definitionLinesInBlobs (ingest.h): the tags path over blobs from other git refs ---- +// +// The DISCLOSE sink of one blob: a degrade sets its slot's status, which is exactly what the caller reads (a blob +// that is not Parsed never confirms a definition, and --whereis says how many such blobs it met). +struct BlobDefsSlot +{ + enum class DisclosureWhy : std::uint8_t + { + NoParser, // ts_parser_new / ts_query_cursor_new gave nothing: the worker parses no blob + GrammarRejected, // the grammar could not be set on the parser (ABI): this blob is not parsed + ParseNull, // tree-sitter returned no tree + ExtractPartial, // the tags pass disclosed a shortfall (ExtractShortfall): its definitions may be incomplete + WorkerThrew, // an exception inside the worker: the slot keeps Failed + }; + BlobDefinitions& slot; + void disclose( DisclosureWhy ) noexcept // every reason records the same fact: no complete answer for this blob + { + slot.lines.clear(); + slot.status = BlobDefsStatus::Failed; + } +}; + +// The worker's non-markdown extraction over one blob, in its order: the .astro frontmatter restriction, the parse, the +// member-macro re-parse when the first parse holds errors, the `#if 0` ranges, the definitions-only tags pass. False when +// the slot is already final (a template-only .astro, or a degrade it discloses). +inline bool tagsDefinitionsOfBlob( std::string_view bytes, const LangEntry& le, TSParser* parser, TSQueryCursor* cursor, MemberMacroReparse& macroWork, + std::vector& defs, BlobDefinitions& slot ) +{ + IncludedRangeGuard rangeGuard; + const AstroFrontmatter astro = restrictAstroToFrontmatter( parser, le, bytes, rangeGuard ); + if( astro == AstroFrontmatter::Unterminated ) + { + DISCLOSE( ( BlobDefsSlot{ slot } ), BlobDefsSlot::DisclosureWhy::ExtractPartial, "whereis: an .astro blob opens a frontmatter fence it never closes — not extracted" ); + return false; + } + if( astro == AstroFrontmatter::None ) + { + slot.status = BlobDefsStatus::Parsed; // a template-only .astro: the index extracts nothing from it either + return false; + } + TreeGuard tree( parseTree( parser, bytes ) ); + if( tree.get() == nullptr ) + { + DISCLOSE( ( BlobDefsSlot{ slot } ), BlobDefsSlot::DisclosureWhy::ParseNull, "whereis: a blob did not parse — its rows confirm no definition" ); + return false; + } + if( ts_node_has_error( ts_tree_root_node( tree.get() ) ) ) // a clean parse has no error bytes: the re-parse never adopts + { + (void)measureHealthAdoptingMemberMacroReparse( parser, le.lang, bytes, tree, macroWork ); // may adopt it, as the worker does + } + const TSNode root = ts_tree_root_node( tree.get() ); + const std::vector ppDead = preprocDeadRangesFor( le, root, bytes ); + std::vector refs; + std::vector binds; + std::vector incs; + ExtractShortfall shortfall; + captureTagsFacts( cursor, le, 0, bytes, root, defs, refs, binds, incs, ppDead, shortfall, TagsCapture::DefinitionsOnly ); + if( shortfall.isShort ) + { + DISCLOSE( ( BlobDefsSlot{ slot } ), BlobDefsSlot::DisclosureWhy::ExtractPartial, "whereis: the tags pass over a blob was partial — its rows confirm no definition" ); + return false; + } + return true; +} + +// One blob through the crawl's language rule and the parse-pool worker's extraction steps, in the worker's order. +// `scan` is a one-slot scan the nesting refusal records into (its --skipped bookkeeping has no reader here: the +// slot's Unread status is this blob's disclosure). +inline void definitionLinesOfOneBlob( const BlobText& blob, std::string_view name, TSParser* parser, TSQueryCursor* cursor, + MemberMacroReparse& macroWork, IngestFileScan& scan, BlobDefinitions& slot ) +{ + const std::string ext = lowerExtensionOf( blob.path ); + const LangEntry* le = lookupLang( ext ); + if( le == nullptr ) + { + slot.status = BlobDefsStatus::NoGrammar; + return; + } + if( blob.bytes.size() > kDefaultMaxFileBytes || looksBinary( blob.bytes ) ) + { + slot.status = BlobDefsStatus::Unread; // the crawl's own ceiling and binary sniff: the index skips these too + return; + } + if( le->ext == ".h" && looksObjC( blob.bytes ) ) + { + if( const LangEntry* objcLe = lookupLang( ".m" ) ) + { + le = objcLe; + } + } + const std::string pathText( blob.path ); + scan.nestRefusedBytes[ 0 ] = 0; + if( refuseNesting( *le, blob.bytes, pathText.c_str(), 0, scan ) ) + { + slot.status = BlobDefsStatus::Unread; + return; + } + if( !prepareParserFor( parser, *le ) ) + { + DISCLOSE( ( BlobDefsSlot{ slot } ), BlobDefsSlot::DisclosureWhy::GrammarRejected, "whereis: a blob's grammar could not be set — its rows confirm no definition" ); + return; + } + std::vector defs; + if( le->lang == Lang::Markdown ) + { + TreeGuard mdTree( parseTree( parser, blob.bytes ) ); + if( mdTree.get() == nullptr ) + { + DISCLOSE( ( BlobDefsSlot{ slot } ), BlobDefsSlot::DisclosureWhy::ParseNull, "whereis: a markdown blob did not parse — its rows confirm no definition" ); + return; + } + std::vector refs; + extractMarkdown( 0, blob.bytes, fs::path( pathText ).stem().string(), ts_tree_root_node( mdTree.get() ), defs, refs ); + } + else if( !tagsDefinitionsOfBlob( blob.bytes, *le, parser, cursor, macroWork, defs, slot ) ) + { + return; // the slot says why (template-only .astro: Parsed with nothing; a degrade: Failed) + } + // The model build's steps that decide which captured definitions become SYMBOLS (ingest.cpp 3a..3a-ter), so a + // blob answers what HEAD's index answers for the same bytes: overlapping patterns collapse to one definition, an + // ObjC @interface declaration shadowed by its same-file @implementation drops, and a field leaves the symbol + // universe (fields are not Symbols, so on HEAD a field line is never a def site either). + dedupRawDefs( defs ); + collapseObjCDeclDefs( defs ); + (void)partitionFieldDefs( defs ); + for( const RawDef& d : defs ) + { + if( d.name == name ) + { + slot.lines.push_back( d.line ); + } + } + std::sort( slot.lines.begin(), slot.lines.end() ); + slot.status = BlobDefsStatus::Parsed; +} + +// The query set, single-threaded and BEFORE any worker: every grammar a blob can reach — its extension's, plus ObjC's for a +// .h (the content reroute is decided inside the worker). compiledQueryFor only ever READS the cache, so a grammar is +// compiled here once and the cache owns it from then on (CompiledQueryCache frees it at teardown). +inline void installBlobQueries( std::span blobs ) +{ + std::vector want; + const auto need = [ & ]( const LangEntry* le ) + { + if( le == nullptr || le->grammar == nullptr || le->querySub.empty() || compiledQueryCache().find( le->grammar() ) != compiledQueryCache().end() ) + { + return; + } + if( std::none_of( want.begin(), want.end(), [ & ]( const LangEntry* w ) { return w->grammar() == le->grammar(); } ) ) + { + want.push_back( le ); + } + }; + for( const BlobText& b : blobs ) + { + const std::string ext = lowerExtensionOf( b.path ); + need( lookupLang( ext ) ); + if( ext == ".h" ) + { + need( lookupLang( ".m" ) ); + } + } + for( const LangEntry* le : want ) + { + if( TSQuery* q = compileQueryStandalone( *le ); q != nullptr ) // a compile error stays absent: captureTagsFacts discloses it + { + compiledQueryCache()[ le->grammar() ] = q; + } + } +} + +// One parse worker: its own parser, query cursor and scratch, claiming blobs in `walkOrder` until none is left. Each blob +// writes only its own slot of `out`. +inline void definitionLinesWorker( std::span blobs, std::string_view name, std::span walkOrder, + std::atomic& nextSlot, std::span out ) noexcept +{ + ParserGuard pg; + TSQueryCursor* cursor = ts_query_cursor_new(); + if( pg.p == nullptr || cursor == nullptr ) + { + for( std::size_t s = nextSlot.fetch_add( 1, std::memory_order_relaxed ); s < blobs.size(); s = nextSlot.fetch_add( 1, std::memory_order_relaxed ) ) + { + DISCLOSE( ( BlobDefsSlot{ out[ walkOrder[ s ] ] } ), BlobDefsSlot::DisclosureWhy::NoParser, "whereis: no tree-sitter parser for a blob worker" ); + } + if( cursor != nullptr ) + { + ts_query_cursor_delete( cursor ); + } + return; + } + MemberMacroReparse macroWork; + IngestFileScan scan = makeFileScan( std::vector{ std::string() } ); + for( std::size_t s = nextSlot.fetch_add( 1, std::memory_order_relaxed ); s < blobs.size(); s = nextSlot.fetch_add( 1, std::memory_order_relaxed ) ) + { + BlobDefinitions& slot = out[ walkOrder[ s ] ]; + try + { + definitionLinesOfOneBlob( blobs[ walkOrder[ s ] ], name, pg.p, cursor, macroWork, scan, slot ); + } + catch( ... ) // a throw escaping a worker thread is std::terminate: the blob degrades instead + { + DISCLOSE( ( BlobDefsSlot{ slot } ), BlobDefsSlot::DisclosureWhy::WorkerThrew, "whereis: a blob worker threw — that blob confirms no definition" ); + } + } + ts_query_cursor_delete( cursor ); +} + +std::vector definitionLinesInBlobs( std::span blobs, std::string_view name ) +{ + std::vector out( blobs.size() ); + if( blobs.empty() ) + { + return out; + } + warmFieldIdTable(); // idempotent; the AST walks behind captureTagsFacts read it lock-free + installBlobQueries( blobs ); + + std::vector walkOrder( blobs.size() ); + std::iota( walkOrder.begin(), walkOrder.end(), std::uint32_t( 0 ) ); + std::stable_sort( walkOrder.begin(), walkOrder.end(), [ & ]( std::uint32_t a, std::uint32_t b ) noexcept + { return blobs[ a ].bytes.size() > blobs[ b ].bytes.size(); } ); // biggest first: no straggler at the tail + const unsigned threadCount = static_cast( std::min( std::max( 1u, std::thread::hardware_concurrency() ), blobs.size() ) ); + std::atomic nextSlot{ 0 }; + std::vector pool; + pool.reserve( threadCount ); + for( unsigned t = 0; t < threadCount; ++t ) + { + pool.emplace_back( [ & ]() noexcept { definitionLinesWorker( blobs, name, walkOrder, nextSlot, out ); } ); + } + for( std::thread& w : pool ) + { + w.join(); + } + return out; +} + } // namespace rw — ingest_astquery.h section of ingest.cpp diff --git a/src/ingest_binds.h b/src/ingest_binds.h index 273a7332f..23963f5a1 100644 --- a/src/ingest_binds.h +++ b/src/ingest_binds.h @@ -1730,7 +1730,7 @@ inline void rubyNoteClassSideDef( TSNode defNode, std::uint32_t fileId, std::str // The bindings for what the `class << self` call `n` declares on the class object's `side` — an accessor, or a // delegation-DSL call (captureRubySingletonAccessors below). An accessor's binding sits at its symbol, -// where captureRubyAttrDefs starts its def (no owner), and carries the side's mark (one a concern's `included do` +// where captureRubyDefs starts its def (no owner), and carries the side's mark (one a concern's `included do` // declares is the includer's); a delegated name has no def, and is read against the class that owns the singleton. inline void rubyNoteSingletonDecl( TSNode n, RubyDefSide side, std::uint32_t fileId, std::string_view src, std::vector& binds ) { @@ -1753,7 +1753,7 @@ inline void rubyNoteSingletonDecl( TSNode n, RubyDefSide side, std::uint32_t fil } // What a `class << self` declares beside its defs is the class object's too (parser version 137): each accessor -// (`attr_accessor :config`, one RubySingletonDef binding per name, where captureRubyAttrDefs starts its def) and each name +// (`attr_accessor :config`, one RubySingletonDef binding per name, where captureRubyDefs starts its def) and each name // the delegation DSL defines (model.h kRubyDelegationCalls — `delegate :reset, to: :instance`: a binding per name against // the owning class; `delegate_missing_to` one with no name, as it answers every name). A file that never writes `<<` is not // walked; the walk is iterative, so a hostile nesting depth costs heap, not stack. diff --git a/src/ingest_cache.h b/src/ingest_cache.h index ed48f68cb..f025d4e9e 100644 --- a/src/ingest_cache.h +++ b/src/ingest_cache.h @@ -312,7 +312,15 @@ constexpr std::uint32_t kCacheVersion = 29; // 29: FE-B (test/receiver // (Py `pkg.mod`, TS `./x`, Rust `crate::a::b`/`mod:x`) — // a target FORMAT change → old caches must be rejected. // 4: Include gained a `bool isAngle` (quote/angle) field -constexpr std::uint32_t kParserVer = 156; // bump on any grammar/.scm/extraction change +constexpr std::uint32_t kParserVer = 158; // bump on any grammar/.scm/extraction change + // 158 = 2026-10-09 (train 26c: PR #339, Rails db/schema.rb columns, merged onto main's 156): + // a Ruby file whose tree holds a `create_table "x", … do |t| … end` call is a rendered + // db/schema.rb BY CONTENT; each `t. "name"` / `t. :name` in the table block + // mints a SymKind::Section def at Lang::Ruby (plus the implicit / `id:` / `primary_key:` + // id rule and `t.timestamps`). DEFINITIONS ONLY: buildGraph's byName skips Section-Ruby, + // so a column takes no call edge. The branch carried 122 -> 126 -> 144 -> 149 across its + // merges with main; 158, not 157: 156's rich file tag is 157. No record layout change + // (kCacheVersion 29, kQSnapCacheScheme 19 stay). // 156 = 2026-10-08 (train 26b: signed lanes merged onto main's 148): cache-key hygiene. // Two merged lanes changed extraction under lane-local numbers, both 145: // honesty-small-068 (Go named-type kinds, `type A = B` aliases) and @@ -2760,7 +2768,12 @@ inline HashMap loadCache( const std::string& path, std:: } fileBuf.resize( std::size_t( spanEnd - spanStart ) ); - if( !preadExact( frame.blob.fd, fileBuf.data(), fileBuf.size(), spanStart ) ) + bool spanRead = false; + { + PROFILE_SCOPE_DESCRIBE( "ingest/loadCache: pread one coalesced span" ); + spanRead = preadExact( frame.blob.fd, fileBuf.data(), fileBuf.size(), spanStart ); + } + if( !spanRead ) { DISCLOSE( Diagnostics::answerUnchanged, "the affected files are reparsed from source: the answer is byte-identical, only slower", "ingest: cache blob read failed mid-load — the unread records are reparsed" ); @@ -2771,7 +2784,12 @@ inline HashMap loadCache( const std::string& path, std:: { const CacheEntry& e = frame.entries[ wanted[i].entryIndex ]; const char* rec = fileBuf.data() + std::size_t( e.recOffset - spanStart ); - if( recordSum32( std::string_view( rec, e.recLength ) ) != e.recSum ) + bool recordSumOk = false; + { + PROFILE_SCOPE_DESCRIBE( "ingest/loadCache: record digest (recordSum32)" ); + recordSumOk = recordSum32( std::string_view( rec, e.recLength ) ) == e.recSum; + } + if( !recordSumOk ) { // A record torn on its own while the table survived: drop THIS file (it reparses) and // keep the rest of the blob. The table is what must be trusted whole, not each record. @@ -2782,7 +2800,12 @@ inline HashMap loadCache( const std::string& path, std:: ByteR r{ rec, rec + e.recLength }; std::string rel; FileFacts ff; - if( !readFileRecord( r, captureValueUses, dictScratch, rel, ff ) ) + bool decoded = false; + { + PROFILE_SCOPE_DESCRIBE( "ingest/loadCache: decode one record (readFileRecord)" ); + decoded = readFileRecord( r, captureValueUses, dictScratch, rel, ff ); + } + if( !decoded ) { continue; // corrupt record → that file reparses (readFileRecord already disclosed) } diff --git a/src/ingest_crawl.h b/src/ingest_crawl.h index d20cc2da4..c7486daff 100644 --- a/src/ingest_crawl.h +++ b/src/ingest_crawl.h @@ -1392,49 +1392,99 @@ bool underGitRoot( const char* rootDir ) } } -GitIgnoreSet collectGitIgnored( const char* rootDir ) +// The probe's PROCESS half, split from its parse so it can run off the main thread (IgnoreProbeSpeculation below): one +// `git ls-files --others --ignored` popen, read whole, classified exactly as collectGitIgnored always classified it — +// a non-zero exit outranks an overflow (the overflow break still reaches pclose, and its status is read first). No +// DISCLOSE and no parse here: the parse below runs on the thread that applies the answer, so the disclosure ledger +// sees exactly the degrades of the answer the crawl used. +struct GitIgnoreProbeRaw { - GitIgnoreSet out; + enum class State : std::uint8_t + { + NotGit, // no `.git` above the root: no git child was started (nongitqmetricscheck) + NotRunnable, // popen failed + Failed, // git exited non-zero + OverCeiling, // more than kMaxIgnoreProbeBytes + Answered, // `bytes` is git's whole -z answer + }; + State state = State::NotGit; + std::string bytes; + bool operator==( const GitIgnoreProbeRaw& ) const = default; +}; + +GitIgnoreProbeRaw runGitIgnoreProbe( const char* rootDir ) +{ + PROFILE_SCOPE_DESCRIBE( "ingest: git ignore probe (the git process; off the main thread when speculated)" ); + GitIgnoreProbeRaw raw; if( !underGitRoot( rootDir ) ) { - return out; + return raw; } const std::string cmd = gitCmd( " -C " ) + shSingleQuote( rootDir == nullptr ? std::string( "." ) : std::string( rootDir ) ) + " -c core.quotepath=false ls-files --others --ignored --exclude-standard --directory -z 2>/dev/null"; std::FILE* pipe = os::popen( cmd.c_str(), "r" ); if( pipe == nullptr ) { - DISCLOSE( out, GitIgnoreSet::DisclosureWhy::GitNotRunnable, "ingest: cannot run git for the ignore probe — full walk" ); - return out; + raw.state = GitIgnoreProbeRaw::State::NotRunnable; + return raw; } - std::string buf; - char chunk[ 8192 ]; - bool overflowed = false; + char chunk[ 8192 ]; + bool overflowed = false; for( std::size_t n = std::fread( chunk, 1, sizeof( chunk ), pipe ); n > 0; n = std::fread( chunk, 1, sizeof( chunk ), pipe ) ) { - if( buf.size() + n > kMaxIgnoreProbeBytes ) + if( raw.bytes.size() + n > kMaxIgnoreProbeBytes ) { overflowed = true; break; } - buf.append( chunk, n ); + raw.bytes.append( chunk, n ); } const int rc = os::pclose( pipe ); - if( rc != 0 ) + raw.state = rc != 0 ? GitIgnoreProbeRaw::State::Failed : overflowed ? GitIgnoreProbeRaw::State::OverCeiling : GitIgnoreProbeRaw::State::Answered; + if( raw.state != GitIgnoreProbeRaw::State::Answered ) { - // Not a git work tree, or no git binary — the DESIGNED degrade: the fallback (a full walk, no - // ignore-set pruning) is safe either way. `available` is already false here (its default), so this - // changes no output — it disclose()s the fact for the self-check ledger instead of leaving the - // degrade to a bare `return`, matching the GitNotRunnable/ProbeOverCeiling sites just above/below. - DISCLOSE( out, GitIgnoreSet::DisclosureWhy::ProbeFailed, "ingest: the git ignore probe exited non-zero — full walk" ); - return out; + raw.bytes.clear(); // only an answer carries bytes, so two probes that failed the same way compare equal } - if( overflowed ) + ENSURES( raw.state == GitIgnoreProbeRaw::State::Answered || raw.bytes.empty(), "a probe that did not answer carries no bytes" ); + return raw; +} + +// The probe's PARSE half: every degrade disclosed exactly where collectGitIgnored disclosed it, then the -z entries. +GitIgnoreSet parseGitIgnoreProbe( const GitIgnoreProbeRaw& raw ) +{ + GitIgnoreSet out; + switch( raw.state ) { - DISCLOSE( out, GitIgnoreSet::DisclosureWhy::ProbeOverCeiling, "ingest: git ignore probe exceeded its byte ceiling — full walk" ); - return out; + case GitIgnoreProbeRaw::State::NotGit: + { + return out; + } + case GitIgnoreProbeRaw::State::NotRunnable: + { + DISCLOSE( out, GitIgnoreSet::DisclosureWhy::GitNotRunnable, "ingest: cannot run git for the ignore probe — full walk" ); + return out; + } + case GitIgnoreProbeRaw::State::Failed: + { + // Not a git work tree, or no git binary — the DESIGNED degrade: the fallback (a full walk, no + // ignore-set pruning) is safe either way. `available` is already false here (its default), so this + // changes no output — it disclose()s the fact for the self-check ledger instead of leaving the + // degrade to a bare `return`, matching the GitNotRunnable/ProbeOverCeiling sites just above/below. + DISCLOSE( out, GitIgnoreSet::DisclosureWhy::ProbeFailed, "ingest: the git ignore probe exited non-zero — full walk" ); + return out; + } + case GitIgnoreProbeRaw::State::OverCeiling: + { + DISCLOSE( out, GitIgnoreSet::DisclosureWhy::ProbeOverCeiling, "ingest: git ignore probe exceeded its byte ceiling — full walk" ); + return out; + } + case GitIgnoreProbeRaw::State::Answered: + { + break; + } } + const std::string& buf = raw.bytes; for( std::size_t i = 0; i < buf.size(); ) { const std::size_t nul = buf.find( '\0', i ); @@ -1473,6 +1523,119 @@ GitIgnoreSet collectGitIgnored( const char* rootDir ) return out; } +GitIgnoreSet collectGitIgnored( const char* rootDir ) +{ + return parseGitIgnoreProbe( runGitIgnoreProbe( rootDir ) ); +} + +// ── the ignore probe, SPECULATED (cli-perf-068) ─────────────────────────────────────────────────────── +// +// The probe is one git process on every call — ~60 ms of a ~400 ms warm call on this repository, measured, and most of +// it git walking the same tree the crawl is about to walk. Its answer rarely changes between two calls, but WHAT can +// change it is wide: any .gitignore in the tree, .git/info/exclude, the index, core.excludesFile and the config that +// names it, and the untracked files themselves (a new `x.log` under an existing `*.log` rule changes the answer while +// no ignore FILE changed). A cache keyed on the files that hold rules would miss that last one; this does not key on +// anything. It SPECULATES: the crawl walks with the answer the previous call got for this root (a sidecar under the +// cache dir) while git computes this call's answer on a worker, and ingest() compares the two — byte for byte, the +// raw -z stream — before anything downstream of the crawl has run (crawlStands). Equal: the crawl that ran IS the +// crawl the fresh answer gives, so nothing is redone. Different: the crawl and the cache load are run again on the +// fresh answer, and the sidecar is replaced. Every answer is therefore git's own answer for THIS call, exactly as +// before; what is saved is the wait — git runs beside the walk and the cache load instead of before them. +// +// Spawn conditions are the ones the probe always had: it starts at the same point of collectSources (a directory root +// whose walk opened), only under a `.git` (no child at all otherwise), and not under --no-ignore. With the cache off +// (--no-cache) there is no sidecar, so the probe runs on the calling thread, as before. One per ingest() call. +class IgnoreProbeSpeculation +{ +public: + explicit IgnoreProbeSpeculation( bool sidecarEnabled ) noexcept : sidecarEnabled_( sidecarEnabled ) {} + + // collectSources' one consult, at the point the probe always ran. On the redo pass (after a lost speculation) it + // hands back the fresh answer the join already holds. + GitIgnoreSet setFor( const char* rootDir, const std::string& rootReal ) + { + if( fresh_ ) + { + return parseGitIgnoreProbe( *fresh_ ); + } + if( !sidecarEnabled_ || !underGitRoot( rootDir ) ) + { + fresh_ = runGitIgnoreProbe( rootDir ); // no sidecar to speculate from, or no git child to overlap: as before + return parseGitIgnoreProbe( *fresh_ ); + } + // Named by the house root key (quality::cacheRootKeyHex, realpath-normalized), so the eviction sweep pins this + // file with every other family of the root (test/evictioncheck.sh (k)/(l)); the header still names the root. + sidecarPath_ = quality::resolveCacheBlobPath( quality::cacheDirLadder(), "ripwire-ignoreprobe-" + quality::cacheRootKeyHex( rootDir ) + ".bin" ); + sidecarHeader_ = std::string( kSidecarMagic ) + rootReal + '\0'; + pending_ = std::async( std::launch::async, [ root = std::string( rootDir ) ] { return runGitIgnoreProbe( root.c_str() ); } ); + speculation_ = loadSidecar(); + if( !speculation_ ) + { + return parseGitIgnoreProbe( join() ); // nothing to speculate from (first call on this root): wait, as before + } + return parseGitIgnoreProbe( *speculation_ ); + } + + // ingest()'s check, before the parse pool: true when the crawl that ran used this call's own git answer — the fresh + // answer itself, or a speculation byte-identical to it. False means the caller redoes the crawl (setFor now returns + // the fresh answer). Publishes the fresh answer for the next call whenever it differs from what was speculated. + bool crawlStands() + { + PROFILE_SCOPE_DESCRIBE( "ingest: ignore probe join (wait for git, compare with the speculation)" ); + if( pending_.valid() ) + { + (void)join(); + } + if( !speculation_ ) + { + return true; // the crawl waited for the fresh answer, or no probe ran at all + } + const bool stands = *speculation_ == *fresh_; + speculation_.reset(); // decided: a redo pass reads fresh_, and a second call to this is a no-op + return stands; + } + +private: + static constexpr std::string_view kSidecarMagic = "ripwire-ignore-speculation/1\n"; + + // The previous call's answer for this root, or nothing. The header names the root in full, so two roots whose names + // hash alike never read each other's answer; anything unreadable or misshapen is simply no speculation. + std::optional loadSidecar() const + { + std::optional blob = pathguard::readRegularFileNoFollow( sidecarPath_ ); + if( !blob || blob->size() > sidecarHeader_.size() + kMaxIgnoreProbeBytes || !blob->starts_with( sidecarHeader_ ) ) + { + return std::nullopt; + } + return GitIgnoreProbeRaw{ GitIgnoreProbeRaw::State::Answered, blob->substr( sidecarHeader_.size() ) }; + } + + const GitIgnoreProbeRaw& join() + { + fresh_ = pending_.get(); // a throw on the worker (allocation) surfaces here, where the synchronous probe threw + if( fresh_->state == GitIgnoreProbeRaw::State::Answered && ( !speculation_ || !( *speculation_ == *fresh_ ) ) ) + { + if( !quality::atomicWriteFile( sidecarPath_, sidecarHeader_ + fresh_->bytes ) ) + { + DISCLOSE( Diagnostics::answerUnchanged, "this call used git's own ignore answer; only the next call's speculation is not saved", + "ingest: the ignore-probe sidecar could not be written — the next call waits for git, as before" ); + } + } + else if( fresh_->state != GitIgnoreProbeRaw::State::Answered && speculation_ ) + { + os::unlink( sidecarPath_.c_str() ); // git stopped answering here: a stale answer must not be speculated on again + } + return *fresh_; + } + + bool sidecarEnabled_ = false; + std::string sidecarPath_; + std::string sidecarHeader_; + std::future pending_; + std::optional speculation_; + std::optional fresh_; +}; + // Sorted-vector membership. A binary search over a contiguous, already-sorted vector beats a hash map at // these sizes and costs no allocation, and — unlike a HashMap — has no iteration order that could reach // output (the G2 container rule's standing caveat). @@ -1503,14 +1666,17 @@ bool pathInIgnoreSet( const std::vector& sorted, std::string_view r // §N6-C — the probe AND the mode it implies, as one decision, so collectSources reads the answer instead // of computing it. One fork per root, paid only for a directory root the walk actually opened: the probe // is worthless on a single-file root and must not be charged to a root the walk already refused. -GitIgnoreSet probeIgnoreSet( const char* rootDir, bool respectGitignore, IgnoreMode& modeOut ) +// `speculation` (ingest()'s, see IgnoreProbeSpeculation) lets the git process run beside the walk; without one the probe +// runs here, on this thread, as it always did. +GitIgnoreSet probeIgnoreSet( const char* rootDir, bool respectGitignore, IgnoreMode& modeOut, IgnoreProbeSpeculation* speculation, + const std::string& rootReal ) { if( !respectGitignore ) { return {}; // modeOut was already set to Off by the caller, before any early return could skip it } PROFILE_SCOPE_DESCRIBE( "ingest: crawl (git ignore probe)" ); - GitIgnoreSet set = collectGitIgnored( rootDir ); + GitIgnoreSet set = speculation != nullptr ? speculation->setFor( rootDir, rootReal ) : collectGitIgnored( rootDir ); modeOut = set.available ? IgnoreMode::Git : set.rootIgnored ? IgnoreMode::RootIgnored : IgnoreMode::Unavailable; return set; } @@ -1567,7 +1733,7 @@ struct CrawlResult // finished walk's list is, so a partial corpus is still a deterministic one; ingest() discloses the stop. CrawlResult collectSources( const char* rootDir, const std::vector& excludeSubstr, std::size_t maxFileBytes, std::string_view excludeLabel = {}, bool respectGitignore = true, - memguard::Watch* memWatch = nullptr ) + memguard::Watch* memWatch = nullptr, IgnoreProbeSpeculation* ignoreSpeculation = nullptr ) { std::vector out; std::vector skipped; @@ -1625,7 +1791,7 @@ CrawlResult collectSources( const char* rootDir, const std::vector& return { std::move( out ), std::move( skipped ), std::move( skips ) }; } - const GitIgnoreSet ignoreSet = probeIgnoreSet( rootDir, respectGitignore, skips.ignoreMode ); // §N6-C + const GitIgnoreSet ignoreSet = probeIgnoreSet( rootDir, respectGitignore, skips.ignoreMode, ignoreSpeculation, rootReal ); // §N6-C const fs::recursive_directory_iterator end; { diff --git a/src/ingest_docpass.h b/src/ingest_docpass.h index 0c98717d0..893c3cd39 100644 --- a/src/ingest_docpass.h +++ b/src/ingest_docpass.h @@ -64,11 +64,17 @@ inline void publishDocBridgeBlob( const std::string& bridgeBlobPath, std::uint32 // cacheEnabled=false (--no-cache) bypasses the sidecar entirely. tmpKey keeps concurrent workers' // unpublished temp files distinct; the publish itself is a whole-file rename, so a concurrent // reader sees every byte or none. +// A bridge doc whose bytes have no cached extraction asks docparse::markitdownMayRun() before the popen: false means +// the shell could not start markitdown, so the answer is "" without the popen — what the popen answered (rc 127), +// minus one shell start per doc per run. The cache is still consulted first, so a blob extracted while markitdown +// was installed keeps answering after it is removed, exactly as before. inline std::string docTextViaBridgeCache( const std::string& path, const std::string& ext, bool cacheEnabled, std::uint32_t tmpKey ) { + EXPECTS( !path.empty(), "doc post-pass: every crawled doc path is a non-empty spelling" ); std::string text; std::string bridgeBlobPath; - if( cacheEnabled && docparse::docKindOf( ext ) == docparse::DocKind::Markitdown ) + const bool viaBridge = docparse::docKindOf( ext ) == docparse::DocKind::Markitdown; + if( cacheEnabled && viaBridge ) { if( const std::optional docBytes = docparse::detail::readWholeFile( path ) ) { @@ -79,6 +85,10 @@ inline std::string docTextViaBridgeCache( const std::string& path, const std::st text = docparse::detail::readWholeFile( bridgeBlobPath ).value_or( std::string() ); // miss ⇒ text stays empty } } + if( text.empty() && viaBridge && !docparse::markitdownMayRun() ) + { + return text; // no cached extraction, and no markitdown the shell could start: the popen's own answer, "" + } if( text.empty() ) { text = docparse::parseDocFile( path, ext ); diff --git a/src/ingest_importcap.h b/src/ingest_importcap.h index ffe957316..b0cacc31f 100644 --- a/src/ingest_importcap.h +++ b/src/ingest_importcap.h @@ -25,26 +25,35 @@ // // THE VOCABULARY IS A FAMILY, AND ONLY ITS FIRST MEMBER IS SPENT. `@import.path` is the only capture name // in use, because C-family imports have no other part: `#include` names a path and nothing else — no -// alias to record, no list of imported names. The other two names the issue sketches are reserved BY NAME -// so the next language lands on them rather than inventing a spelling: `@import.alias` is the local name an -// import renames a target to (`import a.b as c`, `use Foo\Bar as Baz`, `using X = Foo.Bar`), and +// alias to record, no list of imported names — and because Go's alias/dot/blank spellings ride inside the +// declaration clause its Include has always carried (see DepDialect::Go below), not in a field of their own. +// The other two names the issue sketches are reserved BY NAME so the next language lands on them rather +// than inventing a spelling: `@import.alias` is the local name an import renames a target to (`import a.b as c`, `use Foo\Bar as Baz`, `using X = Foo.Bar`), and // `@import.names` is a group of targets ONE directive names (`use Foo\{A, B}`, `alias MyApp.{Bar, Baz}`), // which is why both of those are emitted by the C++ side today through a per-directive helper rather than // by one capture — a capture names one node, and a group is many. No enum value reserves them: an unused // arm that no query can reach is a lie about what the vocabulary supports. // +// SECOND MEMBER: DepDialect::Web (TypeScript, TSX, JavaScript, the .astro frontmatter that rides TypeScript). +// Its three spellings are three patterns on the SAME capture name — `import … from 'x'`, a re-export +// `export … from 'x'`, and the CommonJS / dynamic call `require('x')` / `import('x')` — and one normaliser. +// Neither `@import.alias` nor `@import.names` is used for it, for the same reason C-family has none: the +// Include record a TS/JS import produces names a MODULE and nothing else. The bound names (`import { a as b }`) +// are a different record — captureJsImportFacts' RawBinds, in src/ingest_jsimports.h — which this slice does +// not touch, so there is no current JS/TS semantics for those two names to carry yet. +// // ORDERING. ts_query_cursor_next_match yields matches by the match's start byte, so a capture arrives in // SOURCE order and the emitted Includes do too — the same order captureIncludes' walk produced, because // emitCapturedImport runs straight from the capture dispatch with nothing staging it in between. The // import-role use-site refs move into the tags pass's own window, where orderReferences re-sorts them by // (startByte, name, role, isInherit) — a total key, so the move cannot reorder a published row. // -// WHAT THE WALK STILL OWNS, AND DOES NOT TRAVEL WITH THE MOVE. Three things captureIncludes computed from +// WHAT THE WALK USED TO OWN, AND DOES NOT TRAVEL WITH THE MOVE. Three things captureIncludes computed from // its own frame are re-derived here rather than dropped: the import-container nesting bound -// (importContainerDepth + the same DISCLOSE, below), the lazy bit a TS/JS require needs from `insideFn`, -// and Ruby's innermost-open index for the constant-receiver dedupe. The first is recovered now because it -// is one parent hop per import; the other two are why a language still on the walk cannot move until the -// emission can re-derive them from ancestry too — the tags pass never sees a walk frame. +// (importContainerReach + the same DISCLOSE, below), the lazy bit a TS/JS require needs from `insideFn` +// (the same ancestor loop, one more bit), and Ruby's innermost-open index for the constant-receiver dedupe. +// The first two are recovered because each is one parent hop per import; the third is why Ruby cannot move +// until the emission can re-derive it from ancestry too — the tags pass never sees a walk frame. #include "depdialect.h" // DepDialect + dependencyDialect — the unit one normaliser is written per @@ -54,8 +63,16 @@ namespace rw namespace { -// The import-container REACH of a captured directive, and its DEPTH — the walk's own two rules, -// re-derived from the captured node because the walk no longer runs for this language. +// What importContainerReach reads off one directive's ancestry. +struct ImportReach +{ + std::uint16_t depth = 0; // containers above the directive, stopped at kMaxImportContainerDepth + 1 + bool reachable = true; // every node between the file root and the directive is an import container + bool insideFn = false; // some such container is a function body (kJsFunctionContainers): TS/JS lazy bit +}; + +// The import-container REACH of a captured directive, its DEPTH and its function-body bit — the walk's own +// rules, re-derived from the captured node because the walk no longer runs for this language. // // REACH. captureIncludes seeded the frame stack with root's DIRECT children and then entered only // `isImportContainer` nodes, so a directive was reachable exactly when every node between the file root @@ -69,29 +86,57 @@ namespace // So reach is restored here and the widening is left as its own follow-up. // // The file root is deliberately NOT tested: its children are what the walk always started from, so -// `isImportContainer(root)` being false must not reject a file-scope include. The loop stops when -// `p`'s own parent is null, which is the root. -std::uint16_t importContainerReach( TSNode directive, Lang lang, bool& reachable ) noexcept +// `isImportContainer(root)` being false must not reject a file-scope include. The ancestors read are the +// nodes STRICTLY BETWEEN the root and the directive. +// +// ONE DESCENT FROM THE ROOT, not an upward walk. tree-sitter nodes hold no parent pointer, so every +// ts_node_parent is itself a descent from the root and walking k ancestors up costs k of them (the precedent +// and its measurement: ingest_names.h::enclosingFunctionScope — 600 nested blocks, 19.7 s upward against +// 9.4 s down). The first version of this function walked up, bounded to kMaxImportContainerDepth hops, and +// that bound capped the HOPS, not the cost: a TS/JS file of 2 000 nested `foo( "x", foo( "x", … ) )` calls +// (a hyperscript tree; every one matches the call pattern) took 152 s against 0.5 s for the walk it replaced. +// Descending once is O(depth) per directive, and what the upward loop answered is read off the same descent: +// `run` is how many container ancestors sit directly above the directive (the loop's reach before it met a +// non-container or the bound), `total` is how many ancestors there are. Reachable means the run is all of them. +// A run PAST the bound is reported by `depth` (clamped one past it), which the caller tests BEFORE `reachable`: the +// upward loop stopped at the bound without ever seeing a non-container above it, so a too-deep import is announced +// whether or not the walk would have entered it — the one LOUDER disclosure difference test/importcapcheck.sh pins. +ImportReach importContainerReach( TSNode directive, Lang lang ) noexcept { - std::uint16_t depth = 0; - reachable = true; - for( TSNode p = ts_node_parent( directive ); !ts_node_is_null( p ); p = ts_node_parent( p ) ) + const bool directiveIsNode = !ts_node_is_null( directive ); // hoisted: a promise holds no call (selfcheckcheck C) + EXPECTS( directiveIsNode, "the directive is a node of the parsed tree: importDirectiveOf never returns null for a captured specifier" ); + std::uint32_t total = 0; + std::uint32_t run = 0; // container ancestors directly above the directive (reset by a non-container) + bool fnInRun = false; // some container in `run` is a function body + TSNode n = ts_node_child_with_descendant( ts_tree_root_node( directive.tree ), directive ); + for( ; !ts_node_is_null( n ) && !ts_node_eq( n, directive ); n = ts_node_child_with_descendant( n, directive ) ) { - if( ts_node_is_null( ts_node_parent( p ) ) ) + const char* t = ts_node_type( n ); + ++total; + if( isImportContainer( lang, t ) ) { - break; // p is the file root: reached unconditionally + ++run; + // The walk's `childInsideFn = frame.insideFn || isFunctionLike( lang, t )` is sticky downward, so the + // directive is inside a function exactly when ANY container above it is a function-body kind. + fnInRun = fnInRun || isFunctionLike( lang, t ); } - if( !isImportContainer( lang, ts_node_type( p ) ) ) + else { - reachable = false; // the walk would not have entered this node - break; + run = 0; + fnInRun = false; } - if( ++depth > kMaxImportContainerDepth ) - { - break; // STOPS AT THE BOUND: past it the answer cannot change, and - } // a hostile file must not buy an unbounded parent walk } - return depth; + ImportReach reach; + if( ts_node_is_null( n ) ) + { + reach.reachable = false; // the descent lost the directive: read as "the walk would not have entered", a floor + return reach; + } + reach.depth = static_cast( std::min( run, kMaxImportContainerDepth + 1u ) ); + reach.reachable = ( run == total ); + reach.insideFn = fnInRun; + ENSURES( reach.depth <= kMaxImportContainerDepth + 1, "the depth is clamped one past the bound" ); + return reach; } // One captured `@import.path`, normalised. `text` empty is the "not an import after all" signal and the @@ -101,6 +146,7 @@ struct ImportSpec { std::string text; bool isAngle = false; // CFamily: is external (unresolvable without a build system), "x.h" is not + bool lazyInClosure = false; // Web: this is a require()/import() CALL, so it is Include::isLazy iff it sits in a function body }; // ── DepDialect::CFamily ─────────────────────────────────────────────────────────────────────────────── @@ -131,8 +177,58 @@ ImportSpec normaliseCFamilyImport( TSNode directive, std::string_view raw, std:: return out; } +// ── DepDialect::Web ────────────────────────────────────────────────────────────────────────────────── +// TypeScript, TSX, JavaScript and the .astro frontmatter. Two directive shapes reach this normaliser, told +// apart by the DIRECTIVE node (the captured string's statement, or its call): +// import_statement / export_statement — `import … from 'x'`, `import 'x'`, `import type …`, and the re-exports +// `export * from`, `export * as n from`, `export { a as b } from`, `export type { T } from`. The grammar +// gives all of them the same `source:` string, which is the whole capture; every other export_statement +// has no `source:` and is not matched by the query at all. +// call_expression — the CommonJS `require('x')` and the dynamic `import('x')`. A query cannot say "the +// callee's TEXT is require" (tags-pass predicates never run), so the three guards that keep an ordinary +// call out of the dependency graph stay C++: jsModuleLoadTarget (src/ingest_relations.h), UNCHANGED by +// this round and shared with captureJsImportFacts, reads the callee text, the argument count and the +// string. The query only says where a candidate call is; the guards say whether it is a module load. +// Whether the hit is LAZY (written inside a function body) is the ancestry's answer, not the text's: this +// arm only says the directive is a call (`lazyInClosure`), and emitCapturedImport ANDs it with the walk up. +// `import x = require('y')` (TS) is `import_require_clause` with its own `source:` and matches no pattern — +// the extractor never read it either, so it is still not an edge: a disclosed floor, unchanged by the round. +ImportSpec normaliseWebImport( TSNode directive, std::string_view raw, std::string_view src ) +{ + ImportSpec out; + if( kindIs( ts_node_type( directive ), "call_expression" ) ) + { + out.text = jsModuleLoadTarget( directive, src ); + out.lazyInClosure = !out.text.empty(); // kParserVer 72: a hit found inside a function body is LAZY (the caller knows where it sits) + return out; + } + out.text = std::string( pattern::stripQuotePair( raw ) ); // the captured node is a `string`: strip the one quote pair + return out; +} + +// ── DepDialect::Go ─────────────────────────────────────────────────────────────────────────────────── +// A Go import is ONE `import_declaration` — `import "fmt"`, `import f "fmt"` (aliased), `import . "x"` (dot), +// `import _ "x"` (blank), cgo's `import "C"`, or a parenthesised group of any of those — and the Include +// target it has always carried is the declaration's own CLAUSE (importClauseTarget, shared with the one +// language that still reads it from the walk), not a per-spec specifier. So the captured node is the +// declaration itself (queries/go/tags.scm: `(import_declaration) @import.path`) and the normaliser is the +// clause reader, nothing more. A GROUPED import therefore stays ONE record whose text is the whole group cut +// at 96 bytes, exactly as before: resolve.h's resolveGoImport documents that ("grouped imports collapse to +// one node upstream and are left unresolved"), and splitting a group into one record per spec would add +// edges — a behaviour change, so its own slice. The alias / dot / blank spellings are the same: they ride +// INSIDE the clause text (`f "fmt"`), which is why `@import.alias` and `@import.names` are NOT used here — +// today's Go semantics have no separate alias or names field on an Include. (The alias a Go import binds is +// a different record altogether, the ModuleAlias RawBind captureGoImportFacts writes; it is not a dependency +// edge and is left where it is.) +ImportSpec normaliseGoImport( TSNode declaration, std::string_view src ) +{ + ImportSpec out; + out.text = importClauseTarget( declaration, src ); + return out; +} + // ── the dispatch: ONE normaliser per DepDialect ────────────────────────────────────────────────────── -// CFamily is the only dialect that has moved over so far; every other arm returns an empty ImportSpec, +// CFamily, Web and Go are the dialects that have moved over so far; every other arm returns an empty ImportSpec, // which is what makes the capture INERT rather than wrong on a language whose tags.scm has not adopted // `@import.path` yet. Each `case` is deleted as its language moves, and the compiler then names every // other dialect still to do — the same "one language at a time" landing the issue asks for. @@ -141,10 +237,46 @@ ImportSpec normaliseImportSpecifier( DepDialect dialect, TSNode directive, std:: switch( dialect ) { case DepDialect::CFamily: return normaliseCFamilyImport( directive, raw, src ); - default: return {}; // no language but C-family captures @import.path yet + case DepDialect::Web: return normaliseWebImport( directive, raw, src ); + case DepDialect::Go: return normaliseGoImport( directive, src ); + default: return {}; // no other language captures @import.path yet } } +// True for a dialect whose directives come from `@import.path` and NOT from captureIncludes' walk. The one +// place the walk is switched off (captureSideFacts) asks this, so a dialect cannot be moved in the normaliser +// above and still be walked a second time, or walked-off without a normaliser. +inline bool importsFromCapture( DepDialect dialect ) noexcept +{ + switch( dialect ) // one case per dialect normaliseImportSpecifier has an arm for + { + case DepDialect::CFamily: + case DepDialect::Web: + case DepDialect::Go: + return true; + default: + return false; + } +} + +// The DIRECTIVE a captured specifier belongs to — the node both records are SITED at. For a C include or a +// JS import/re-export it is the specifier's parent; for a call it is the call, one hop further, because the +// specifier sits in the call's `arguments`. +TSNode importDirectiveOf( DepDialect dialect, TSNode pathNode ) noexcept +{ + if( dialect == DepDialect::Go ) + { + return pathNode; // the Go capture IS the whole import_declaration + } + const TSNode parent = ts_node_parent( pathNode ); + const bool hasParent = !ts_node_is_null( parent ); + ASSUME( hasParent, "a captured specifier is a string or path token, never the file root" ); + const TSNode directive = kindIs( ts_node_type( parent ), "arguments" ) ? ts_node_parent( parent ) : parent; + const bool hasDirective = !ts_node_is_null( directive ); + ENSURES( hasDirective, "an `arguments` list always belongs to a call" ); + return directive; +} + // One captured `@import.path` → the Include record plus its ABS-3 import-role use-site ref, which is // exactly the pair captureIncludes' `emitDirective` lambda minted for a C-family directive. // @@ -178,23 +310,28 @@ ImportSpec normaliseImportSpecifier( DepDialect dialect, TSNode directive, std:: void emitCapturedImport( TSNode pathNode, std::uint32_t fileId, Lang lang, std::string_view src, std::vector& includes, std::vector& refs, ExtractShortfall& shortfall ) { - const TSNode directive = ts_node_parent( pathNode ); - bool reachable = false; - if( importContainerReach( directive, lang, reachable ) > kMaxImportContainerDepth ) + const TSNode directive = importDirectiveOf( dependencyDialect( lang ), pathNode ); + // TEXT FIRST, ancestry second. The Web call pattern matches every bare `f( "s" … )`, so most captures here are + // not imports at all, and the reach read below is a descent from the root — paying it to learn that `t( "key" )` + // is not a dependency is the cost that made a 2 000-deep hyperscript tree 300x slower. Dropping a non-import + // BEFORE the bound test also means a `#pragma once` / `foo( "x" )` under a too-deep nest no longer announces + // "an import was cut" when none was (the walk announced on any too-deep container, import or not). + const ImportSpec spec = normaliseImportSpecifier( dependencyDialect( lang ), directive, nodeTextOf( pathNode, src ), src ); + if( spec.text.empty() ) + { + return; + } + const ImportReach reach = importContainerReach( directive, lang ); + if( reach.depth > kMaxImportContainerDepth ) { DISCLOSE( shortfall, ExtractShortfall::DisclosureWhy::ImportNestingTooDeep, "ingest: import-container nesting past the depth bound — deeper imports not captured" ); return; // the same degrade the walk performed: not captured, file still indexed } - if( !reachable ) + if( !reach.reachable ) { return; // a non-container ancestor the walk would never have entered — see importContainerReach } - const ImportSpec spec = normaliseImportSpecifier( dependencyDialect( lang ), directive, nodeTextOf( pathNode, src ), src ); - if( spec.text.empty() ) - { - return; - } if( std::string name = importName( spec.text ); !name.empty() ) { RawRef r; @@ -205,7 +342,7 @@ void emitCapturedImport( TSNode pathNode, std::uint32_t fileId, Lang lang, std:: r.name = std::move( name ); refs.push_back( std::move( r ) ); } - includes.push_back( { fileId, spec.isAngle, false, false, ts_node_start_byte( directive ), false, spec.text } ); + includes.push_back( { fileId, spec.isAngle, spec.lazyInClosure && reach.insideFn, false, ts_node_start_byte( directive ), false, spec.text } ); } } // namespace diff --git a/src/ingest_metrics.h b/src/ingest_metrics.h index ba093e357..23f1b685b 100644 --- a/src/ingest_metrics.h +++ b/src/ingest_metrics.h @@ -1213,7 +1213,8 @@ struct Complexity { std::uint32_t cx; std::uint32_t ccx; std::uint32_t maxNest; // per node and gets locals=0, which the caller (this file, RawDef→Symbol) leaves at 0 and serialize.h // never emits (absent, not a bare "0" — see localsCountedLang's own comment). inline Complexity complexityOf( TSNode root, std::string_view src, Lang lang ) // one fused DFS → cx, ccx, maxNest, locals, ppAlt, the nesting profile, AND ev -{ // A4-F25: NOT noexcept — cc_walk (and kids here) allocate +{ + PROFILE_SCOPE_DESCRIBE( "ingest/metrics: complexityOf one function (fused DFS)" ); // A4-F25: NOT noexcept — cc_walk (and kids here) allocate CcAccum acc; const bool countLocals = localsCountedLang( lang ); // essential complexity rides the SAME walk (zero new tree-sitter queries). ONE arena/label/pending set diff --git a/src/ingest_names.h b/src/ingest_names.h index 0f3d19b3d..d466f38ca 100644 --- a/src/ingest_names.h +++ b/src/ingest_names.h @@ -1163,9 +1163,18 @@ inline void captureRubyAttrDefsCall( TSNode n, std::uint32_t fileId, std::string } ); } -inline void captureRubyAttrDefs( TSNode root, std::uint32_t fileId, std::string_view src, std::vector& defs ) -{ - if( src.find( "attr" ) == std::string_view::npos ) // file signal: every family name contains "attr" +// ONE pre-order walk of the whole tree dispatching BOTH Ruby def-side lanes — the class-level +// attribute DSL and the rendered-schema column capture — from the same explicit stack (the duplication +// the maintainer's quality-delta flagged: two byte-identical walkers). The per-file signals (contains +// "attr" / contains "create_table") decide which dispatch fires before the walk starts, exactly like +// each lane's former early-out. Everything else in each lane is unchanged. +// The schema table helper is defined with its lane below (this walker lives with the attr lane). +inline void rubySchemaTableCall( TSNode n, std::uint32_t fileId, std::string_view src, std::vector& defs ); +inline void captureRubyDefs( TSNode root, std::uint32_t fileId, std::string_view src, std::vector& defs ) +{ + const bool wantAttr = src.find( "attr" ) != std::string_view::npos; // every family name contains "attr" + const bool wantSchema = src.find( "create_table" ) != std::string_view::npos; // the schema gate name must appear + if( !wantAttr && !wantSchema ) { return; } @@ -1185,10 +1194,17 @@ inline void captureRubyAttrDefs( TSNode root, std::uint32_t fileId, std::string_ stack.pop_back(); if( kindIs( ts_node_type( n ), "call" ) ) { - const std::string_view fam = rubyNamedDirective( n, src, kRubyAttrFamilyNames ); - if( !fam.empty() && rubyAttrAtClassBodyLevel( n, src ) ) + if( wantAttr ) + { + const std::string_view fam = rubyNamedDirective( n, src, kRubyAttrFamilyNames ); + if( !fam.empty() && rubyAttrAtClassBodyLevel( n, src ) ) + { + captureRubyAttrDefsCall( n, fileId, src, fam, defs ); + } + } + if( wantSchema && fieldIdentifierText( n, NodeField::Method, src ) == "create_table" ) { - captureRubyAttrDefsCall( n, fileId, src, fam, defs ); + rubySchemaTableCall( n, fileId, src, defs ); } } collectChildren( n, cursor.cur, kids ); @@ -1199,6 +1215,315 @@ inline void captureRubyAttrDefs( TSNode root, std::uint32_t fileId, std::string_ } } + +// ─── the Rails schema capture (test/rubyschemacheck.sh): RUBY db/schema.rb COLUMNS ARE Section DEFS ─── +// A schema file is recognized BY CONTENT, never by path: any Ruby file whose tree holds a +// `create_table "x", … do |t| … end` call — the call's `block:` field is a do_block whose `parameters:` +// field is a block_parameters carrying ONE bare identifier — is a rendered-schema document (the block +// is the create_table call's OWN do_block, found by named-child kind because the shared NodeField set +// has no Block enumerator). Path plays no part: a rendered dump sits at file level or under +// `ActiveRecord::Schema[].define … do`. WHAT A WALK OF THE FILE IS NOT: a MIGRATION wraps its +// create_table inside a class/module body, and rubySchemaTableCall's Gate 0 refuses every +// class/module-nested table — so the container DOES matter exactly there (the migration shape +// contributes ZERO defs, pinned by the three migration fixtures). The file signal below is a +// contains-"create_table" reject like the attribute-family capture's contains-"attr" reject; the +// semantic gate is the AST shape. +// +// Each `t. "name"` / `t. :name` receiver-CALL in the table's do-block body mints ONE +// SymKind::Section def at Lang::Ruby — the same data-kind slot as a YAML key / doc heading (model.h; +// note the wording change there: a Section is no longer "isolated in the graph" once these are +// Lang::Ruby and admit call edges) — with span = the name token and name = the literal (both quote +// spellings via stringLiteralText, or a simple_symbol's text minus ':'). The DSL CALLS keep their +// external-surface reference posture exactly like the attr family: no def named `string`/`datetime` +// appears. `t.timestamps` (a receiver call with NO arguments field) mints `created_at` AND `updated_at` +// — the DSL names them literally, symmetric with the id rule; the two defs anchor at the method +// token's start/end bytes, since no literal name tokens exist in the source. +// +// Non-column DSL — t.index / t.references / t.belongs_to / t.polymorphic — names no column and defines +// nothing (stated floor: `t.references :owner` would otherwise mint a phantom "owner" column). A column +// call whose receiver is not a bare identifier (`registry.string`, a receiver-less `string`) is not a +// column; a string named by a leading-comment-skipped FIRST argument is (the attribute-DSL comment posture). +// +// The id rule — the four grounded spellings, all pinned by test/rubyschemacheck.sh: +// – no column named `id`, no `id:` pair, no primary_key pair → an implicit `id` def anchored at +// the TABLE-NAME string's content start (no `id` token exists; the anchor is per-table unique, so +// the multi-def floor — `id` on N tables → --uses=id defs="N" — holds naturally) +// – `id: false` (the pair VALUE is the anonymous `false` token) → no id def (a key-less table) +// – `id: :uuid` (pair value a simple_symbol) → `id` def AT THE PAIR KEY (the key's +// text minus its trailing ':'; the uuid TYPE is not modelled — stated floor) +// – `primary_key: "x"` (pair value a string) → a def named the string's content +// instead of the implicit `id` +// Duplicate column NAMES across tables stay SEPARATE defs (distinct nameByte identities — the dedup +// ladder only folds same-byte captures). +inline constexpr std::array kRubySchemaNonColumns = { "index", "references", "belongs_to", "polymorphic", "check_constraint", "exclusion_constraint", "unique_constraint" }; + +// The FIRST non-comment named child of an argument_list — the attribute-DSL comment posture ("attribute( # note +// then :x"): a comment is an n-ary extra, never an argument. Null when the list is all comments/empty. +inline TSNode rubyFirstNonCommentArg( TSNode args ) noexcept +{ + ChildCursor cursor( args ); + TSNode first{}; + bool got = false; + forEachNamedChild( args, cursor.cur, [ & ]( TSNode c ) + { + if( kindIs( ts_node_type( c ), "comment" ) ) + { + return true; // skip the comment train, keep looking + } + first = c; + got = true; + return false; // stop at the first real argument + } ); + return got ? first : TSNode{}; +} + +// A pair node's KEY text: a hash_key_symbol carries its trailing ':' (the whole `force:` token) — strip +// it so the id/primary_key comparisons see the bare word. A key without the colon (already bare) passes +// through unchanged. +inline std::string_view rubyPairKeyText( TSNode pair, std::string_view src ) noexcept +{ + const TSNode key = fieldChild( pair, NodeField::Key ); + std::string_view t = nodeTextOf( key, src ); + if( !t.empty() && t.back() == ':' ) + { + t.remove_suffix( 1 ); + } + return t; +} + +// One `t. ` receiver-call inside a table block: a Section def per the rules above. The +// RECEIVER must BE the block parameter the create_table call bound (`t` in a rendered schema's do |t|); +// any other bare identifier (`helper.string "x"`) is somebody's own helper call and names no column. +// The `sawId` out-param records an explicit `id` column so the implicit rule can see it. +inline void rubySchemaColumnCall( TSNode c, std::uint32_t fileId, std::string_view src, std::string_view receiverName, std::vector& defs, bool& sawId ) +{ + if( !kindIs( ts_node_type( c ), "call" ) ) + { + return; + } + const TSNode recv = fieldChild( c, NodeField::Receiver ); + if( ts_node_is_null( recv ) || !kindIs( ts_node_type( recv ), "identifier" ) || nodeTextOf( recv, src ) != receiverName ) + { + return; // receiver-less, receiver-qualified, or a DIFFERENT bare name: not a column on the block parameter + } + const std::string_view m = fieldIdentifierText( c, NodeField::Method, src ); + if( m.empty() ) + { + return; + } + const std::uint32_t line = ts_node_start_point( c ).row + 1; + if( m == "timestamps" && ts_node_is_null( fieldChild( c, NodeField::Arguments ) ) ) + { + const TSNode meth = fieldChild( c, NodeField::Method ); + const std::uint32_t s = ts_node_start_byte( meth ), e = ts_node_end_byte( meth ); + RawDef d; + d.fileId = fileId; d.line = line; d.startByte = s; d.endByte = e; d.loc = 1; + d.kind = SymKind::Section; d.lang = Lang::Ruby; + d.nameByte = s; d.name = "created_at"; defs.push_back( d ); // the two anchors are the + d.nameByte = e; d.name = "updated_at"; defs.push_back( d ); // method token's edges — no literal tokens exist + return; + } + if( std::find( kRubySchemaNonColumns.begin(), kRubySchemaNonColumns.end(), m ) != kRubySchemaNonColumns.end() ) + { + return; // t.index / references / belongs_to / polymorphic name no column (stated floor) + } + const TSNode args = fieldChild( c, NodeField::Arguments ); + if( ts_node_is_null( args ) ) + { + return; + } + const TSNode nameNode = rubyFirstNonCommentArg( args ); + if( ts_node_is_null( nameNode ) ) + { + return; + } + const std::string lit = stringLiteralText( nameNode, src ); // "" when not a string node + std::string_view txt = lit; + if( txt.empty() && kindIs( ts_node_type( nameNode ), "simple_symbol" ) ) + { + txt = nodeTextOf( nameNode, src ); + if( txt.size() > 1 && txt.front() == ':' ) + { + txt.remove_prefix( 1 ); + } + } + if( txt.empty() ) + { + return; // `""` / `: ` defensive: an empty name is no column + } + const std::uint32_t s = ts_node_start_byte( nameNode ), e = ts_node_end_byte( nameNode ); + RawDef d; + d.fileId = fileId; d.line = line; d.startByte = s; d.endByte = e; d.loc = 1; + d.kind = SymKind::Section; d.lang = Lang::Ruby; d.nameByte = s + 1; // content start — past the quote or ':' + d.name = std::string( txt ); + if( d.name == "id" ) + { + sawId = true; + } + defs.push_back( d ); +} + +// One create_table call: the schema-file gate plus the whole table's defs (name, options, columns, id). +inline void rubySchemaTableCall( TSNode n, std::uint32_t fileId, std::string_view src, std::vector& defs ) +{ + std::string_view receiverName; // the block parameter's name (`t`); empty until Gate 2 passes + // Gate 0: a rendered schema's tables live at file level or under `ActiveRecord::Schema[].define … do` + // — NEVER inside a class/module body. A MIGRATION's `create_table "users" … do |t|` (class-wrapped, + // often string-named) is the same AST shape, and without this gate it would mint the SAME columns as + // the schema that migration created — double-counting every added column on every corpus with both + // files indexed. `add_column`/`remove_column` are not read anywhere (a column added then dropped never + // registers; pinned by the migration fixtures), but the create_table of a real migration must not + // double its schema either. Excluding at least one `class`/`module` ancestor keeps the capture + // content-gated (still no path heuristic): a migration file next to the schema contributes ZERO defs. + // Checked FIRST — the cheapest gate — before the argument/block walks. + { + for( TSNode p = ts_node_parent( n ); !ts_node_is_null( p ); p = ts_node_parent( p ) ) + { + const std::string_view k( ts_node_type( p ) ); + if( k == "class" || k == "module" ) // a locals distilled walk — no cross-helper dependency + { + return; + } + } + } + const TSNode args = fieldChild( n, NodeField::Arguments ); + if( ts_node_is_null( args ) ) + { + return; + } + // Gate 1: a STRING table name as the first argument (a symbol-named table is a migration spelling, + // not a rendered-schema shape — stated floor; no name node → no schema surface either). + const TSNode tableName = rubyFirstNonCommentArg( args ); + if( ts_node_is_null( tableName ) || !kindIs( ts_node_type( tableName ), "string" ) ) + { + return; + } + // Gate 2: the call's OWN do_block, and its parameters: field carries ONE bare identifier (the `|t|`). + TSNode block{}; + { + ChildCursor cursor( n ); + forEachNamedChild( n, cursor.cur, [ & ]( TSNode c ) + { + if( kindIs( ts_node_type( c ), "do_block" ) ) + { + block = c; + return false; + } + return true; + } ); + } + if( ts_node_is_null( block ) ) + { + return; + } + const TSNode params = fieldChild( block, NodeField::Parameters ); + if( ts_node_is_null( params ) ) + { + return; // `create_table "x" do … end` with no handle — not the schema shape + } + { + ChildCursor cursor( params ); + TSNode only{}; + std::size_t named = 0; + forEachNamedChild( params, cursor.cur, [ & ]( TSNode c ) { only = c; ++named; return true; } ); + if( named != 1 || !kindIs( ts_node_type( only ), "identifier" ) ) + { + return; // multi-parameter or non-identifier handle — not a rendered schema's do |t| + } + receiverName = nodeTextOf( only, src ); // the `|t|` spelling — column calls must use it + } + // Options: the id/primary_key pairs among the call's arguments. + bool idFalse = false, primaryKey = false; + TSNode idKey{}, pkValue{}; + { + ChildCursor cursor( args ); + forEachNamedChild( args, cursor.cur, [ & ]( TSNode a ) + { + if( !kindIs( ts_node_type( a ), "pair" ) ) + { + return true; + } + const std::string_view k = rubyPairKeyText( a, src ); + const TSNode v = fieldChild( a, NodeField::Value ); + if( k == "id" ) + { + if( kindIs( ts_node_type( v ), "false" ) ) + { + idFalse = true; + } + else if( kindIs( ts_node_type( v ), "simple_symbol" ) ) + { + idKey = fieldChild( a, NodeField::Key ); + } + } + else if( k == "primary_key" ) + { + primaryKey = true; + pkValue = v; + } + return true; + } ); + } + // The columns: the do_block's body: field (its body_statement) — walk its named children directly + // (a fresh cursor; the outer walks cannot be reused mid-flight, tschildren.h). + bool sawIdColumn = false; + const TSNode body = fieldChild( block, NodeField::Body ); + if( !ts_node_is_null( body ) ) + { + ChildCursor cursor( body ); + forEachNamedChild( body, cursor.cur, [ & ]( TSNode c ) + { + rubySchemaColumnCall( c, fileId, src, receiverName, defs, sawIdColumn ); + return true; + } ); + } + // The id rule, in the grounded order: an explicit id column wins; then the pairs decide. + if( sawIdColumn || idFalse ) + { + return; + } + if( primaryKey && !ts_node_is_null( pkValue ) && kindIs( ts_node_type( pkValue ), "string" ) ) + { + const std::string name = stringLiteralText( pkValue, src ); + if( name.empty() ) + { + return; + } + const std::uint32_t s = ts_node_start_byte( pkValue ), e = ts_node_end_byte( pkValue ); + RawDef d; + d.fileId = fileId; d.line = ts_node_start_point( pkValue ).row + 1; d.startByte = s; d.endByte = e; d.loc = 1; + d.kind = SymKind::Section; d.lang = Lang::Ruby; d.nameByte = s + 1; d.name = name; + defs.push_back( d ); + return; + } + if( !ts_node_is_null( idKey ) ) + { + const std::uint32_t s = ts_node_start_byte( idKey ), e = ts_node_end_byte( idKey ); + RawDef d; + d.fileId = fileId; d.line = ts_node_start_point( idKey ).row + 1; d.startByte = s; d.endByte = e; d.loc = 1; + d.kind = SymKind::Section; d.lang = Lang::Ruby; d.nameByte = s; d.name = "id"; + defs.push_back( d ); + return; + } + // A primary_key pair with a NON-string value (composite `["a","b"]`, hand-written `:symbol`): the + // key is NOT an `id` and the rendered columns are the block's own explicit `t.` lines (already + // minted above) — suppress the implicit id entirely (the composite case is pinned by the fixture). + if( primaryKey ) + { + return; + } + // Implicit `id` — anchored at the table-name string's content start (a stable per-table byte). + { + const std::uint32_t s = ts_node_start_byte( tableName ), e = ts_node_end_byte( tableName ); + RawDef d; + d.fileId = fileId; d.line = ts_node_start_point( tableName ).row + 1; d.startByte = s; d.endByte = e; d.loc = 1; + d.kind = SymKind::Section; d.lang = Lang::Ruby; d.nameByte = s + 1; d.name = "id"; + defs.push_back( d ); + } +} + + + // F5: a Swift LOCAL binding — `let a = f()` / `var b = ...` inside a function/closure body — parses to the // same `property_declaration` node as a real stored/computed MEMBER property, so the @definition.var pattern // captures it as a spurious top-level `var` symbol AND (being the nearest enclosing symbol above the body's diff --git a/src/ingest_parsepool.h b/src/ingest_parsepool.h index 02ca53606..f583c78a0 100644 --- a/src/ingest_parsepool.h +++ b/src/ingest_parsepool.h @@ -407,7 +407,10 @@ inline void runParseWorker( ParsePoolShared& sh, unsigned t ) const TSNode root = ts_tree_root_node( pending.tree ); const std::size_t firstNewDefIndex = out.defs.size(); ExtractShortfall tagsShortfall; - captureTagsFacts( cursor, *pending.le, pending.fileId, pending.bytes, root, out.defs, out.refs, out.binds, out.incs, pending.ppDead, tagsShortfall ); + underExtractScope( pending.le->lang, [ & ] + { + captureTagsFacts( cursor, *pending.le, pending.fileId, pending.bytes, root, out.defs, out.refs, out.binds, out.incs, pending.ppDead, tagsShortfall ); + } ); notePartialExtract( scan, pending.fileId, tagsShortfall, pending.bytes.size() ); buildLexForNewDefs( out.defs, firstNewDefIndex, pending.bytes ); // B0.2: bytes still in memory ts_tree_delete( pending.tree ); @@ -565,7 +568,7 @@ inline void runParseWorker( ParsePoolShared& sh, unsigned t ) { continue; } - TreeGuard mdTree( parseTree( pg.p, bytes ) ); + TreeGuard mdTree( underExtractScope( le->lang, [ & ] { return parseTree( pg.p, bytes ); } ) ); if( mdTree.get() == nullptr ) { continue; @@ -602,7 +605,7 @@ inline void runParseWorker( ParsePoolShared& sh, unsigned t ) continue; } - TreeGuard tree( parseTree( pg.p, bytes ) ); + TreeGuard tree( underExtractScope( le->lang, [ & ] { return parseTree( pg.p, bytes ); } ) ); if( tree.get() == nullptr ) { continue; @@ -616,7 +619,10 @@ inline void runParseWorker( ParsePoolShared& sh, unsigned t ) // above), which is why it is computed here and not inside either driver. std::vector ppDead = preprocDeadRangesFor( *le, root, bytes ); ExtractShortfall shortfall; // every pass's disclosure for THIS file; noted on its slot once both have run - captureSideFacts( *le, static_cast( fileId ), bytes, root, out.refs, out.incs, out.binds, out.ffis, out.routeDefs, out.routeUses, out.constOpens, sh.captureValueUses, ppDead, shortfall ); + underExtractScope( le->lang, [ & ] + { + captureSideFacts( *le, static_cast( fileId ), bytes, root, out.refs, out.incs, out.binds, out.ffis, out.routeDefs, out.routeUses, out.constOpens, sh.captureValueUses, ppDead, shortfall ); + } ); notePartialExtract( scan, fileId, shortfall, bytes.size() ); // before a queued file's tags pass, which notes its own appendBlankedMacroUses( macroWork, le->lang, static_cast( fileId ), bytes, out.refs, ppDead ); @@ -636,7 +642,10 @@ inline void runParseWorker( ParsePoolShared& sh, unsigned t ) waitForQueryPrewarm( &sh.gate ); } const std::size_t firstNewDefIndex = out.defs.size(); - captureTagsFacts( cursor, *le, static_cast( fileId ), bytes, root, out.defs, out.refs, out.binds, out.incs, ppDead, shortfall ); + underExtractScope( le->lang, [ & ] + { + captureTagsFacts( cursor, *le, static_cast( fileId ), bytes, root, out.defs, out.refs, out.binds, out.incs, ppDead, shortfall ); + } ); notePartialExtract( scan, fileId, shortfall, bytes.size() ); buildLexForNewDefs( out.defs, firstNewDefIndex, bytes ); // B0.2: bytes still in memory } diff --git a/src/ingest_relations.h b/src/ingest_relations.h index b3f1e9ea5..f0148ded3 100644 --- a/src/ingest_relations.h +++ b/src/ingest_relations.h @@ -477,6 +477,7 @@ void captureMacroBodyCalls( TSNode defineNode, std::uint32_t fileId, Lang lang, // child, collecting type nodes at both depths (Rust is a separate pass — impl Trait for T is a sibling). void captureBases( TSNode classNode, std::uint32_t fileId, Lang lang, std::string_view src, std::vector& refs ) { + PROFILE_SCOPE_DESCRIBE( "ingest/relations: base classes of one class" ); // O(children) at all three levels. These child lists LOOK grammar-bounded — a class node's clauses, a // clause's base types — and the earlier class-3 reasoning said so, but EXTRAS refute it: a comment run // between two base types is spliced straight into the clause's own child array (src/infra/tschildren.h), @@ -789,6 +790,7 @@ inline WrittenPointee stdSmartPointee( TSNode typeNode, std::string_view src ) n // `w_.m()` does not (kParserVer 113). Every other qualified template (`std::vector v_;`) still records nothing. void captureFields( TSNode classNode, std::uint32_t fileId, Lang lang, std::string_view src, std::vector& refs ) { + PROFILE_SCOPE_DESCRIBE( "ingest/relations: fields of one class" ); if( lang != Lang::Cpp ) { return; // C++ only for S5-E; extend for Python/TS later @@ -1099,8 +1101,10 @@ inline std::string phpUseTarget( TSNode useNode, std::string_view src ) // TS/JS `require("./x")` and dynamic `import("./x")` → the written specifier, or empty when this call // expression is not a module load. Its own function for the same reason csharpUsingTarget and phpUseTarget -// are: one AST shape per language stays a one-line call inside directiveTargetOf, whose grain is one -// branch per grammar spelling rather than one branch plus one nested scan. +// are: one AST shape per language stays a one-line call at the use site rather than one branch plus one nested +// scan. #358: its readers are now the DepDialect::Web normaliser (src/ingest_importcap.h — the captured +// `require`/`import` call) and captureJsImportFacts (src/ingest_jsimports.h — the binding a destructured +// require makes); directiveTargetOf no longer reads it, because the walk no longer visits TS/JS. // // THREE conditions, all required, and each is what keeps an ordinary call out of the dependency graph: // * the callee is the BARE identifier `require` or `import` — a member expression @@ -1867,7 +1871,8 @@ inline bool isPreprocConditional( const char* type ) noexcept // --expand's sibling lift, the SameInclude call-resolution tier, and --impact's import tier) was reading // an empty table and reporting the emptiness as a horizontal, cycle-free architecture. The four entries // below are exactly the statement forms a top-level `require` is written in; the call node itself is read -// by directiveTargetOf, which is what keeps every OTHER call expression out. +// by jsModuleLoadTarget (read by the Web normaliser in src/ingest_importcap.h since #358), which is what keeps +// every OTHER call expression out. // // EVERY ENTRY HAS A FIXTURE ARM in test/nestedimportfix — an entry with no arm is an untested claim, and // every parent chain below was read off a real parse with `--match`, never predicted from the grammar. @@ -1994,7 +1999,10 @@ inline bool isFunctionLike( Lang lang, const char* type ) noexcept return false; } -// TS/JS: every container a `require("./x")` / `import("./x")` call can legitimately sit under. +// TS/JS: every container a `require("./x")` / `import("./x")` call can legitimately sit under. #358: no walk +// reads this table for TS/JS any more — it is now the REACH RULE src/ingest_importcap.h re-derives from a +// captured `@import.path` node's ancestry (a specifier under a node that is not listed here was never an edge, +// and still is not), and the `insideFn` source for the lazy bit (kJsFunctionContainers, below). // // The first five are the statement forms that wrap a TOP-LEVEL require, kParserVer 71's set — read off // real parses, not predicted: @@ -2028,7 +2036,7 @@ inline bool isFunctionLike( Lang lang, const char* type ) noexcept // function: (_) @f)'` returns a real `function:` field of text "import" here, so the earlier miss was the // missing container, never a grammar shape jsModuleLoadTarget could not read). // EVERY ENTRY still needs the SAME three jsModuleLoadTarget guards (bare require/import callee, one arg, a -// string literal) — this table only widens WHERE the walk looks, never what counts as a hit. +// string literal) — this table only widens WHERE a specifier may sit, never what counts as a hit. inline constexpr std::array kJsImportContainers = { "lexical_declaration", "variable_declaration", "variable_declarator", "expression_statement", "assignment_expression", "statement_block", "return_statement", "labeled_statement", "export_statement", @@ -2094,25 +2102,29 @@ constexpr std::uint16_t kMaxImportContainerDepth = 256; // READS them stay separately readable — the walk is one shape, this is one branch per grammar spelling. // // Node types confirmed per grammar: Python import_statement/import_from_statement; Go/Swift -// import_declaration; Rust use_declaration + mod_item; C# using_directive; TS/JS call_expression for the -// CommonJS `require("./x")` and dynamic `import("./x")` spellings; plus the kParserVer-81 four (bash +// import_declaration; Rust use_declaration + mod_item; C# using_directive; plus the kParserVer-81 four (bash // `command`, lua `function_call`, ruby `call`, elixir `call`). LEVER-B B0: non-C imports capture the // CLEAN written specifier via grammar child fields (module path / quoted specifier / use argument), not a // sliced clause — the sound resolver input. // -// #358: the C-family rows are GONE from this list. C++ preproc_include (path field, "" local vs <> +// #358: the C-family rows are GONE from this list, and so is Go's (the Go `import_declaration` clause is read by +// the DepDialect::Go normaliser; only Swift still reaches the `import_declaration` branch below). C++ preproc_include (path field, "" local vs <> // external) and C++ preproc_call with directive `#import` are captured as `@import.path` by the grammar's // own query and read by the DepDialect::CFamily normaliser (src/ingest_importcap.h), so this function is -// never asked about a C-family node. What is left is every language whose extraction needs the walk's own -// frame state — which is also the list of languages that have NOT moved over yet. +// never asked about a C-family node. The TS/JS rows (import_statement `source:`, the export_statement +// re-export, the CommonJS / dynamic call_expression) went the same way: `@import.path` in +// queries/{typescript,tsx,javascript}/tags.scm, DepDialect::Web normaliser. What is left is every language +// whose extraction needs the walk's own frame state — which is also the list of languages that have NOT +// moved over yet. // -// `lang` exists for exactly one branch: `call_expression` is a node type in most of our grammars, and a -// C++ or Rust function that happens to be named `require` must never manufacture a dependency edge. The -// language gate makes that impossible by construction rather than by relying on where the walk goes. +// `lang` exists for the language-gated branches: `call_expression` / `call` / `command` are node types in +// most of our grammars, and a C++ or Rust function that happens to be named `require` must never +// manufacture a dependency edge. The language gate makes that impossible by construction rather than by +// relying on where the walk goes. // -// `isLazy` (kParserVer 72, TS/JS; Ruby since parser version 82/83): true when `insideFn` says this call sits -// inside a closure container — see kJsFunctionContainers, the Ruby closure kinds, and captureIncludes' -// `insideFn` propagation below — or when the directive is a Ruby `autoload`. Allocates a std::string → +// `isLazy` (Ruby since parser version 82/83; TS/JS since kParserVer 72, now computed in ingest_importcap.h): +// true when `insideFn` says this call sits inside a closure container — see the Ruby closure kinds and +// captureIncludes' `insideFn` propagation below — or when the directive is a Ruby `autoload`. Allocates a std::string → // not noexcept. `isSymbolic` (parser version 82, Ruby only): the target is a CONSTANT resolved through the // corpus's own class/module index, never a path — see model.h Include::isSymbolic. // @@ -2122,8 +2134,38 @@ constexpr std::uint16_t kMaxImportContainerDepth = 256; // gone rather than documented as always-false. struct DirectiveTarget { std::string target; bool isLazy; bool isSymbolic; bool isReceiver; }; -// `insideFn` exists for exactly the same one branch `lang` does: whether the call_expression being read -// sits inside a TS/JS function body, per captureIncludes' walk — meaningless (and ignored) everywhere else. +// The written CLAUSE of an `import_declaration` — the declaration's own text with its leading keyword +// dropped (everything after the first space), at most 96 bytes, trailing `;` / space / newline / CR +// trimmed. This is the Include target Go has always carried (`"fmt"`, `f "fmt"`, `. "x"`, `_ "x"`, or a +// whole `( … )` group collapsed to ONE record), and Swift's `import Foundation`. Shared by the Swift walk +// branch below and by the DepDialect::Go normaliser (src/ingest_importcap.h) so the two cannot drift; the +// quirks are PRE-EXISTING and preserved byte-for-byte on purpose (#358 is a pure refactor): the keyword is +// dropped at a SPACE, so `import"os"` keeps the word `import`; a group is cut at 96 bytes, mid +// character if it lands there; a tab is not trimmed. Empty for an empty or out-of-range span. +std::string importClauseTarget( TSNode n, std::string_view src ) +{ + std::string target; + const std::uint32_t a = ts_node_start_byte( n ), b = ts_node_end_byte( n ); + if( a < b && b <= src.size() ) + { + std::string_view s = src.substr( a, b - a ); + const std::size_t sp = s.find( ' ' ); // drop the leading keyword + if( sp != std::string_view::npos ) + { + s = s.substr( sp + 1 ); + } + target.assign( s.data(), s.size() < 96 ? s.size() : 96 ); + while( !target.empty() && ( target.back() == ';' || target.back() == ' ' || target.back() == '\n' || target.back() == '\r' ) ) + { + target.pop_back(); + } + } + return target; +} + +// `insideFn` is the walk's sticky "inside a closure" bit, now read by Ruby alone (a constant receiver written +// inside a method / lambda / block is lazy). TS/JS used to be the other reader: its require()/import() branch +// moved to `@import.path` (#358), whose lazy bit is re-derived from ancestry in src/ingest_importcap.h. DirectiveTarget directiveTargetOf( TSNode n, const char* t, std::string_view src, Lang lang, bool insideFn ) { std::string target; @@ -2131,36 +2173,19 @@ DirectiveTarget directiveTargetOf( TSNode n, const char* t, std::string_view src bool isSymbolic = false; bool isReceiver = false; - if( kindIs( t, "import_statement" ) ) // Python `import a` / TS `import … from 'x'` + if( kindIs( t, "import_statement" ) ) // Python `import a` { // Prefer the grammar's specifier field over slicing the whole statement (LEVER-B B0: the resolver - // needs the REAL written specifier, not the clause). Empirically confirmed node shapes: + // needs the REAL written specifier, not the clause). Empirically confirmed node shape: // Python: import_statement name:(dotted_name|aliased_import) → the dotted module `pkg.mod`. - // TS/JS: import_statement source:(string) → the quoted specifier `'./x'`. - // A grammar that lacks the field resolves it to id 0, and fieldChild returns null for it, so a single - // capture covers both grammars without a per-language branch. - if( const TSNode src_ = fieldChild( n, NodeField::Source ); !ts_node_is_null( src_ ) ) - { - target = importSpecifierText( src_, src ); // TS/JS: strip the surrounding quotes - } - else if( const TSNode nm = fieldChild( n, NodeField::Name ); !ts_node_is_null( nm ) ) + // #358: the TS/JS `import … from 'x'` spelling of this node (and the `export … from 'x'` re-export that + // used to follow it) is GONE from this function — it is `@import.path` in queries/{typescript,tsx,javascript} + // /tags.scm now, read by the DepDialect::Web normaliser (src/ingest_importcap.h). + if( const TSNode nm = fieldChild( n, NodeField::Name ); !ts_node_is_null( nm ) ) { target = importSpecifierText( nm, src ); // Python: the dotted module head } } - else if( kindIs( t, "export_statement" ) && ( lang == Lang::TypeScript || lang == Lang::JavaScript ) ) - { - // kParserVer 124 (#220 part 2): a RE-EXPORT — `export { x } from './y'`, `export * from './y'`, - // `export * as ns from './y'`, `export type { T } from './y'` — loads its module exactly as an import does, - // and a barrel file is made of nothing else, so an import graph without them misses every cycle through a - // barrel. The grammar gives the specifier the same `source:` field import_statement has; every other - // export_statement (`export function …`, `export { x }`, `export default …`) has none and reads empty, and - // the walk still descends into it (it is a kJsImportContainers entry) for the requires inside. - if( const TSNode src_ = fieldChild( n, NodeField::Source ); !ts_node_is_null( src_ ) ) - { - target = importSpecifierText( src_, src ); - } - } else if( kindIs( t, "import_from_statement" ) ) // Python `from pkg.mod import Z` { // module_name:(dotted_name) → `pkg.mod`; module_name:(relative_import) → `.rel` / `..up` (leading @@ -2170,12 +2195,6 @@ DirectiveTarget directiveTargetOf( TSNode n, const char* t, std::string_view src target = importSpecifierText( mn, src ); } } - else if( kindIs( t, "call_expression" ) - && ( lang == Lang::TypeScript || lang == Lang::JavaScript ) ) // TS/JS `require("./x")` / `import("./x")` - { - target = jsModuleLoadTarget( n, src ); - isLazy = insideFn && !target.empty(); // kParserVer 72: a hit found inside a function body is LAZY - } else if( kindIs( t, "command" ) && lang == Lang::Bash ) // Bash `source x.sh` / `. x.sh` { target = bashSourceTarget( n, src ); @@ -2255,25 +2274,14 @@ DirectiveTarget directiveTargetOf( TSNode n, const char* t, std::string_view src } } } - else if( kindIs( t, "import_declaration" ) ) // Go / Swift — captured but NOT precise-resolved + else if( kindIs( t, "import_declaration" ) ) // Swift — captured but NOT precise-resolved { - // Go (needs go.mod module-root) and Swift (whole-module, no path) are DEFERRED — the precise - // resolver leaves them unresolved. Keep the best-effort target for --uses / --deps back-compat. - const uint32_t a = ts_node_start_byte( n ), b = ts_node_end_byte( n ); - if( a < b && b <= src.size() ) - { - std::string_view s = src.substr( a, b - a ); - const std::size_t sp = s.find( ' ' ); // drop the leading keyword - if( sp != std::string_view::npos ) - { - s = s.substr( sp + 1 ); - } - target.assign( s.data(), s.size() < 96 ? s.size() : 96 ); - while( !target.empty() && ( target.back() == ';' || target.back() == ' ' || target.back() == '\n' || target.back() == '\r' ) ) - { - target.pop_back(); - } - } + // #358: Go moved onto `@import.path` (queries/go/tags.scm + the DepDialect::Go normaliser in + // src/ingest_importcap.h), so this branch is reached by Swift ALONE now. The clause reader is the + // one both languages shared and is kept as ONE function (importClauseTarget) rather than copied: + // Swift (whole-module, no path) is DEFERRED in the precise resolver, and its slice has its own + // landing. Keep the best-effort target for --uses / --deps back-compat. + target = importClauseTarget( n, src ); } else if( kindIs( t, "using_directive" ) ) { // C# `using Foo.Bar;` / `using static Foo;` / `using X = Foo.Bar;` @@ -2293,14 +2301,12 @@ DirectiveTarget directiveTargetOf( TSNode n, const char* t, std::string_view src // Capture #include / import directives (physical dependencies) by walking the file's top-level nodes — // and the bodies of anything that WRAPS a directive, which tree-sitter does not flatten: preprocessor -// conditionals in the C family and C#, and ordinary language constructs in Python / Rust / C# / TS / JS -// (see isImportContainer). Each node is read by directiveTargetOf above. -// kParserVer 72 cost note: TS/JS's kJsImportContainers now includes call_expression/object/arguments — -// containers a require() sits under with no other purpose — so the walk over a TS/JS file approaches full -// AST size rather than "top-level statements only". Bounded per-node (each node is visited once) and still -// depth-capped by kMaxImportContainerDepth; no perf gate exists in this tree to budget against (see -// CLAUDE.md's "best tool first, then fast" note), and CommonJS's own weight — an entire module system's -// worth of edges was previously invisible (kParserVer 71's LB-H measurement) — is the justification. +// conditionals in C# (the C family and TS/JS read `@import.path` instead, #358), and ordinary language +// constructs in Python / Rust / C# (see isImportContainer). Each node is read by directiveTargetOf above. +// kParserVer 72 cost note (HISTORICAL for TS/JS since #358, which retired this walk for them — the cost is gone +// with it): TS/JS's kJsImportContainers included call_expression/object/arguments — containers a require() +// sits under with no other purpose — so the walk over a TS/JS file approached full AST size rather than +// "top-level statements only". // ABS-3: each directive ALSO emits an import-role RawRef (name = the importable final segment) so the // use-site index reports import sites. The ref is file-scope (fromSymbol=kNoNode) — that is correct for // a directive at any container depth, and it NEVER enters the call graph (role != Call → skipped in @@ -2441,10 +2447,11 @@ void captureIncludes( TSNode root, Lang lang, std::uint32_t fileId, std::string_ // order, which keeps `incs`/`refs` in SOURCE order: the determinism contract is byte-identity, and an // order that depended on the walk shape would break it. Only ALLOWLISTED containers are entered, so a // language whose imports are top-level by rule (Go, Java) still costs exactly the old scan. - // `insideFn` (kParserVer 72, TS/JS only): true once the walk has descended through a function-body - // container (kJsFunctionContainers) — sticky for every descendant, never cleared, exactly like `depth` + // `insideFn` (kParserVer 72, TS/JS; Ruby's closure kinds since 83): true once the walk has descended through a + // function-body container (isFunctionLike) — sticky for every descendant, never cleared, exactly like `depth` // is monotonic. It rides the frame rather than being recomputed from ancestry because the walk never - // keeps the ancestor chain around: this is the one bit of it a lazy-require call needs. + // keeps the ancestor chain around: this is the one bit of it a lazy-require call needs. #358: TS/JS no longer + // walks (its lazy bit is re-derived from ancestry in ingest_importcap.h), so Ruby is its only reader here. // `openIdx` (parser version 83, Ruby only): the index in `constOpens` of the innermost class/module open the // frame sits inside — kNoOpenIdx at file level. It rides the frame for the same reason `insideFn` does (the // walk keeps no ancestor chain) and exists for the RECEIVER DEDUPE: a constant receiver is recorded once per diff --git a/src/ingest_sidecap.h b/src/ingest_sidecap.h index 6ea1140e7..9925103fa 100644 --- a/src/ingest_sidecap.h +++ b/src/ingest_sidecap.h @@ -1150,6 +1150,90 @@ bool prepareParserFor( TSParser* parser, const LangEntry& le ) return true; } +// ── cold-ingest profile scopes by LANGUAGE FAMILY (cli-perf-068 STEP 0) ────────────────────────────────────────── +// A PROFILE_SCOPE names one code site, so the parse pool's per-file phases (parse, side captures, tags captures) are +// opened through underExtractScope, which picks one of eight family-named sites per phase: the self-profile then splits +// a cold index's parse and extraction cost by language family (the LLVM memory run reads these). In a build without +// RIPWIRE_PROFILE the macro is nothing and this is a switch that calls `work` — same work, same order, same answer. +enum class ExtractFamily : std::uint8_t { CFamily, Python, JsTs, JvmDotNet, GoRustSwiftDart, Scripting, Bash, DocsData, Count }; +enum class ExtractPhase : std::uint8_t { Parse, SideFacts, TagsFacts, Count }; + +// One row per Lang, in enumerator order (model.h): the size assert below fails when a language is appended without a row. +inline constexpr ExtractFamily kExtractFamilyByLang[] = { + ExtractFamily::CFamily, // Cpp + ExtractFamily::Python, // Python + ExtractFamily::JsTs, // TypeScript + ExtractFamily::GoRustSwiftDart, // Go + ExtractFamily::GoRustSwiftDart, // Rust + ExtractFamily::GoRustSwiftDart, // Swift + ExtractFamily::CFamily, // ObjC + ExtractFamily::DocsData, // Markdown + ExtractFamily::JsTs, // JavaScript + ExtractFamily::Bash, // Bash + ExtractFamily::JvmDotNet, // Java + ExtractFamily::Scripting, // Ruby + ExtractFamily::DocsData, // Unknown + ExtractFamily::DocsData, // Json + ExtractFamily::JvmDotNet, // CSharp + ExtractFamily::CFamily, // C + ExtractFamily::DocsData, // Toml + ExtractFamily::DocsData, // Yaml + ExtractFamily::Scripting, // Php + ExtractFamily::Scripting, // Lua + ExtractFamily::Scripting, // Elixir + ExtractFamily::GoRustSwiftDart, // Dart + ExtractFamily::JvmDotNet, // Kotlin + ExtractFamily::Scripting, // GDScript +}; +static_assert( std::size( kExtractFamilyByLang ) == kLangCount, "one profile family row per Lang, in enumerator order" ); +static_assert( kExtractFamilyByLang[ std::size_t( Lang::Kotlin ) ] == ExtractFamily::JvmDotNet && kExtractFamilyByLang[ std::size_t( Lang::C ) ] == ExtractFamily::CFamily, + "the table follows Lang's enumerator order" ); + +constexpr ExtractFamily extractFamilyOf( Lang lang ) noexcept +{ + const std::size_t langIdx = static_cast( lang ); + return langIdx < kLangCount ? kExtractFamilyByLang[ langIdx ] : ExtractFamily::DocsData; +} + +inline constexpr const char* kExtractScopeNames[ std::size_t( ExtractPhase::Count ) ][ std::size_t( ExtractFamily::Count ) ] = { + { "ingest/extract[c-family]: parse one file", "ingest/extract[python]: parse one file", "ingest/extract[js-ts]: parse one file", + "ingest/extract[jvm-dotnet]: parse one file", "ingest/extract[go-rust-swift-dart]: parse one file", "ingest/extract[scripting]: parse one file", + "ingest/extract[bash]: parse one file", "ingest/extract[docs-data]: parse one file" }, + { "ingest/extract[c-family]: side captures (relations, binds, includes, values)", "ingest/extract[python]: side captures (relations, binds, includes, values)", + "ingest/extract[js-ts]: side captures (relations, binds, includes, values)", "ingest/extract[jvm-dotnet]: side captures (relations, binds, includes, values)", + "ingest/extract[go-rust-swift-dart]: side captures (relations, binds, includes, values)", "ingest/extract[scripting]: side captures (relations, binds, includes, values)", + "ingest/extract[bash]: side captures (relations, binds, includes, values)", "ingest/extract[docs-data]: side captures (relations, binds, includes, values)" }, + { "ingest/extract[c-family]: tags captures (defs, refs, metrics)", "ingest/extract[python]: tags captures (defs, refs, metrics)", + "ingest/extract[js-ts]: tags captures (defs, refs, metrics)", "ingest/extract[jvm-dotnet]: tags captures (defs, refs, metrics)", + "ingest/extract[go-rust-swift-dart]: tags captures (defs, refs, metrics)", "ingest/extract[scripting]: tags captures (defs, refs, metrics)", + "ingest/extract[bash]: tags captures (defs, refs, metrics)", "ingest/extract[docs-data]: tags captures (defs, refs, metrics)" }, +}; + +template +decltype( auto ) inExtractScope( Work&& work ) +{ + PROFILE_SCOPE_DESCRIBE( kExtractScopeNames[ std::size_t( Phase ) ][ std::size_t( Family ) ] ); + return work(); +} + +template +decltype( auto ) underExtractScope( Lang lang, Work&& work ) +{ + switch( extractFamilyOf( lang ) ) + { + case ExtractFamily::CFamily: return inExtractScope( work ); + case ExtractFamily::Python: return inExtractScope( work ); + case ExtractFamily::JsTs: return inExtractScope( work ); + case ExtractFamily::JvmDotNet: return inExtractScope( work ); + case ExtractFamily::GoRustSwiftDart: return inExtractScope( work ); + case ExtractFamily::Scripting: return inExtractScope( work ); + case ExtractFamily::Bash: return inExtractScope( work ); + case ExtractFamily::DocsData: break; + case ExtractFamily::Count: break; + } + return inExtractScope( work ); +} + TSTree* parseTree( TSParser* parser, std::string_view src ) { TSTree* tree = nullptr; @@ -1546,11 +1630,12 @@ void captureSideFacts( const LangEntry& le, std::uint32_t fileId, std::string_vi #endif // Elixir directives share the lexical tags context below (ElixirContext), and as of #358 so do the - // C-family ones: their `@import.path` captures are normalised and emitted by captureTagsFacts, and - // captureIncludes no longer reads a preproc_include / preproc_call at all. The walk is pure cost - // for those languages — it descends only allowlisted containers, and a C-family include lives at - // file scope or inside a preprocessor guard, which the tags query already reaches unanchored. - if( le.lang != Lang::Elixir && dependencyDialect( le.lang ) != DepDialect::CFamily ) + // C-family ones and TypeScript/JavaScript's: their `@import.path` captures are normalised and emitted + // by captureTagsFacts, and captureIncludes no longer reads a preproc_include / preproc_call / + // import_statement / export_statement / require() call at all. The walk is pure cost for those + // languages — it descends only allowlisted containers, and the tags query already reaches the same + // nodes unanchored (importsFromCapture names the dialects; ingest_importcap.h keeps the reach rule). + if( le.lang != Lang::Elixir && !importsFromCapture( dependencyDialect( le.lang ) ) ) { captureIncludes( root, le.lang, fileId, src, incs, refs, binds, constOpens, shortfall ); } @@ -1725,6 +1810,14 @@ inline bool isCFamilyTypeSpecifier( Lang lang, TSNode node ) noexcept return kindIs( t, "enum_specifier" ) || kindIs( t, "struct_specifier" ) || kindIs( t, "union_specifier" ) || kindIs( t, "class_specifier" ); } +// The two postures of one tags pass. Full: every fact the index keeps. DefinitionsOnly (definitionLinesInBlobs, --whereis on +// other refs): the SAME definitions — every step that decides a definition's existence, name, line, kind or scope runs +// unchanged — without the work only references, imports and metrics read: the @reference arm, @import emission, +// complexityOf, the function-scope span, the def arm's IS-A / HAS-A / macro-body reference minting and Ruby's +// reference and bind passes. Measured on 104 C++ header blobs: those were ~60% of this function's samples (cc_walk alone +// ~30%, the reference arm's qualifier / receiver parent walks ~30%). +enum class TagsCapture : std::uint8_t { Full, DefinitionsOnly }; + /// Append definitions and references captured by the language query, with language-specific filtering. /// Captured spans refer to src and root; a null cursor appends nothing. Existing output rows are retained. /// `ppDead` is this file's decided-dead byte ranges (preprocDeadRangesFor, computed ONCE per file by the @@ -1732,8 +1825,9 @@ inline bool isCFamilyTypeSpecifier( Lang lang, TSNode node ) noexcept /// file's tags pass runs later, at the prewarm flush). void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t fileId, std::string_view src, TSNode root, std::vector& defs, std::vector& refs, std::vector& binds, std::vector& includes, - const std::vector& ppDead, ExtractShortfall& shortfall ) + const std::vector& ppDead, ExtractShortfall& shortfall, TagsCapture mode = TagsCapture::Full ) { + const bool full = mode == TagsCapture::Full; if( cursor == nullptr ) { return; @@ -1860,7 +1954,10 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t // order while costing an allocation and a drain block. case CapRole::Import: { - emitCapturedImport( cap.node, fileId, le.lang, src, includes, refs, shortfall ); + if( full ) + { + emitCapturedImport( cap.node, fileId, le.lang, src, includes, refs, shortfall ); + } } break; } @@ -2101,7 +2198,7 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t // A literal-bound name measures its LITERAL; every other def measures defNode, as before. const bool literalBound = !ts_node_is_null( own.literal ); const TSNode metricNode = literalBound ? own.literal : defNode; - const auto [ cxVal, ccxVal, nestVal, localsVal, ppAltVal, humpsVal, deepVal, evVal, evWhyVal ] = fnOrMethod ? complexityOf( spanThroughBody ? body : metricNode, src, le.lang ) : Complexity{ 0u, 0u, 0u, 0u, 0u, 0u, 0u, 0u, {} }; + const auto [ cxVal, ccxVal, nestVal, localsVal, ppAltVal, humpsVal, deepVal, evVal, evWhyVal ] = fnOrMethod && full ? complexityOf( spanThroughBody ? body : metricNode, src, le.lang ) : Complexity{ 0u, 0u, 0u, 0u, 0u, 0u, 0u, 0u, {} }; d.cx = cxVal; d.ccx = ccxVal; d.locals = localsVal; // Phase 1: floor count, C/C++ only (model.h localsCountedLang) — 0 elsewhere, never emitted there @@ -2176,7 +2273,7 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t d.internalLinkage = internalLinkageBit( le.lang, defNode, src ); // A FUNCTION bound inside another function's body records that function's span (ingest_names.h // enclosingFunctionScope): graph.h reachableByName ranks it below every def a call outside it can name. - if( kind == SymKind::Function && !bindsOutsideItsFunction( roleNode, le.lang ) ) + if( full && kind == SymKind::Function && !bindsOutsideItsFunction( roleNode, le.lang ) ) { // Descend to the OUTER of the two def nodes: a C-family def's defNode climbed from its declarator (the role // node) to the function_definition, which must not read as the function enclosing itself; a multi-name @@ -2261,6 +2358,10 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t // holds an error (fileHasError, one O(1) flag test per file) — see parseRecoveredBits. d.recovered = fileHasError ? parseRecoveredBits( defNode, kind, le.lang, d.scope.empty() ) : std::uint8_t( 0 ); defs.push_back( std::move( d ) ); + if( !full ) + { + continue; // TagsCapture::DefinitionsOnly: the reference edges below feed no definition + } if( kind == SymKind::Class || isStructOrNamedType( kind ) || kind == SymKind::Interface ) { captureBases( defNode, fileId, le.lang, src, refs ); // IS-A: inheritance edges (derived → base) @@ -2271,7 +2372,7 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t captureMacroBodyCalls( roleNode, fileId, le.lang, src, refs ); // macro-edges: the graph connects THROUGH the macro } } - else if( isRef ) + else if( isRef && full ) { // H4: a C++ cast keyword is not a call — see isCppCastKeyword. Valid input, skipped, no alert. if( le.lang == Lang::Cpp && isCppCastKeyword( nameTxt ) ) @@ -2426,7 +2527,11 @@ void captureTagsFacts( TSQueryCursor* cursor, const LangEntry& le, std::uint32_t // `include` in `class << self` — into binds, beside the inherit reference each mixin already is. if( le.lang == Lang::Ruby ) { - captureRubyAttrDefs( root, fileId, src, defs ); + captureRubyDefs( root, fileId, src, defs ); // #339: the Definitions-only walker (attr defs + rendered db/schema.rb columns) + if( !full ) + { + return; // TagsCapture::DefinitionsOnly: the passes below write references and binds only + } captureRubyBareCalls( root, fileId, src, refs ); captureRubyFactories( root, fileId, src, binds ); captureRubyHelperMethods( root, fileId, src, binds ); diff --git a/src/landingplan.h b/src/landingplan.h index a69fbbda3..0069333de 100644 --- a/src/landingplan.h +++ b/src/landingplan.h @@ -73,6 +73,7 @@ struct PlanResult bool nonGitRoot = false; // --stray-content's own refusal reasons, passed through bool tooManyRefs = false; // verbatim — this verb adds no refusal of its own bool filterMatchedNothing = false; // H7: the filter selected no ref NAME — a refusal, never refs="0" + bool blobsUnread = false; // K75: the sweep's blob batch was unread — passed through, a refusal crossref::StrayResult stray; // the full sweep (headSha/refsScanned/mergedRefs/refs) std::vector scouted; // indices into stray.refs: the landing set fed to merge-scout std::vector bounded; // indices into stray.refs: unmerged, but cut by the size bound @@ -126,6 +127,7 @@ inline PlanResult computePlan( const std::string& root, std::string_view filter, result.nonGitRoot = result.stray.nonGitRoot; result.tooManyRefs = result.stray.tooManyRefs; result.filterMatchedNothing = result.stray.filterMatchedNothing; + result.blobsUnread = result.stray.blobsUnread; return result; } diff --git a/src/lsp.h b/src/lsp.h index f5a2fcdad..634e40c1b 100644 --- a/src/lsp.h +++ b/src/lsp.h @@ -99,7 +99,7 @@ inline LspFrame lspReadMessage( std::FILE* in ) std::string low = head; std::transform( low.begin(), low.end(), low.begin(), - [ ]( unsigned char c ) { return static_cast( std::tolower( c ) ); } ); + [ ]( char c ) { return static_cast( std::tolower( static_cast( c ) ) ); } ); // char in, explicit narrowing: an unsigned char parameter is G1's implicit sign-change on a byte >= 0x80 const std::size_t k = low.find( "content-length:" ); if( k == std::string::npos ) { f.eof = true; f.malformed = true; return f; } std::size_t len = 0; @@ -681,7 +681,7 @@ inline std::string lspWorkspaceSymbol( const rw::IngestResult& ing, const std::s const auto lowerCopy = [ ]( std::string s ) { std::transform( s.begin(), s.end(), s.begin(), - [ ]( unsigned char c ) { return static_cast( std::tolower( c ) ); } ); + [ ]( char c ) { return static_cast( std::tolower( static_cast( c ) ) ); } ); // char in, explicit narrowing: an unsigned char parameter is G1's implicit sign-change on a byte >= 0x80 return s; }; const std::string qLower = lowerCopy( std::string( query ) ); diff --git a/src/main.cpp b/src/main.cpp index ee4a09577..ef956a3e7 100644 --- a/src/main.cpp +++ b/src/main.cpp @@ -89,6 +89,7 @@ static_assert( rw::kTestGateCcxBarMirror == rw::quality::kCcxBar, "situ.h kTestG #include "didyoumean.h" // §P12.1 / §B6 M8: the ONE near-miss suggester, now shared with the MCP refusal table #include "selectorrefuse.h" // §B4.2: the ONE file:name selector not-found refusal — all six SYM-taking verbs #include "gitmine.h" +#include "historyread.h" // K51: the disclosure of a git history walk that could not be read (churn=/amp=) #include "ownersview.h" // §P6.4: countUniformOwnership/ownershipRowsToPrint — shared with mcpverbs.h's `owners` verb #include "mention.h" // B8: query-mention anchoring — files/modules/symbols NAMED in the --for text #include "siblift.h" // r4 EXPERIMENT: env-gated same-directory sibling lift (inert by default) @@ -1080,6 +1081,12 @@ inline std::FILE* openTokenBudgetBuffer( rw::MemoryStream& stream, TokenBudgetSt // here the same way the layer will compact it and repriced by the same rule (compactlegend.h compactRepricedTokens), // so the number decided on is the number the root prints. A body the dialect cannot shape keeps its own price — an // empty one (the map streamed unbuffered) and a --json one (no posture) included. +// map-inventory-d7: the number decided on is READ from the compacted body — the root's est_tokens=, or the map header's +// field when --stable keeps the root free of it — never recomputed from compacted.size(). That recount priced the bytes +// AFTER the layer had rewritten the est digits themselves (a 5-digit full price repriced to 4 digits is 2 bytes shorter +// in the root and the header), so on that boundary the gate said one token less than the header it was asserting +// (tokenbudgetcheck #9: src --top-k=233 on 7608d1f6, header 9965, gate 9964). The recount stays only as the fallback for +// a compacted body that prints no price at all. static std::size_t compactPostureMapPrice( const rw::Config& cfg, std::string_view body, std::size_t fullEstTokens ) { if( cfg.legend != "compact" || fullEstTokens == 0 ) @@ -1091,6 +1098,29 @@ static std::size_t compactPostureMapPrice( const rw::Config& cfg, std::string_vi { return fullEstTokens; } + const std::string_view doc = compacted; + const rw::CompactRootInfo root = rw::findCompactRoot( doc ); + if( const std::size_t printed = root.tag.empty() ? 0 : rw::rootUnsignedAttr( doc.substr( root.openBegin, root.openEnd - root.openBegin ), "est_tokens" ); + printed > 0 ) + { + return printed; + } + const std::string_view header = rw::compactMapHeader( doc ); + if( const std::size_t at = header.find( " est_tokens=" ); at != std::string_view::npos ) + { + std::size_t v = 0; + bool any = false; + for( std::size_t i = at + 12; i < header.size() && header[ i ] >= '0' && header[ i ] <= '9'; ++i ) + { + v = v * 10 + std::size_t( header[ i ] - '0' ); + any = true; + } + if( any && v > 0 ) + { + return v; + } + } + // no printed price to read (a body whose root and header carry none): the recount is the only number there is return static_cast( rw::compactRepricedTokens( static_cast( fullEstTokens ), body.size(), compacted.size() ) ); } @@ -1205,6 +1235,28 @@ inline std::string churnDecayWindowLabel( std::string_view minedSpan ) return label; } +// The crawl-scope flags a pasted continuation must replay so it indexes the SAME files this run did: --exclude, +// --max-file-size, --no-ignore, --ignore-tests (the enumeration below says why these four and no others). Each is +// returned with its leading space; empty at the defaults. Shared by scopedMapNextInvocation and leanMapPointer. +inline std::string crawlScopeFlags( const rw::Config& cfg ) +{ + std::string inv; + for( const std::string& x : cfg.excludes ) + { + inv += " " + rw::nextFlag( "--exclude=", x ); + } + // The ceiling is replayed as the BYTE COUNT it resolved to, not as the caller's "2K": parseByteSize accepts + // plain digits, and the number is what shaped the crawl. Omitted at the default, so the common hint is + // unchanged to the byte. + if( cfg.maxFileBytes != rw::kDefaultMaxFileBytes ) + { + inv += " --max-file-size=" + std::to_string( cfg.maxFileBytes ); + } + if( cfg.noIgnore ) { inv += " --no-ignore"; } + if( cfg.ignoreTests ) { inv += " --ignore-tests"; } + return inv; +} + // C1-b (2026-09-12): --in=DIR's page of rows, from the SAME mining pass as the global block. The prefix is // matched on the ROOT-RELATIVE spelling the map prints (serialize.h pathRel: rootRelativeUri against the crawl root as // typed), so a scoped row is byte-identical to its global twin — a sub-root-relative spelling missed every held-out gold @@ -1268,19 +1320,7 @@ inline std::string scopedMapNextInvocation( const rw::Config& cfg, std::string_v { inv += " " + rw::nextFlag( "--since=", cfg.since ); } - for( const std::string& x : cfg.excludes ) - { - inv += " " + rw::nextFlag( "--exclude=", x ); - } - // The ceiling is replayed as the BYTE COUNT it resolved to, not as the caller's "2K": parseByteSize accepts - // plain digits, and the number is what shaped the crawl. Omitted at the default, so the common hint is - // unchanged to the byte. - if( cfg.maxFileBytes != rw::kDefaultMaxFileBytes ) - { - inv += " --max-file-size=" + std::to_string( cfg.maxFileBytes ); - } - if( cfg.noIgnore ) { inv += " --no-ignore"; } - if( cfg.ignoreTests ) { inv += " --ignore-tests"; } + inv += crawlScopeFlags( cfg ); if( withIn ) { inv += " " + rw::nextFlag( "--in=", scopeDir ); @@ -1525,6 +1565,83 @@ inline constexpr const char* kMapRidesAlongFmt = " note=\"the ranked top-{} map below rides along with the requested payload; --top-k=0 for the payload " "alone (--top-k=1 for a minimal map)\""; +// ── expand-lean-k64: --expand/--outline whose requested definitions were ALL served ride no map ───────────────────── +// A multi-definition name used to ship the generic top-200 map beside bodies that already answer it (0 of 185 rows named +// the symbol). When every requested definition is served there is nothing left to disambiguate: top-k defaults to 0, the +// root says so (topk_default="0", the exact-name default's own disclosure) and carries ONE recoverable pointer to the map +// it did not ship (map_next=, a shell line: the map is the default verb of `ripwire ROOT`). The bodies' headers already +// carry p= l= t= for every definition. When some were NOT served (cut by the byte budget, unreadable), the root names +// exactly those — unserved_total= (the true count) and unserved_next= (the --expand/--outline call serving them, quoted by +// the next= house rule) — never the generic map. kMaxUnservedSelectors is a blow-up guard (the kMaxExpandSibs precedent), +// not a target: past it unserved_next= serves the FIRST kMaxUnservedSelectors and unserved_listed= says how many it lists; +// the rest are recovered by the same call with a larger --pack-budget-bytes (the legend says so). Language-neutral: the +// decision reads the served-id record, not any language's syntax. An EXPLICIT --top-k keeps today's shape. +inline constexpr std::size_t kMaxUnservedSelectors = 16; + +// `ripwire ROOT…`: EVERY root this call crawled (a multi-root call's map is the map of all of them — an empty mapRootArg +// there used to fall back to ".", the parent tree), each quoted by the next= house rule, plus the crawl-scope flags that +// decided which files were indexed, so the pasted call maps the same corpus. Presentation and payload flags stay out. +inline std::string leanMapPointer( const rw::Config& cfg ) +{ + std::string inv = "ripwire"; + for( const std::string_view root : cfg.roots ) + { + inv += " " + rw::nextFlag( "", root ); + } + if( cfg.roots.empty() ) + { + inv += " ."; + } + return inv + crawlScopeFlags( cfg ); +} + +struct UnservedDefs +{ + std::size_t requested = 0; // valid, non-module-scope requests + std::size_t unserved = 0; // of those, the ones with no body/outline in the answer + std::string attrs; // ` unserved_total="N" [unserved_listed="K"] unserved_next="--expand=P:L:N,…"` — empty when unserved == 0 +}; + +inline UnservedDefs unservedDefsOf( const rw::IngestResult& ing, const std::vector& requested, std::vector served, + std::string_view verbFlag, std::string_view rootArg ) +{ + UnservedDefs u; + std::sort( served.begin(), served.end() ); + const std::string rootPrefix = rootArg.empty() ? std::string() : rw::sarif::rootPrefixOf( rootArg ); + std::string selectors; + std::size_t listed = 0; + for( rw::NodeId id : requested ) + { + if( id >= ing.symbols.size() || ing.symbols[ id ].kind == rw::SymKind::ModuleScope ) + { + continue; + } + ++u.requested; + if( std::binary_search( served.begin(), served.end(), id ) ) + { + continue; + } + ++u.unserved; + if( listed >= kMaxUnservedSelectors ) + { + continue; + } + const rw::Symbol& sym = ing.symbols[ id ]; + const std::string_view path = rootArg.empty() ? std::string_view( ing.files[ sym.fileId ] ) : rw::sarif::rootRelativeUri( ing.files[ sym.fileId ], rootPrefix ); + selectors += ( listed == 0 ? "" : "," ); + selectors += std::string( path ) + ":" + std::to_string( sym.line ) + ":" + sym.name; + ++listed; + } + if( u.unserved > 0 ) + { + // unserved_listed= only when the guard cut the list: unserved_next= then serves the first `listed`, not all of them. + u.attrs = " unserved_total=\"" + std::to_string( u.unserved ) + "\"" + + ( listed < u.unserved ? " unserved_listed=\"" + std::to_string( listed ) + "\"" : std::string() ) + + rw::nextAttrXml( rw::nextFlag( std::string( verbFlag ) + "=", selectors ), "unserved_next" ); + } + return u; +} + // ── ONE PRICE FOR BOTH SERVING CANDIDATES (CodeRabbit, PR #215, second round on this comparison) ───── // THE DEFECT was not a missing addend, it was two counters. The bundle candidate was priced to the byte // (envelope, root attributes, the unproven residue, the map, the rendered , the closing tag) by the @@ -1872,6 +1989,7 @@ int runDefaultMap( const MainDispatch& d ) mapAnn.viaLegendStripped = cfg.legend == "compact" && !cfg.json; // FE-B: the rewrite would strip the map's via comment mapAnn.notesDegraded = d.notesDegraded; // L3 follow-up (CodeRabbit 4053600616): onto every this run emits mapAnn.codeFirstRows = isDefaultMapScope; // the code-first row pick + its data_sections_cut= / next= (serialize.h) + mapAnn.inventoryTier = isDefaultMapScope && !cfg.noInventory; // D7: the tier (mapInventoryWanted also drops it under --max-tokens) // C1-b (2026-09-12): --in=DIR — the scoped block and the map stub, filled by assignment like seed. The two next= strings // outlive every serialize() call below (mapAnn holds views into them). The scoped next= is the SAME run at the next // offset, page size carried when the caller set one; the stub's next= is the same run without in= (the map it stubbed). @@ -2319,21 +2437,64 @@ int runDefaultMap( const MainDispatch& d ) // measurement). Built only when a body is expanded; the same vector rides the two emissions below, so the // charged render and the emitted one cannot differ. const std::vector calleeOrder = expandNodes.empty() ? std::vector{} : rw::calleeNameSpecificity( ing ); + EmittedBodies expandRecord; // expand-lean-k64: which requested bodies the render above actually emitted if( !expandNodes.empty() ) { bodiesSection = rw::chargeSection( [ & ]( std::FILE* f ) - { packBodies( f, ing, expandNodes, cfg.packBudgetBytes, g.outOff, g.outTargets, g.outNameOnly, cfg.compress, redactPtr, - expandRanges.empty() ? nullptr : &expandRanges, d.notesPtr, /*outEmitted=*/nullptr, + { expandRecord = EmittedBodies{}; // a re-render starts from empty: the record is what THIS render emitted + packBodies( f, ing, expandNodes, cfg.packBudgetBytes, g.outOff, g.outTargets, g.outNameOnly, cfg.compress, redactPtr, + expandRanges.empty() ? nullptr : &expandRanges, d.notesPtr, /*outEmitted=*/&expandRecord, /*truncateOversizedFirst=*/true, /*withFileContext=*/true, mapRootArg, &calleeOrder ); }, // V1: octocode F2 sibs=/inc= rw::kBytesPerTokenBody ); } + std::vector outlineShown; // expand-lean-k64: the outlines the render below emitted if( !outlineNodes.empty() ) { outlineSection = rw::chargeSection( [ & ]( std::FILE* f ) - { packOutline( f, ing, outlineNodes, cfg.packBudgetBytes, cfg.compress, redactPtr, mapRootArg ); }, + { outlineShown.clear(); + packOutline( f, ing, outlineNodes, cfg.packBudgetBytes, cfg.compress, redactPtr, mapRootArg, &outlineShown ); }, rw::kBytesPerTokenBody ); } + // expand-lean-k64 (see kMaxUnservedSelectors above): a payload verb whose requested definitions were ALL served rides no + // map. Decided HERE, after the payload rendered, because "all served" is a property of the render; nothing between the + // exact-name default above and this point reads mapTopK. Same exclusions as the exact-name default (composed verbs claim + // the map for their own purpose; --max-tokens sizes the map; --json has its own twin), plus an explicit --top-k. + // Scope: every --expand the exact-name default above did not take (one ambiguous name, several tokens, several tokens + // naming one definition) and ONE --outline token (any count — a single-definition --outline rode the same map). + // --outline with several tokens keeps the ordinary default. Consequence: wherever the M6 mode choice runs, the map is + // already gone, so its bundleDoc.mapBytes term below is 0 from the CLI (kept: the price must stay honest if a future + // posture rides a map there). + std::string ctxLeanAttr = exactNameExpandDefault ? std::string( " topk_default=\"0\"" ) : std::string(); + std::string ctxUnservedAttr; + const bool leanEligible = !exactNameExpandDefault && !cfg.topKExplicit && !cfg.packSignatures && cfg.packTopN == 0 && cfg.query.empty() + && !cfg.adaptive && cfg.maxTokens == 0 && !cfg.json + && ( ( !cfg.expand.empty() && cfg.outline.empty() && !expandNodes.empty() && bodiesSection.isRendered ) + || ( cfg.outline.size() == 1 && cfg.expand.empty() && !outlineNodes.empty() && outlineSection.isRendered ) ); + if( leanEligible ) + { + std::vector served; + if( cfg.expand.empty() ) + { + served = outlineShown; + } + else + { + for( const EmittedBody& b : expandRecord.kept ) + { + served.push_back( b.id ); + } + } + const UnservedDefs un = unservedDefsOf( ing, cfg.expand.empty() ? outlineNodes : expandNodes, std::move( served ), + cfg.expand.empty() ? "--outline" : "--expand", mapRootArg ); + if( un.requested > 0 ) + { + mapTopK = 0; + ctxLeanAttr = " topk_default=\"0\"" + nextAttrXml( leanMapPointer( cfg ), "map_next" ); + ctxUnservedAttr = un.attrs; + } + } + // The --expand fallback: estimateExpandBodyTokens is no longer the primary estimate (measured bytes are), // but it remains the honest answer on the degrade path — better than dropping the block out of the // charge entirely, which is the defect this whole item is about. @@ -2378,7 +2539,7 @@ int runDefaultMap( const MainDispatch& d ) // the INITIAL value so it survives even when expandAutoServeScope below does not run (a range slice, or // a pre-render degrade) — every path exactNameExpandDefault can reach ends up with SOME to // decorate, since hasExtension is provably true whenever --expand is non-empty. - std::string ctxOpenStr = exactNameExpandDefault ? "" : ""; + std::string ctxOpenStr = ""; // R-E fix (2026-08-19): the payload document root DISCLOSES the root its p= are relative to, on the two // shapes where the ride-along map's is NOT there to do it — no map at all (mapTopK==0, which // the exact-name --expand default always picks) and whole-file mode. The first R-E landing made @@ -2448,11 +2609,11 @@ int runDefaultMap( const MainDispatch& d ) // because every byte on that path is raw code text — the same rate and the same pricedRootAttr fixpoint // the whole-file emission below applies. topk_default="0" and the unproven residue ride BOTH openers, so // the first is charged to both documents and the second is passed as the shared root bytes. - const std::size_t topkDefaultBytes = exactNameExpandDefault ? ( sizeof( " topk_default=\"0\"" ) - 1 ) : 0; + const std::size_t topkDefaultBytes = ctxLeanAttr.size(); ExpandServeDocument bundleDoc; bundleDoc.payloadBytes = bodiesSection.xml.size(); bundleDoc.mapBytes = mapTopK > 0 ? measureEmittedMapBytes( mapTopK, payloadTokens ) : 0; - bundleDoc.rootAttrBytes = ctxRootBytesWhenNoMap + topkDefaultBytes + noteBytes; // root=/est_tokens= only where no map carries them; note= only where a map does + bundleDoc.rootAttrBytes = ctxRootBytesWhenNoMap + topkDefaultBytes + ctxUnservedAttr.size() + noteBytes; // root=/est_tokens= only where no map carries them; note= only where a map does ExpandServeDocument fileDoc; fileDoc.payloadBytes = wholeFile.xml.size(); // as EMITTED, not the raw file bytes fileDoc.rootAttrBytes = ctxRootAttr.size() + topkDefaultBytes + ctxNotesDegradedBytes; // whole-file mode always carries root= (no rides with it) and, when degraded, the marker too @@ -2488,9 +2649,9 @@ int runDefaultMap( const MainDispatch& d ) // the ROOT ATTRIBUTES ride too: the compact legend reads root=/est_tokens=/topk_default= etc. present-only, so a // candidate without them would price a legend shorter than the one delivered. The est_tokens= values are the // candidates' own prices (the layer reprices them, and a placeholder of another digit count would move the delta). - const std::string topk = exactNameExpandDefault ? " topk_default=\"0\"" : ""; + const std::string topk = ctxLeanAttr; const std::string fileEst = std::to_string( static_cast( double( wholeFile.xml.size() + kExpandWholeFileLegend.size() + ctxRootAttr.size() ) / rw::kBytesPerTokenBody ) + 1 ); - const std::string bundleRoot = ""; + const std::string bundleRoot = ""; const std::string fileRoot = ""; // in the order the bundle is EMITTED — the bodies first, the ride-along map after them: the layer reads its // head terms off the root's first child, so a map-first candidate priced a different legend (rv-r1-L1-2: 107 B) @@ -2505,13 +2666,14 @@ int runDefaultMap( const MainDispatch& d ) ASSUME( ctxOpenStr.rfind( "" — insert // right after " page from the bundle + // the pack-task row below must keep serving. + { "candidate-page", []( const rw::Config& c ) noexcept { return ( !c.forTask.empty() || c.packTaskFlag ) && c.tokenBudget != 0 && ( c.pageLimit > 0 || c.pageOffset > 0 ); }, CompactKeyMark::RootAttr, "above_cliff" }, { "pack-task", []( const rw::Config& c ) noexcept { return c.packTaskFlag || !c.packTask.empty(); }, CompactKeyMark::RootAttr, "budget_tokens" }, { "from-trace", []( const rw::Config& c ) noexcept { return !c.fromTrace.empty() || !c.runTrace.empty(); }, CompactKeyMark::RootAttr, "task" }, { "expand", []( const rw::Config& c ) noexcept { return !c.expand.empty(); }, CompactKeyMark::None, {} }, @@ -3847,7 +4013,11 @@ static bool nativeCompactLegendVerb( const rw::Config& c ) noexcept // L1 fix round (rv-r1-L1 LOW-1): --batch outranks --for in dispatch, so `--for=X --batch=F` answers the batch envelope — // an answer this layer shapes. Only a run --for actually answers is skipped. const bool batchAnswers = !c.batchFile.empty(); // the batch envelope answers, and this layer shapes it - return !batchAnswers && !c.forTask.empty(); + // the budgeted-bundle CANDIDATE PAGE (#294/PR #362) is NOT the --for bundle this skip protects: + // its legend is pure prose (no data comment), so the layer's rewrite is safe — and required, or + // --legend=compact is accepted and ignored on the page (the #362 review's item 5). + const bool forCandidatePage = !c.forTask.empty() && c.tokenBudget != 0 && ( c.pageLimit > 0 || c.pageOffset > 0 ); + return !batchAnswers && !c.forTask.empty() && !forCandidatePage; } // L1 fix round (rv-r1-L1 LOW-3): a DEFAULTED posture captured every run through a tmpfile, a 1.28 MB `--lint --sarif` @@ -5086,6 +5256,7 @@ static int dispatchMain( const rw::Config& cfg, char** argv ) DISCLOSE( Diagnostics::answerRefused, "main: over the memory guard's hard limit between workspace roots — exit 5, one stderr line" ); rw::emitTo( stderr, "ripwire: {}\n", rw::memguard::hardStopLine( "ingest of workspace root " + std::to_string( parts.size() ) + " of " + std::to_string( ws.size() ) + " (" + r.label + ")" ) ); + rw::abandonGitWalks(); // the refusal is out: do not wait for the history walks' futures (gitmine.h) return 5; } } @@ -5110,10 +5281,12 @@ static int dispatchMain( const rw::Config& cfg, char** argv ) rw::memguard::answerStops(); // the refusal answers for any stop the ingest recorded DISCLOSE( Diagnostics::answerRefused, "main: over the memory guard's hard limit after the ingest — exit 5, one stderr line" ); rw::emitTo( stderr, "ripwire: {}\n", rw::memguard::hardStopLine( "ingest" ) ); + rw::abandonGitWalks(); // the refusal is out: do not wait for the history walks' futures (gitmine.h) return 5; } if( const int rc = memoryStopExit( ing, cfg, verbPrec.winner ); rc != 0 ) { + rw::abandonGitWalks(); // same: historyWalks joins on this return, and the answer it fed is refused return rc; } if( cfg.ignoreTests ) @@ -5182,6 +5355,7 @@ static int dispatchMain( const rw::Config& cfg, char** argv ) { DISCLOSE( Diagnostics::answerRefused, "main: over the memory guard's hard limit after the graph build — exit 5, one stderr line" ); rw::emitTo( stderr, "ripwire: {}\n", rw::memguard::hardStopLine( "graph build" ) ); + rw::abandonGitWalks(); // the refusal is out: do not wait for the history walks' futures (gitmine.h) return 5; } @@ -5252,6 +5426,23 @@ static int dispatchMain( const rw::Config& cfg, char** argv ) // sets concatenate (commits are disjoint across repos) and churn accumulates per file. ASSUME( historyWalks.size() == ( multiRoot ? ws.size() : 1u ), "this block runs under startHistoryWalks' condition (--help-task has already answered), so every root's walk was started" ); + // K51: a walk git could not finish arrives flagged (it is never cached either); the document says so (historyread.h). + HistoryReadReport& historyReport = historyRead(); + const auto joinWalk = [ & ]( std::size_t r ) + { + quality::HistoryWalk walk = historyWalks[ r ].get(); + if( walk.raw.unread && walk.raw.notRun ) + { + DISCLOSE( historyReport, HistoryReadReport::DisclosureWhy::WalkNotRun, + "main: git did not run for the history walk — churn= is absent and amp= counts callers only, and the document says so" ); + } + else if( walk.raw.unread ) + { + DISCLOSE( historyReport, HistoryReadReport::DisclosureWhy::WalkStopped, + "main: the git history walk stopped part-way — churn= and amp= count only the commits read (floors), and the document says so" ); + } + return walk; + }; std::vector> commits; if( multiRoot ) { @@ -5261,7 +5452,7 @@ static int dispatchMain( const rw::Config& cfg, char** argv ) // Y2: the memoized form — skips the 431 ms `git log --name-only` walk on a // warm (repo, HEAD sha, window, boundary-sha) hit; see quality.h's qchurn family. std::vector> part = - quality::resolveHistoryWalk( historyWalks[ r ].get(), ing, + quality::resolveHistoryWalk( joinWalk( r ), ing, cfg.forTask.empty() ? 0u : 12u, cfg.forTask.empty() ? nullptr : &rootChurn, r ); for( std::vector& c : part ) @@ -5284,7 +5475,7 @@ static int dispatchMain( const rw::Config& cfg, char** argv ) else { // Y2: memoized — see the multi-root branch above. - commits = quality::resolveHistoryWalk( historyWalks[ 0 ].get(), ing, + commits = quality::resolveHistoryWalk( joinWalk( 0 ), ing, cfg.forTask.empty() ? 0u : 12u, cfg.forTask.empty() ? nullptr : &forChurn ); } @@ -5360,6 +5551,9 @@ static int dispatchMain( const rw::Config& cfg, char** argv ) ampPtr, cboPtr, testedPtr, lcom4Ptr, impurePtr, forChurn, redactCounts, redactPtr, notesPtr, notesDegraded, grepPhases.valid ? &grepPhases : nullptr, needsValueUses }; + // Every verb from here on answers from the built graph: its query and its emit are this scope (the self-profile's + // "after the graph" share; the verbs' own scopes nest inside it). + PROFILE_SCOPE_DESCRIBE( "main: query + emit (the verb, after the graph)" ); if( std::optional handled = runForLens( dsp ) ) { return *handled; diff --git a/src/mapinventory.h b/src/mapinventory.h new file mode 100644 index 000000000..b80eee816 --- /dev/null +++ b/src/mapinventory.h @@ -0,0 +1,386 @@ +#pragma once +// mapinventory.h — the default map's INVENTORY TIER (): every indexed file the ranked rows did not show is +// named (code) or counted (tests, docs, examples, config), and the program entry points are named. +// +// WHY. The ranked map prints the top-k symbols (200 by default), which covers a few dozen files of a few hundred. On +// orient questions ("map this codebase: subsystems, key files, entry points") that cut WAS the miss: subsystem files +// ranked just below it (tmux input.c #208, key-bindings.c #202 of 6361) were absent with nothing saying they existed. +// Entry points are structurally worse: `main` has no in-repo caller, so PageRank ranks it at the bottom (htop main +// #2548 of 3206) and no re-rank can surface it without demoting something else. An inventory is complete and +// disclosed where a re-rank would only move the miss. +// +// WHAT. computeMapInventory is the ONE selection both dialects render (serialize.h: the XML element and the +// JSON "inv" object), so they cannot disagree on what was listed. It reads only the index and the call graph; it +// never touches the rank vector or the ranked rows (the tier is appended after them). +// • entries — a def named `main` by each language's OWN entry convention, never by the name alone: +// – a Function `main` with no in-repo caller in a language whose entry IS such a function (C, C++, ObjC, Go, +// Rust, Kotlin, Dart); +// – a Method `main`/`Main` with no in-repo caller only where the entry is a static method (Java, C#); +// – a Function `main` in a module-scope language (Python, JS, TS, Ruby, Bash, PHP, Lua) only when its OWN +// file's module scope calls it and nothing else does (Python's `if __name__ == "__main__": main()`, Bash's +// `main "$@"`, a script's top-level `main()`): an uncalled `main` there is a library function or dead code; +// – a Python `__main__.py` module scope. +// A method named main anywhere else (a Python/JS/Ruby class member) is never an entry. Any other language: no +// claim (a missed entry is disclosed by the legend; a false one would not be). Outside test paths and the +// rolled-up directories below. Not detected (stated in the legend): entries declared in a manifest +// (package.json main/bin/exports, console_scripts, Cargo [[bin]]) or by an attribute (Swift @main, Kotlin +// @JvmStatic) — the index does not read those as entry declarations. +// • ls rows — per directory, the unshown files: code files NAMED (basename), every other file COUNTED. A file under +// a test/bench/fixture/example/doc directory rolls up to that directory (counted, never named): those trees are +// large (textual's docs/examples, tmux's regress) and are not where an orient answer's key files live. +#include +#include +#include +#include +#include + +#include "model.h" +#include "filter.h" // isTestSymbol — the shared test-path convention +#include "resolve.h" // includerDir — the shared directory-of-a-path splitter +#include "infra/namesplit.h" // namesplit::afterLast — the shared basename splitter +#include "nextverb.h" // nextFlag — the one shell-safe next= argv composer + +namespace rw +{ + +struct MapInventoryDir +{ + std::string dir; // root-relative directory ("." for the root); a rolled-up subtree's top dir + std::uint32_t total = 0; // unshown files under it + std::vector named; // the code files among them (file ids, path order) +}; + +struct MapInventory +{ + bool active = false; // false ⇒ the tier is not emitted at all (byte-identical map) + std::size_t listed = 0; // files the ranked rows showed (the map's groups) + std::size_t unlisted = 0; // every other indexed file; listed + unlisted = files= + std::vector entries; // path order, then symbol id; at most kInventoryEntryCap + std::size_t entryTotal = 0; // every entry found (entries.size() < entryTotal ⇒ cut, disclosed) + std::vector dirs; // path order + std::size_t namesTotal = 0; // every unshown code file a directory row could name + bool namesCapped = false; // the name ceiling left some of them counted, not named (disclosed) + // Recovery of a cut (PROCESS rule 5: every cut recoverable via next=): the argv fragment, on the same root, that + // lists what the guard left out. Empty unless that guard fired. + std::string entriesNext; // --graph-query over every def that could be a cut entry (a superset) + std::string namesNext; // --tree over every indexed file (a superset of the count-only names) +}; + +// RUNAWAY GUARDS (PROCESS rule 5a), far above any measured answer: the eleven orient repos of the round-1 table name at +// most 199 files (htop) and list at most 5 entries (gotestsum); golang.org/x/tools, a Go tool monorepo, has 60 real +// entries and golang.org/x/text 31, so the entry cap sits well above both. A monorepo of thousands of unshown code +// files would otherwise print every name (~13 B each). Past the ceiling, directories in path order keep n= and drop f= +// (names_capped="1" names_total=N names_next=); past the entry cap, entries_capped="1" entries_total=N entries_next=. +// Each next= is an argv on the same root that lists what was cut (a superset of it), so neither cut is a dead end. +inline constexpr std::size_t kInventoryNameCeiling = 2000; +inline constexpr std::size_t kInventoryEntryCap = 256; + +// Directory components whose subtree rolls up count-only. pathTierOf's test/bench/fixture conventions plus the +// presentational ones (examples, docs, regress, samples) — local to the inventory on purpose: widening pathTierOf +// itself would re-tier --grep/--callers ordering on every verb. +inline constexpr std::string_view kInventoryRollupDirs[] = { + "test/", "tests/", "__tests__/", "spec/", "specs/", "e2e/", "bench/", "benches/", "benchmark/", "benchmarks/", + "fixture/", "fixtures/", "testdata/", "example/", "examples/", "_example/", "_examples/", "doc/", "docs/", + "regress/", "sample/", "samples/", +}; +// Deliberately NOT rolled up: integration/ and testing/. Both are real package names (hono's src/helper/testing is a +// library helper; an integration/ package is common), so their files stay named; a main under them is still judged +// by the entry rule. e2e/ is an end-to-end test tree by name and rolls up. Go's _examples/ convention (a leading +// underscore keeps it out of the module build) and benchmark/ (grpc-go's benchmark mains) roll up like examples/. + +// The rolled-up directory a root-relative path falls under: the prefix through the FIRST matching component, or empty. +inline std::string_view inventoryRollupDirOf( std::string_view rel ) noexcept +{ + std::size_t best = std::string_view::npos; + std::size_t bestLen = 0; + for( std::string_view seg : kInventoryRollupDirs ) + { + std::size_t pos = 0; + while( ( pos = rel.find( seg, pos ) ) != std::string_view::npos ) + { + if( pos == 0 || rel[ pos - 1 ] == '/' ) + { + if( pos < best ) { best = pos; bestLen = seg.size(); } + break; + } + ++pos; + } + } + if( best == std::string_view::npos ) + { + return {}; + } + return rel.substr( 0, best + bestLen - 1 ); // without the trailing '/' +} + +// A code symbol: anything a reader can open as a definition — not a data Section (doc heading, JSON/YAML key) and +// not the synthetic module-scope row every script file carries. +inline bool isInventoryCodeSymbol( const Symbol& s ) noexcept +{ + return s.kind != SymKind::Section && s.kind != SymKind::ModuleScope; +} + +// How a language spells its program entry (the header's entry rule). EntryFn: a caller-less Function `main` IS the +// entry. EntryStaticMethod: a caller-less Method `main`/`Main` is (Java/C# static main). ModuleScopeCall: a Function +// `main` is an entry only when its own file's module scope calls it. None: no claim. +enum class InventoryEntryStyle : std::uint8_t { None, EntryFn, EntryStaticMethod, ModuleScopeCall }; + +// The languages of each style, as data (a reader checks one list, not a switch). +inline constexpr Lang kInventoryEntryFnLangs[] = { Lang::C, Lang::Cpp, Lang::ObjC, Lang::Go, Lang::Rust, Lang::Kotlin, Lang::Dart }; +inline constexpr Lang kInventoryStaticMethodLangs[] = { Lang::Java, Lang::CSharp }; +inline constexpr Lang kInventoryModuleScopeLangs[] = { Lang::Python, Lang::JavaScript, Lang::TypeScript, Lang::Ruby, Lang::Bash, + Lang::Php, Lang::Lua }; + +inline constexpr InventoryEntryStyle inventoryEntryStyle( Lang l ) noexcept +{ + const auto in = [ l ]( const auto& langs ) noexcept { return std::find( std::begin( langs ), std::end( langs ), l ) != std::end( langs ); }; + return in( kInventoryEntryFnLangs ) ? InventoryEntryStyle::EntryFn + : in( kInventoryStaticMethodLangs ) ? InventoryEntryStyle::EntryStaticMethod + : in( kInventoryModuleScopeLangs ) ? InventoryEntryStyle::ModuleScopeCall + : InventoryEntryStyle::None; +} + +// A def whose name and kind fit its language's entry convention (callers are judged separately). +inline bool isInventoryEntryShape( const Symbol& s ) noexcept +{ + switch( inventoryEntryStyle( s.lang ) ) + { + case InventoryEntryStyle::EntryFn: return s.kind == SymKind::Function && s.name == "main"; + // a script language indexes a class member as a Function with its class as scope (Python `def main(self)`): + // only a TOP-LEVEL main is the script's entry + case InventoryEntryStyle::ModuleScopeCall: return s.kind == SymKind::Function && s.name == "main" && s.scope.empty(); + case InventoryEntryStyle::EntryStaticMethod: return s.kind == SymKind::Method && ( s.name == "main" || s.name == "Main" ); + case InventoryEntryStyle::None: break; + } + return false; +} + +inline bool isInventoryMainModule( const IngestResult& ing, const Symbol& s ) noexcept +{ + return s.kind == SymKind::ModuleScope && s.fileId < ing.files.size() + && namesplit::afterLast( rootRelPath( ing, s.fileId ), "/" ) == "__main__.py"; +} + +// The recovery argv for entries past the cap: a --graph-query naming every def the cut entries could be (each name a +// cut entry carries, plus the __main__.py module scopes when one was cut), --limit sized to every symbol it can match +// so its own window never cuts. A superset — it also lists non-entries of those names (tests, methods, callees). +inline std::string inventoryEntriesNext( const IngestResult& ing, const MapInventory& inv, std::size_t from ) +{ + bool on[ 3 ] = { false, false, false }; // a cut def named main, one named Main, a cut __main__.py scope + for( std::size_t i = from; i < inv.entries.size(); ++i ) + { + const Symbol& s = ing.symbols[ inv.entries[ i ] ]; + const bool scope = s.kind == SymKind::ModuleScope; + on[ 0 ] = on[ 0 ] || ( !scope && s.name == "main" ); + on[ 1 ] = on[ 1 ] || ( !scope && s.name == "Main" ); + on[ 2 ] = on[ 2 ] || scope; + } + static constexpr std::string_view kParts[ 3 ] = { "name(\"main\")", "name(\"Main\")", + "and(file(all,\"(^|/)__main__\\.py$\"),kind(all,modscope))" }; + std::string joined; + int n = 0; + for( int i = 0; i < 3; ++i ) + { + if( on[ i ] ) { joined += n++ ? "," : ""; joined += kParts[ i ]; } + } + std::size_t limit = 0; // every symbol the expression can match, so the query's own window never cuts + for( const Symbol& s : ing.symbols ) + { + limit += ( on[ 0 ] && s.name == "main" ) || ( on[ 1 ] && s.name == "Main" ) || ( on[ 2 ] && isInventoryMainModule( ing, s ) ); + } + return nextFlag( "--graph-query=", n > 1 ? "or(" + joined + ")" : joined ) + " --limit=" + std::to_string( limit ); +} + +// The program entries by each language's convention (the header's rule), outside the rolled-up directories and test +// code; and the __main__.py module scopes. Path order, at most kInventoryEntryCap kept (entryTotal counts them all; +// entriesNext lists the rest). +template +inline void collectInventoryEntries( MapInventory& inv, const IngestResult& ing, const std::vector& outOff, + const std::vector& outTargets, const PathRel& pathRel ) +{ + const std::size_t F = ing.files.size(); + const std::size_t S = ing.symbols.size(); + // test PATHS need no clause here: isTestSymbol (below) covers a candidate def, and every test directory isTestPath + // knows (test/, tests/, __tests__/) is a rolled-up directory, which covers a __main__.py scope + const auto entryEligibleFile = [ & ]( std::uint32_t f ) noexcept { return inventoryRollupDirOf( rootRelPath( ing, f ) ).empty(); }; + std::vector candidate( S, 0 ); + std::vector scopeCalled( S, 0 ); // its own file's module scope calls it + for( NodeId id = 0; id < S; ++id ) + { + const Symbol& s = ing.symbols[ id ]; + candidate[ id ] = isInventoryEntryShape( s ) && s.fileId < F && !isTestSymbol( ing, id ) && entryEligibleFile( s.fileId ); + } + // a candidate with an in-repo caller other than its own file's module scope is a library function, not an entry + for( NodeId caller = 0; caller < S; ++caller ) + { + const Symbol& c = ing.symbols[ caller ]; + for( std::uint32_t e = outOff[ caller ]; e < outOff[ caller + 1 ]; ++e ) + { + const NodeId t = outTargets[ e ]; + if( t >= S || !candidate[ t ] ) + { + continue; + } + if( c.kind == SymKind::ModuleScope && c.fileId == ing.symbols[ t ].fileId ) + { + scopeCalled[ t ] = 1; + } + else + { + candidate[ t ] = 0; + } + } + } + for( NodeId id = 0; id < S; ++id ) + { + const Symbol& s = ing.symbols[ id ]; + // a module-scope language's main is an entry only when that scope runs it; elsewhere the caller-less def is + const bool conventionHolds = inventoryEntryStyle( s.lang ) != InventoryEntryStyle::ModuleScopeCall || scopeCalled[ id ]; + const bool mainModule = isInventoryMainModule( ing, s ) && entryEligibleFile( s.fileId ); + if( ( candidate[ id ] && conventionHolds ) || mainModule ) + { + inv.entries.push_back( id ); + } + } + std::stable_sort( inv.entries.begin(), inv.entries.end(), [ & ]( NodeId a, NodeId b ) + { return pathRel( ing.symbols[ a ].fileId ) < pathRel( ing.symbols[ b ].fileId ); } ); + inv.entryTotal = inv.entries.size(); + if( inv.entries.size() > kInventoryEntryCap ) + { + inv.entriesNext = inventoryEntriesNext( ing, inv, kInventoryEntryCap ); + inv.entries.resize( kInventoryEntryCap ); + } +} + +// One row per directory over the unshown files, path order. A file under a rolled-up directory groups under that +// directory and is counted; elsewhere a code file (codeSyms > 0, not test-named) is named, every other file counted. +template +inline void collectInventoryDirs( MapInventory& inv, const IngestResult& ing, const std::vector& shownFile, + const std::vector& codeSyms, const PathRel& pathRel ) +{ + struct Keyed { std::string key; bool rolled; std::uint32_t f; }; + std::vector keyed; + for( std::uint32_t f = 0; f < ing.files.size(); ++f ) + { + if( shownFile[ f ] ) + { + continue; + } + // classify on the ROOT-RELATIVE path (a checkout living under a tests/ dir is not a test tree — #228), group + // and print in the display form every uses + const std::string_view rel = rootRelPath( ing, f ); + const std::string_view disp = pathRel( f ); + const std::string_view rolled = inventoryRollupDirOf( rel ); + if( rolled.empty() ) + { + const std::string_view dir = includerDir( disp ); // the shared directory splitter; the root's files group under "." + keyed.push_back( Keyed{ std::string( dir.empty() ? std::string_view( "." ) : dir ), false, f } ); + continue; + } + const std::size_t prefix = disp.ends_with( rel ) ? disp.size() - rel.size() : 0; + keyed.push_back( Keyed{ std::string( disp.substr( 0, prefix ) ) + std::string( rolled ), true, f } ); + } + std::stable_sort( keyed.begin(), keyed.end(), [ & ]( const Keyed& a, const Keyed& b ) + { return a.key != b.key ? a.key < b.key : pathRel( a.f ) < pathRel( b.f ); } ); + for( const Keyed& k : keyed ) + { + if( inv.dirs.empty() || inv.dirs.back().dir != k.key ) + { + inv.dirs.push_back( MapInventoryDir{ k.key, 0, {} } ); + } + MapInventoryDir& d = inv.dirs.back(); + ++d.total; + if( !k.rolled && codeSyms[ k.f ] > 0 && !isTestPath( rootRelPath( ing, k.f ) ) ) + { + d.named.push_back( k.f ); + } + } +} + +// The name ceiling: whole directories, path order — a directory is named completely or not at all, so f= is never a +// silent part-list; past the ceiling every later directory is count-only too (one contiguous cut). +inline void applyInventoryNameCeiling( MapInventory& inv ) noexcept +{ + for( const MapInventoryDir& d : inv.dirs ) + { + inv.namesTotal += d.named.size(); + } + std::size_t namedSoFar = 0; + for( MapInventoryDir& d : inv.dirs ) + { + if( namedSoFar + d.named.size() > kInventoryNameCeiling ) + { + inv.namesCapped = inv.namesCapped || !d.named.empty(); + d.named.clear(); + namedSoFar = kInventoryNameCeiling; + continue; + } + namedSoFar += d.named.size(); + } + ENSURES( namedSoFar <= kInventoryNameCeiling, "the printed names stay within the ceiling" ); +} + +// `shownFile[f]` != 0 ⇔ the ranked rows printed a group for file f. `pathRel` maps a file id to its display path +// (the same root-relative form every uses). +template +inline MapInventory computeMapInventory( const IngestResult& ing, const std::vector& outOff, + const std::vector& outTargets, const std::vector& shownFile, + const PathRel& pathRel ) +{ + const std::size_t F = ing.files.size(); + EXPECTS( shownFile.size() == F, "one shown bit per indexed file" ); + EXPECTS( outOff.size() >= ing.symbols.size() + 1 || ing.symbols.empty(), "outOff is the CSR offset array over the symbols" ); + MapInventory inv; + + // code symbols per file: a file with none is counted, never named + std::vector codeSyms( F, 0 ); + for( const Symbol& s : ing.symbols ) + { + if( s.fileId < F && isInventoryCodeSymbol( s ) ) + { + ++codeSyms[ s.fileId ]; + } + } + // Present-only: the tier rides a map whose ranked rows LEFT OUT a file with code in it. A map that shows every + // such file has nothing to inventory (its unshown files are empty or data-only) and stays byte-identical. + bool anyCut = false; + for( std::size_t f = 0; f < F; ++f ) + { + if( !shownFile[ f ] ) { ++inv.unlisted; } else { ++inv.listed; } + anyCut = anyCut || ( !shownFile[ f ] && codeSyms[ f ] > 0 ); + } + if( !anyCut ) + { + return MapInventory{}; + } + inv.active = true; + collectInventoryEntries( inv, ing, outOff, outTargets, pathRel ); + collectInventoryDirs( inv, ing, shownFile, codeSyms, pathRel ); + applyInventoryNameCeiling( inv ); + if( inv.namesCapped ) + { + // --tree lists every indexed file with a symbol, ranked, paging by its own next=; --limit= files= keeps it to + // one page. A superset of the count-only names (it repeats the ranked and the named files). + inv.namesNext = "--tree --limit=" + std::to_string( F ); + } + ENSURES( inv.listed + inv.unlisted == F, "every indexed file is listed or inventoried" ); + return inv; +} + +// The legend clause (full dialect). No double hyphen: it rides inside an XML comment (G4). +inline constexpr std::string_view kMapInventoryLegend = + ""; + +} // namespace rw diff --git a/src/mcp.h b/src/mcp.h index bd1fac0c8..470a53a4c 100644 --- a/src/mcp.h +++ b/src/mcp.h @@ -31,7 +31,7 @@ #include #include // std::find — the --mcp-tools duplicate check and the batch-served lookup #include // std::popcount — the --mcp-tools profile check -#include "infra/tablelookup.h" // findByField — mcpToolIndex, the same row lookup wrap and ingest use +#include "infra/tablelookup.h" // findIndexByField — mcpToolIndex, the same row lookup wrap and ingest use #include // ::realpath — the workspace-pin canonicalization (mcpCanonRoot) #include // PATH_MAX #include "infra/os.h" // rw::os::getcwd — R2a: the launch-cwd assumed root (resolved once at startup) @@ -141,8 +141,10 @@ inline constexpr McpToolMask kMcpAllToolsMask = ( McpToolMask{ 1 } << kMcpVerbCo // kMcpVerbTable's row for `name`, or kMcpVerbCount when the name is not an advertised tool. constexpr std::size_t mcpToolIndex( std::string_view name ) noexcept { - const McpVerbInfo* const row = findByField( kMcpVerbTable, &McpVerbInfo::name, name ); - return row == nullptr ? kMcpVerbCount : static_cast( row - kMcpVerbTable ); + // The INDEX-shaped lookup, not findByField's row pointer: g++ 15 rejected the pointer form's + // row-address arithmetic in constant evaluation under -fsanitize=address (the static_asserts + // below) — the same GCC class findIndexByField exists for (#347). Same values on every compiler. + return findIndexByField( kMcpVerbTable, &McpVerbInfo::name, name ); } // The named profiles. `members` is a comma list of tool names; "" means every tool. @@ -1374,7 +1376,7 @@ inline McpDispatchResult dispatchMcpLine( const std::string& line, int topK, boo + mcprefuse::toolMetadataFor( "edit_check", pathIsRequired ) + "}," // The cross-branch + dark-content verbs. Read-only git plumbing, no index // coupling for the first two (they read OTHER refs' blobs, which the index never ingested). - + mcprefuse::gitOnlyStanza( omitGitVerbs, "{\"name\":\"whereis\",\"description\":\"WHERE DOES THIS CONTENT LIVE? Which branch's tree defines or mentions a symbol, HEAD first, with on-head=0 naming the case this verb exists for: content that lives only on a branch (a finished fix stranded on 1 of 30 refs). Each distinct blob is read once (content-addressed), so N branches cost about one tree. kind=def on a REF row is a LEXICAL heuristic (ref blobs are raw text, never ingested) — for HEAD's parsed answer use find_symbol/fetch_body. symbol = the name, or an @FILE:LINE seed; kind = optional ref-name substring filter, echoed as filter=; limit/offset page the hits (first 60). at= is HEAD's sha; +dirty = changed paths read from the working tree (worktree=). Single-root; read-only.\"," + + mcprefuse::gitOnlyStanza( omitGitVerbs, "{\"name\":\"whereis\",\"description\":\"WHERE DOES THIS CONTENT LIVE? Which branch's tree defines or mentions a symbol, HEAD first, with on-head=0 naming the case this verb exists for: content that lives only on a branch (a finished fix stranded on 1 of 30 refs). Each distinct blob is read once (content-addressed), so N branches cost about one tree. kind=def is a parser's label on any row: HEAD's from the index, another ref's from parsing its blob; kind=text = a def-like line not parsed. symbol = the name, or an @FILE:LINE seed; kind = optional ref-name substring filter, echoed as filter=; limit/offset page the hits (first 60). at= is HEAD's sha; +dirty = changed paths read from the working tree (worktree=). Single-root; read-only.\"," + mcprefuse::toolMetadataFor( "whereis", pathIsRequired ) + "}," ) + mcprefuse::gitOnlyStanza( omitGitVerbs, "{\"name\":\"stray_content\",\"description\":\"Per branch: the lines its own divergent work AUTHORED (vs its merge-base with HEAD) that the live line does NOT have. Four verdicts (unmerged+superseded+merged+unknown=refs): v=unmerged is genuinely absent; v=superseded means the live line re-implemented the work — the case `git cherry` structurally cannot see; merged branches are omitted and counted; v=unknown is a branch this scan could NOT analyse at all (no merge-base, unrelated history), not a fourth kind of divergence. Every file row carries its raw del/redone/sim evidence. Line-granular, not semantic. kind = optional ref-name substring filter, echoed as filter=; limit/offset page the refs. Single-root; read-only.\"," + mcprefuse::toolMetadataFor( "stray_content", pathIsRequired ) + "}," ) + @@ -1498,6 +1500,10 @@ inline McpDispatchResult dispatchMcpLine( const std::string& line, int topK, boo // whereis arm, a non-string `listing:5` decoded to "" after that gate had run, its shape refusal was dropped, and the // closed-set refusal then told the caller it sent '' (CodeRabbit 5468003465; test/crossrefcheck.sh LEAN (L10b)). const std::string listingArg = strArg( "listing" ); + // grep: code|any (the CLI --grep-in twin). The `listing` sibling (CodeRabbit 5468003465, checklist 26): read inside the + // grep arm, a non-string `in:5` decoded to "" after the shape gate had run, its refusal was dropped, and the arm then + // answered the default tier as if nothing had been sent. test/greptiercheck.sh (9e). + const std::string grepInArg = strArg( "in" ); const std::string from = strArg( "from" ); // path verb: source symbol const std::string to = strArg( "to" ); // path verb: destination symbol const std::string trace = strArg( "trace" ); // L4 from_trace: the raw trace TEXT @@ -2206,7 +2212,7 @@ inline McpDispatchResult dispatchMcpLine( const std::string& line, int topK, boo // registers the field and the batch surface already refuses loudly. Both dialects now // read the value through grepInModeFromArg, so they cannot disagree about the set. rw::GrepIn grepInMode = rw::GrepIn::Code; - const std::string inRefusal = rw::grepInModeFromArg( strArg( "in" ), grepInMode ); + const std::string inRefusal = rw::grepInModeFromArg( grepInArg, grepInMode ); if( !inRefusal.empty() ) { resp = errResultMsg( -32602, inRefusal ); @@ -2288,10 +2294,9 @@ inline McpDispatchResult dispatchMcpLine( const std::string& line, int topK, boo // --token-budget beside --limit — the page has no byte ceiling to shape against. resp = pagedResult( [ & ]( McpPageArgs pg ) { - if( ( pg.limit > 0 || pg.offset > 0 ) && budgetArg.isPresent ) - { - return errResultMsg( -32602, "for: limit/offset select the file page, which has no token budget to shape against — drop budget_tokens, or drop limit/offset for the budgeted bundle" ); - } + // PAGING-POC follow-up (issue #294): budget + window no longer refuses — + // forTaskText serves the budgeted-bundle CANDIDATE PAGE for it (the CLI twin's + // own document); the budgetless FILE PAGE still answers a windowless-budget call. const std::optional answer = forTaskText( path, task, redactPtr, budgetArg.isPresent ? std::size_t( budgetArg.value ) : 0, noRoute, pg, sections ); // L2: "" (default) = stub; the validated closed set otherwise @@ -2323,11 +2328,17 @@ inline McpDispatchResult dispatchMcpLine( const std::string& line, int topK, boo resp = !listingRefusal.empty() ? errResultMsg( -32602, listingRefusal ) : pagedResult( [ & ]( McpPageArgs pg ) { bool seedFault = false; - const std::string t = whereisText( path, symbol, kind, crossref::kWhereisHits, pg, &seedFault, listing ); + bool blobFault = false; + const std::string t = whereisText( path, symbol, kind, crossref::kWhereisHits, pg, &seedFault, listing, &blobFault ); if( seedFault ) { return errResultMsg( -32602, mcprefuse::notFound( getIndex( path ).ing, "symbol", symbol ) ); } + if( blobFault ) + { + DISCLOSE( Diagnostics::answerRefused, "whereis (MCP): the blob stream did not serve every blob — a -32603 error names it, no hit count" ); + return errResultMsg( -32603, crossref::blobsUnreadCause( crossref::UnreadBlobBatch::Trees ) + " — no hit count rather than one nothing measured" ); + } return t.empty() ? errResult( -32602, "not a git repository (or no HEAD commit) — no refs to search" ) : textResult( t ); } ); } @@ -2336,7 +2347,13 @@ inline McpDispatchResult dispatchMcpLine( const std::string& line, int topK, boo // `kind` doubles as the optional ref-name substring filter (no dedicated arg needed). resp = pagedResult( [ & ]( McpPageArgs pg ) // M13 { - const std::string t = strayContentText( path, kind, crossref::kStrayFilesPerRef, pg ); + bool blobFault = false; + const std::string t = strayContentText( path, kind, crossref::kStrayFilesPerRef, pg, &blobFault ); + if( blobFault ) + { + DISCLOSE( Diagnostics::answerRefused, "stray_content (MCP): the blob batch did not serve every blob - a -32603 error names it, no verdicts" ); + return errResultMsg( -32603, crossref::blobsUnreadCause( crossref::UnreadBlobBatch::Branches ) ); + } return t.empty() ? errResult( -32602, "not a git repository (or no HEAD commit) — no refs to compare" ) : textResult( t ); } ); } @@ -2566,6 +2583,20 @@ inline McpDispatchResult dispatchMcpLine( const std::string& line, int topK, boo // `pack_task` is the dispatch-only alias (same branch, not separately advertised in tools/list above). else if( ( name == "explore" || name == "pack_task" ) && !path.empty() && !task.empty() ) { + // PAGING-POC (issue #294, ask (b)): explore's limit/offset — the same bounded pair the + // `for` twin takes (mcpPageArgs → pageWindow), refused-together with partition: a + // partitioned bundle has N+1 slices and no single page to walk. + const McpPageParse explorePage = mcpPageArgs( args ); + if( !explorePage.refusal.empty() ) + { + resp = errResultMsg( -32602, explorePage.refusal ); + } + else if( ( explorePage.page.limit > 0 || explorePage.page.offset > 0 ) && partitionCount > 0 ) + { + resp = errResult( -32602, "partition and limit/offset are mutually exclusive — a partitioned bundle has N+1 slices and no single page to walk" ); + } + else + { // W3FIX H4: `partition` used to be a bare `std::uint32_t( partitionArg )` cast — N3's // radius bug, one field over — so partition:4294967299 ran a 3-way fan-out and // partition:2^32 served a single bundle, both silently. The band is a DECLARED domain now @@ -2576,7 +2607,9 @@ inline McpDispatchResult dispatchMcpLine( const std::string& line, int topK, boo static_assert( kMcpRecallTopKMax == 1000, "the top_k refusal names the band 1..1000 in mcprefusal.h's kMcpValueFields and in the " "tools/list memory_recall stanza — move all three together" ); - resp = textResult( packTaskText( path, task, budgetTokens, redactPtr, partitionCount, noRoute ) ); + resp = textResult( packTaskText( path, task, budgetTokens, redactPtr, + PackTaskForm{ partitionCount, noRoute, explorePage.page } ) ); + } } // L4: `from_trace` — maps a pasted stack-trace/sanitizer/compiler-error TEXT onto indexed symbols // (fromTraceBundleText, tracelocus.h) — the SAME assembler --from-trace's CLI path calls. diff --git a/src/mcprefusal.h b/src/mcprefusal.h index 075069dbd..f068a73f0 100644 --- a/src/mcprefusal.h +++ b/src/mcprefusal.h @@ -1059,7 +1059,10 @@ inline constexpr McpVerbFields kMcpVerbFields[] = { { "affected", "path paths files legend" }, { "path_between", "path paths from to legend" }, { "connect", "path paths symbols radius legend" }, - { "explore", "path paths task budget_tokens partition legend no_route" }, + // PAGING-POC (issue #294, ask (b)): limit/offset are DECLARED because explore now HONORS them — the + // same file-grain widening page the `for` twin serves (one inputSchema row drives this table, the + // tools/list stanza and mcpw3fixcheck's enumeration together). + { "explore", "path paths task budget_tokens partition legend no_route limit offset" }, { "from_trace", "path paths trace budget_tokens legend" }, // 2026-09-10: limit/offset are DECLARED here because the verb now HONORS them (mcpPageArgs -> the // unflagged-row window in editcheck.h), the same rule the `impact`/`uses` rows above state. They diff --git a/src/mcpverbs.h b/src/mcpverbs.h index dfdfa611b..183af6438 100644 --- a/src/mcpverbs.h +++ b/src/mcpverbs.h @@ -449,7 +449,7 @@ inline std::string analyzeToString( const std::string& root, int topK, bool stab // W2-F: the map's convergence disclosure. The CLI map carries pr_iters= and // this one must too — "the clause landed at 3 of its 5 echo sites" is the // §B4 family, and mcpclidiffcheck is the gate that keeps the two surfaces one. - /*ann=*/rw::MapAnnotations{ .prDisclosure = ix.prDisclosure, .codeFirstRows = ix.isCleanWorkingSet }, + /*ann=*/rw::MapAnnotations{ .prDisclosure = ix.prDisclosure, .codeFirstRows = ix.isCleanWorkingSet, .inventoryTier = ix.isCleanWorkingSet }, /*statsFirstScreen=*/true, anRootArg, &ix.g.locPinOut, ix.g.externalCalls, &ix.g.declinedOut, ix.g.gateDeclinedCalls, ix.g.outNameOnly.empty() ? nullptr : &ix.g.outNameOnly ); } ); } @@ -517,7 +517,7 @@ inline std::string rankByText( const std::string& root, std::string_view mode, i ix.g.bindLabel.empty() ? nullptr : &ix.g.bindLabel, /*autoOrder=*/false, /*outEstTokens=*/nullptr, /*extraPayloadTokens=*/0, - /*ann=*/rw::MapAnnotations{ .rankByLabel = rankByLabel, .prDisclosure = disclosure, .codeFirstRows = rankByLabel == nullptr }, // pagerank: the map scope's code-first pick + /*ann=*/rw::MapAnnotations{ .rankByLabel = rankByLabel, .prDisclosure = disclosure, .codeFirstRows = rankByLabel == nullptr, .inventoryTier = rankByLabel == nullptr }, // pagerank: the map scope's code-first pick /*statsFirstScreen=*/true, rbRootArg, &ix.g.locPinOut, ix.g.externalCalls, &ix.g.declinedOut, ix.g.gateDeclinedCalls, ix.g.outNameOnly.empty() ? nullptr : &ix.g.outNameOnly ); } ); } @@ -560,7 +560,8 @@ inline std::string rankByText( const std::string& root, std::string_view mode, i // than answering a question the caller did not ask. inline std::string whereisText( const std::string& root, const std::string& symbol, const std::string& filter, std::size_t maxHits, McpPageArgs page = {}, bool* seedFault = nullptr, - crossref::WhereisListing listing = crossref::WhereisListing::ShorterOfDefsAll ) + crossref::WhereisListing listing = crossref::WhereisListing::ShorterOfDefsAll, + bool* blobFault = nullptr ) { std::string sel = symbol; std::string seedSpec; @@ -581,6 +582,11 @@ inline std::string whereisText( const std::string& root, const std::string& symb { return {}; } + if( res.blobsUnread ) // the CLI twin's refusal: a hit count over blobs never read is not an answer + { + if( blobFault != nullptr ) { *blobFault = true; } + return {}; + } res.seedSpec = std::move( seedSpec ); // The tree zero stays an answer; the near-miss only says WHICH zero it is. Computed only on the zero, // so a real hit list costs nothing and is byte-identical to before. @@ -602,11 +608,12 @@ inline std::string whereisText( const std::string& root, const std::string& symb // M13: --stray-content is in cli.h's honorsPaging set — writeStrayContentPage is the entry point the // CLI already calls with cfg.pageLimit/cfg.pageOffset, so the twin passes the same pair rather than 0,0. inline std::string strayContentText( const std::string& root, const std::string& filter, std::size_t maxFiles, - McpPageArgs page = {} ) + McpPageArgs page = {}, bool* blobFault = nullptr ) { const crossref::StrayResult res = crossref::computeStrayContent( root, filter ); if( !res.ok ) { + if( res.blobsUnread && blobFault != nullptr ) { *blobFault = true; } // K75: the host answers an error naming the cause, not "not a git repository" return {}; } return captureXml( [ & ]( std::FILE* f ) { crossref::writeStrayContentPage( f, res, maxFiles, page.limit, page.offset ); } ); @@ -2068,41 +2075,38 @@ inline McpForAssembly assembleMcpFor( const McpForAssemblyInputs& in, bool payFr // late-insertion technique for kOverCeilingLegend) knows whether to add it. bool mcpSectionsWillStub = false; { - const bool mcpWantLego = sectionsWant( sections, "lego" ); - const bool mcpWantCompose = sectionsWant( sections, "compose" ); - const bool mcpLegoCandidate = !legoStr.empty() && !mcpWantLego; - const bool mcpComposeCandidate = !composeStr.empty() && !mcpWantCompose; - if( mcpLegoCandidate || mcpComposeCandidate ) - { - std::string sectionsNextMcp = nextFlag( "--for=", task ); - sectionsNextMcp += ' '; - sectionsNextMcp += nextFlag( "--sections=", "lego,compose" ); - const SectionStubPricing legoPricing = mcpLegoCandidate - ? priceSectionStub( "lego", legoPreCapCount, sectionsNextMcp, /*hasRenderedBytes=*/true, legoStr.size() ) - : SectionStubPricing{}; - const SectionStubPricing composePricing = mcpComposeCandidate - ? priceSectionStub( "compose", composePreCapCount, sectionsNextMcp, /*hasRenderedBytes=*/true, composeStr.size() ) - : SectionStubPricing{}; - const bool mcpLegoWillStub = mcpLegoCandidate && legoPricing.collapse; - const bool mcpComposeWillStub = mcpComposeCandidate && composePricing.collapse; - // postcondition of the rule itself, checked BEFORE the swap below discards the original size: - // whichever section actually collapses is, by construction, smaller than what it replaced. - ENSURES( !mcpLegoWillStub || legoPricing.stubXml.size() < legoStr.size(), - "R2-L2' MCP twin: a lego stub collapsed without being smaller than the section it replaced" ); - ENSURES( !mcpComposeWillStub || composePricing.stubXml.size() < composeStr.size(), - "R2-L2' MCP twin: a compose stub collapsed without being smaller than the section it replaced" ); - if( mcpLegoWillStub ) - { - legoStr = legoPricing.stubXml; - } - if( mcpComposeWillStub ) - { - composeStr = composePricing.stubXml; - } - // R2-L2p: same truth table as the CLI's `if( legoWillStub || composeWillStub ) sectionsStubNote = …` - // (verbs_for.h ~2858): the legend rides iff at least one section actually collapsed. - mcpSectionsWillStub = mcpLegoWillStub || mcpComposeWillStub; - } + // the CLI twin's one plan (rw::planForSectionStubs): this surface always renders whole, so rendered=true + const ForStubbableSection mcpLegoServedIn{ .hasContent = !legoStr.empty(), .preCapTotal = legoPreCapCount, .rendered = true, + .renderedBytes = legoStr.size() }; + const ForStubbableSection mcpComposeServedIn{ .hasContent = !composeStr.empty(), .preCapTotal = composePreCapCount, .rendered = true, + .renderedBytes = composeStr.size() }; + const ForSectionStubPlan plan = planForSectionStubs( sections, task, mcpLegoServedIn, mcpComposeServedIn ); + // lego-charge (train 26c seam): the charge plan the budget follows (forTaskText's mcpSigChargePlan, the same + // rule: rw::forSectionsSigChargePlan), recomputed here from the sections AS SERVED — the count-floor lego re-render + // above may have changed the section's size since forTaskText charged the budget, and the invariant below is the + // two planners' agreement on one input: the served stub never exceeds the stub the charge plan collapses it to. + const ForSectionStubPlan mcpSigChargePlan = forSectionsSigChargePlan( sections, budgetTokens > 0, task, mcpLegoServedIn, mcpComposeServedIn ); + const bool mcpLegoWillStub = plan.legoWillStub; + const bool mcpComposeWillStub = plan.composeWillStub; + // postcondition of the rule itself, checked BEFORE the swap below discards the original size: + // whichever section actually collapses is, by construction, smaller than what it replaced. + ENSURES( !mcpLegoWillStub || plan.lego.stubXml.size() < legoStr.size(), + "R2-L2' MCP twin: a lego stub collapsed without being smaller than the section it replaced" ); + ENSURES( !mcpComposeWillStub || plan.compose.stubXml.size() < composeStr.size(), + "R2-L2' MCP twin: a compose stub collapsed without being smaller than the section it replaced" ); + ENSURES( !( mcpLegoWillStub && mcpSigChargePlan.legoWillStub ) || plan.lego.stubXml.size() <= mcpSigChargePlan.lego.stubXml.size(), + "lego-charge MCP twin: the served lego stub is longer than the stub the budget was charged for" ); + if( mcpLegoWillStub ) + { + legoStr = plan.lego.stubXml; + } + if( mcpComposeWillStub ) + { + composeStr = plan.compose.stubXml; + } + // R2-L2p: same truth table as the CLI's `if( legoWillStub || composeWillStub ) sectionsStubNote = …` + // (verbs_for.h ~2858): the legend rides iff at least one section actually collapsed. + mcpSectionsWillStub = mcpLegoWillStub || mcpComposeWillStub; } const bool mcpLegoCountAttrs = legoStr.find( rw::kLegoCountAttrPrefix ) != std::string::npos; // false on a stub (its own attributes only) std::fwrite( sigsStr.data(), 1, sigsStr.size(), mem ); @@ -2145,6 +2149,14 @@ inline McpForAssembly assembleMcpFor( const McpForAssemblyInputs& in, bool payFr return McpForAssembly{ std::move( out ), mcpSigsCut }; } +// #362 review round 4, follow-up (quality bar): the twins' B2 rank-flags fragment, named once - +// the inline conditional appeared in both twins' page calls and the second read pushed +// packTaskText past its complexity bar. +inline std::string_view mcpPageRankFlags( bool noRoute ) +{ + return noRoute ? std::string_view( " --no-route" ) : std::string_view(); +} + inline std::optional forTaskText( const std::string& root, const std::string& task, RedactCounts* redact = nullptr, std::size_t budgetTokens = 0, bool noRoute = false, McpPageArgs page = {}, // L-W: limit/offset select the FILE PAGE (forpage.h), the CLI --for --limit twin @@ -2299,6 +2311,25 @@ inline std::optional forTaskText( const std::string& root, const st const std::string_view mcpRootArg = ing.realPaths.empty() ? std::string_view( root ) : std::string_view(); if( page.limit > 0 || page.offset > 0 ) { + // PAGING-POC follow-up (issue #294): budget + window is NOT this file page — it is the + // BUDGETED-BUNDLE CANDIDATE PAGE, the same document the CLI --for serves (forpage.h's + // forCandidatePageDoc, one implementation for every dialect). Serving the budgetless file + // page here would ignore budget_tokens — the accept-and-ignore class (mcpverbscheck 6e). + if( budgetTokens > 0 ) + { + const AdaptiveCut cut = adaptiveCut( lensRank, 5, std::size_t( forTopN ), /*scanFullDistribution=*/true ); + // #362 review round 4 (B1): the twins' inline pipeline always scored the full distribution + // (pruneTopK=0) and the CLI pages now force fullDistribution under a window, so the five page + // paths rank ONE list — the page carries the same pasteable CLI next= as the CLI pages, with + // no_route echoed (B2) so the pasted continuation re-ranks identically. The compact dialect + // emitted in-doc (schema= present) so the ref posture reduces it like any other spec-keyed + // answer — the outer layer sees AlreadyCompact and passes it through. + return forCandidatePageDoc( ing, lensRank, cut, ForCandidatePageReq{ task, "--for=", + mcpPageRankFlags( noRoute ), + /*pasteHandle=*/true, /*compactLegend=*/true, + routeNoteOf( rc, shape, noRoute ), mcpRootArg, + redact, gitstamp::stampAt( root ), budgetTokens, page.limit, page.offset } ); + } const ForFilePage filePage = computeForFilePage( ing, lensRank, mcpEvidence ); // PR #215 review item 4: this page composed "routed: " + rc.reason by hand and so answered in a spelling // row 6 retired everywhere else — a parity break with the CLI page AND with this server's own bundle two @@ -2528,10 +2559,17 @@ inline std::optional forTaskText( const std::string& root, const st const std::size_t mcpAtLegendExemptBytes = !flRootArg.empty() && !mcpForAtAttrStr.empty() ? rw::kForAtStampProse.size() : 0; // r2-LO, above // e=: a disclosure on the same contract (the CLI twin exempts the same clause) const std::size_t mcpEndLineExemptBytes = mcpEndLineLegend.size(); + // lego-charge: / cost what they are SERVED as (the CLI twin's rule, rw::forSectionsSigChargePlan): + // a collapsing section its stub; under an explicit budget_tokens, a section `sections` opts into its full render. + const ForStubbableSection mcpLegoChargeIn{ .hasContent = !legoStr.empty(), .preCapTotal = legoPreCapCount, .rendered = true, + .renderedBytes = legoStr.size() }; + const ForStubbableSection mcpComposeChargeIn{ .hasContent = !composeStr.empty(), .preCapTotal = composePreCapCount, .rendered = true, + .renderedBytes = composeStr.size() }; + const ForSectionStubPlan mcpSigChargePlan = forSectionsSigChargePlan( sections, budgetTokens > 0, task, mcpLegoChargeIn, mcpComposeChargeIn ); const std::size_t fixedBytes = headerStr.size() - rw::kForFileTailLegend.size() - mcpConfidenceExemptBytes - mcpIdRouteExemptBytes - mcpAtLegendExemptBytes - mcpEndLineExemptBytes - rw::forZeroNoteBytes( headerStr ) // lean-answers: the zero reading never costs a row - + legoStr.size() + composeStr.size() + routeStr.size() + 6; // + "" + + mcpSigChargePlan.servedBytes( mcpLegoChargeIn, mcpComposeChargeIn ) + routeStr.size() + 6; // + "" const std::size_t sigsBudget = forBudgetBytes > fixedBytes ? forBudgetBytes - fixedBytes : 1; // ≥1: 0 = "no budget" // knob-honesty-068: the CLI twin's continuation (serialize.h SigsCutContinuation). This surface is signatures-only, // so its sig side IS the ceiling: charged under a caller's budget_tokens, exempt at the default. NO CLI argv here — an @@ -4345,9 +4383,24 @@ inline std::pair qualityBaselineJson( const std::strin // what it is FOR; a separate verb would duplicate the whole task/budget contract for one integer. 0 (or any // value outside 2..16, which is silently clamped OFF rather than erroring an otherwise valid explore call) // ⇒ the plain single-bundle form, byte-identical to before. +// (#362 review, item 2): the form options in ONE struct — packTaskText's parameter list was the +// quality gate's finding (6 -> 8 as the paging round grew it; the form bundles the family: the +// partition fan-out, the route posture, and the page window, all of them optional knobs of the +// same bundle). Aggregate-init order is the declaration order. +struct PackTaskForm +{ + std::uint32_t partitionCount = 0; // --partition=N over MCP (0 = one un-split bundle) + bool noRoute = false; // the CLI --no-route twin + McpPageArgs page; // limit/offset: the candidate page under a budget, the file page without +}; + inline std::string packTaskText( const std::string& root, const std::string& task, std::size_t budgetTokens, - RedactCounts* redact = nullptr, std::uint32_t partitionCount = 0, bool noRoute = false ) + RedactCounts* redact = nullptr, const PackTaskForm& form = {} ) { + const std::uint32_t partitionCount = form.partitionCount; + const bool noRoute = form.noRoute; + const int pageLimit = form.page.limit; + const int pageOffset = form.page.offset; const McpIndex& ix = getIndex( root ); const IngestResult& ing = ix.ing; const Graph& g = ix.g; @@ -4441,6 +4494,32 @@ inline std::string packTaskText( const std::string& root, const std::string& tas const notes::NoteIndex noteIndex = notes::loadNoteIndex( root ); const notes::NoteIndex* const notesPtr = noteIndex.empty() ? nullptr : ¬eIndex; + // PAGING-POC (#294 + the #362 review round, item 6): the page branch sits AFTER every ranking + // boost (the early position returned before applyDocMentionBoost above, and the page ranked a + // different candidate set than the un-paged answer this same function serves — total= 1155 vs + // 901 on the same task; the parity arm in test/mcpverbscheck.sh (6e) pins the two together). + // budget + window is the CANDIDATE PAGE; a window with no budget is the FILE-GRAIN WIDENING + // PAGE (computeForFilePage/renderForFilePageXml, forpage.h) — the for twin's own page, byte-identical. + if( pageLimit > 0 || pageOffset > 0 ) + { + const std::string_view mcpRootArg = ing.realPaths.empty() ? std::string_view( root ) : std::string_view(); + if( budgetTokens > 0 ) + { + const AdaptiveCut cut = adaptiveCut( lr.rank, 5, std::size_t( kForLensDefaultTopN ), /*scanFullDistribution=*/true ); + // #362 review round 4 (B1/B2): one list across all five page paths — the same pasteable + // next= as the CLI pages, no_route echoed so the pasted continuation re-ranks identically. + return forCandidatePageDoc( ing, lr.rank, cut, ForCandidatePageReq{ task, "--pack-task=", + mcpPageRankFlags( noRoute ), + /*pasteHandle=*/true, /*compactLegend=*/true, + lr.routeNote, mcpRootArg, + redact, gitstamp::stampAt( root ), budgetTokens, pageLimit, pageOffset } ); + } + const ForFilePage filePage = computeForFilePage( ing, lr.rank, lr.evidence ); + const std::string pageRootOpen = ctxRootOpen( task, lr.routeNote, mcpRootArg ); + return renderForFilePageXml( ing, filePage, ForPageRenderParts{ task, pageRootOpen, forCoveragePct( lr.evidence, topLensId( lr.rank ) ), + pageLimit, pageOffset, mcpRootArg, /*compactLegend=*/false } ); + } + PackTaskInputs in; in.budgetTokens = budgetTokens; in.fanIn = &fanIn; @@ -5863,7 +5942,8 @@ inline bool mcpVerbDeclaresLegend( std::string_view verb ) noexcept inline std::string_view mcpCompactLegendHint( std::string_view verb ) noexcept { if( verb == "analyze" || verb == "rank_by" ) { return "map"; } // lane/t10-mcp-coverage: rank_by's XML root is the SAME shape analyze serves - if( verb == "explore" || verb == "pack_task" ) { return "pack-task"; } + if( verb == "explore" || verb == "pack_task" ) { return "pack-task"; } // their CANDIDATE PAGE (root ) adjusts to candidate-page inside the layer + if( verb == "for" ) { return "candidate-page"; } // the for twin's candidate page (root ); the un-paged answer has no spec row here and passes through, as it did under the empty hint if( verb == "from_trace" ) { return "from-trace"; } if( verb == "lego" ) { return "lego"; } if( verb == "exemplar" ) { return "exemplar"; } diff --git a/src/model.h b/src/model.h index 8a94f8948..cd14d4626 100644 --- a/src/model.h +++ b/src/model.h @@ -94,7 +94,9 @@ inline const char* symTag( SymKind k ) noexcept case SymKind::Struct: return "struct"; case SymKind::Interface: return "iface"; case SymKind::Var: return "var"; - case SymKind::Section: return "sec"; // markdown heading (doc structure; isolated in the graph) + case SymKind::Section: return "sec"; // data-kind defs: doc headings, JSON/YAML keys, and Ruby db/schema.rb + // columns — the Ruby ones are definitions only, NO call edges + // (graph.h buildGraph's byName skips them; see isMeasurableKind) case SymKind::Macro: return "macro"; // #define (disclosed-degraded: replacement text, not a parsed body) case SymKind::Field: return "field"; // member variable (id=path::Owner::field; use-sites via --uses=Owner.field) case SymKind::Other: return "other"; diff --git a/src/pathgaps.h b/src/pathgaps.h index 9b66f209c..5c682abb9 100644 --- a/src/pathgaps.h +++ b/src/pathgaps.h @@ -326,7 +326,7 @@ inline PathGapsXml pathUnreachedXml( const IngestResult& ing, const PathSearchGa return out; } const std::size_t shown = std::min( gaps.syms.size(), kPathGapRows ); - std::string next = "--expand="; + std::string nextSelectors; // the comma-joined file:name list; nextFlag quotes it when a shell would split or expand it for( std::size_t i = 0; i < shown; ++i ) { const Symbol& s = ing.symbols[ gaps.syms[i].id ]; @@ -334,12 +334,12 @@ inline PathGapsXml pathUnreachedXml( const IngestResult& ing, const PathSearchGa out.rows += ""; // file:name, the selector spelling --expand resolves to this one definition (the same p= the row prints) - next += ( i == 0 ? "" : "," ) + std::string( rp ) + ":" + s.name; + nextSelectors += ( i == 0 ? "" : "," ) + std::string( rp ) + ":" + s.name; } out.rootAttrs = " searched=\"" + std::to_string( gaps.searched ) + "\" gaps=\"" + pathGapCountsValue( gaps.totals ) + "\" gap_syms=\"" + std::to_string( gaps.syms.size() ) + "\"" + ( gaps.syms.size() > shown ? " gap_syms_capped=\"1\"" : "" ) + " hint=\"no path through resolved call edges, but the search is incomplete: it met calls or hand-offs it could not follow in full (gaps=) — read the gap rows, or try " - + ex( say.connect ) + " (undirected: finds a shared caller)" + std::string( say.tail ) + "\"" + nextAttrXml( next ); + + ex( say.connect ) + " (undirected: finds a shared caller)" + std::string( say.tail ) + "\"" + nextAttrXml( nextFlag( "--expand=", nextSelectors ) ); return out; } diff --git a/src/quality.h b/src/quality.h index 3b8b98450..b6842aafb 100644 --- a/src/quality.h +++ b/src/quality.h @@ -2281,7 +2281,9 @@ inline std::string cacheRootKeyHex( const std::string& root ) // not include this header; it relies on ingest.cpp including quality.h (line 13) before ingest_cache.h, and a reorder // that broke that fails the build on the undeclared name rather than passing. constexpr std::uint32_t kIngestCacheVersionMirror = 29; // MUST equal ingest.cpp's kCacheVersion (gated); 29 = FE-B ref memberPath/memberCtor + 4 bind kinds, 28 = FE-A ref memberCall/memberRoot, 27 = corrected fnScope values, 26 = function-local def scope span (25 = #157 + #150) -constexpr std::uint32_t kIngestParserVerMirror = 156; // MUST equal ingest.cpp's kParserVer (gated) +constexpr std::uint32_t kIngestParserVerMirror = 158; // MUST equal ingest.cpp's kParserVer (gated) + // 158 = 2026-10-09 (train 26c: PR #339 Rails schema columns; above 156 / rich tag 157; see the + // kParserVer note) // 156 = 2026-10-08 (train 26b: above every lane build, up to 154 / rich tag 155; see the // kParserVer note) // 148 = 2026-10-04 (train 26a: PR #373 merged; above the branch's 145 and the 145–147 @@ -2659,11 +2661,18 @@ inline std::string headSnapExclHex( const std::vector& excludes, st // only WRITTEN into its shard the next time it is (re)computed. A blob absent from both locations is a clean // miss, created fresh, in its shard. `evictOldCacheFamily` below sweeps both layouts, so an old flat blob // still ages out on schedule even if its key is never rewritten. +// A 64-bit key spelled as its 16 lowercase hex digits — the ack ledger's key (ackMapKey), the shard below, and the +// ingest's per-root sidecar names. +inline std::string hex16( std::uint64_t key ) +{ + char hex[ 20 ]; + rw::formatTo( hex, sizeof( hex ), "{:016x}", static_cast( key ) ); + return hex; +} + inline std::string blobShardHex( std::string_view filename ) { - char hex[ 3 ]; - rw::formatTo( hex, sizeof( hex ), "{:02x}", static_cast( fnv1a64( filename ) & 0xff ) ); - return std::string( hex ); + return hex16( fnv1a64( filename ) ).substr( 14 ); // the low byte, two hex digits: "{:02x}" of ( hash & 0xff ) } inline std::string resolveCacheBlobPath( const std::string& dir, const std::string& filename ) @@ -4470,7 +4479,11 @@ computeWindowRefBodyHashes( const std::string& root, std::uint32_t days, // would keep serving the old stream to a fixed binary: the exact silent-zero the fix removes, restored by // cache. Bumping the scheme makes every pre-existing blob a clean miss (deserialize checks it), not a wrong // answer. This is the qchurn cache's OWN version — not kParserVer, which keys extraction and is untouched. -constexpr std::uint32_t kQChurnCacheScheme = 2; +// SCHEME 3 (2026-10-10, train 26c, K51): a binary before K51 stored a walk git could not finish (an empty stream, or the +// newest prefix of the window) as if it were the window, and the key cannot tell that blob from a real read. K51 never +// stores one, but a blob already on disk would keep serving its silent zero or prefix, with no history_unread=, until HEAD +// moved. The bump makes every pre-K51 blob a clean miss; the cost is one cold walk per repository. +constexpr std::uint32_t kQChurnCacheScheme = 3; constexpr char kQChurnMagic[4] = { 'Q', 'C', 'H', 'N' }; inline std::string serializeRawCommitStream( const RawCommitStream& raw, const std::string& keyMat ) @@ -4580,7 +4593,7 @@ inline bool deserializeRawCommitStream( const std::string& blob, const std::stri // corrupt/foreign blob — is a clean miss that falls through to a full recompute, never a wrong answer). No git repo // degrades to an empty stream; no resolvable HEAD to the uncached walk (which itself degrades to an empty stream — // gitLogNameOnlyRaw's contract). Thread-safe: every memo it reaches (the HEAD anchor, the toplevel) takes its own -// mutex, and the blob write is an atomic rename. +// mutex, and the blob write is an atomic rename. K51: a walk git could not complete comes back with `unread` set and is never stored. inline RawCommitStream gitRawCommitStreamCached( const std::string& root, const char* coSince ) { if( !hasEnclosingGitRepo( root ) ) @@ -4590,7 +4603,20 @@ inline RawCommitStream gitRawCommitStreamCached( const std::string& root, const const std::string headSha = gitHeadSha( root ); if( headSha.empty() ) { - return gitLogNameOnlyRaw( root, coSince ); + // No resolvable HEAD: an unborn branch has no history to read, and git log's failure there (git's own fatal exit, + // 128) is that fact, not an I/O error — so that walk is "read, empty" (K51 floor: a HEAD that is corrupt rather than + // unborn exits 128 too and reads the same here). Any OTHER failure means git did not answer at all — not on PATH + // (the shell's 127), not executable, no popen — and gitHeadSha's "" was that, not an unborn branch: it stays unread. + RawCommitStream noHead = gitLogNameOnlyRaw( root, coSince ); + if( noHead.unread && noHead.exitCode == 128 ) + { + noHead.unread = false; + } + else if( noHead.unread ) + { + noHead.notRun = true; + } + return noHead; } const std::string repoHex = cacheRootKeyHex( root ); @@ -4609,8 +4635,16 @@ inline RawCommitStream gitRawCommitStreamCached( const std::string& root, const } raw = gitLogNameOnlyRaw( root, coSince ); // cold — the 431 ms walk + if( raw.unread || gitWalksAbandoned() ) + { + // K51: a walk that could not be read is NOT stored. Stored, its empty stream would answer "read, no commits" under this + // (repo, HEAD, window) key for as long as HEAD stayed put; unstored, the next call walks again and the caller discloses. + // The flag rides the returned stream to main.cpp's amp=/churn= block, which DISCLOSEs it into the document (historyread.h). + // An abandoned read (a memory-guard refusal stopped the walk) is a prefix too, even when git exited 0: never cached. + return raw; + } atomicWriteFile( cachePath, serializeRawCommitStream( raw, keyMat ) ); // best-effort; a failed - // write just recomputes next time + // write just recomputes next time return raw; } @@ -6470,9 +6504,7 @@ inline std::optional rescoreAckRecord( const AckRecord& r, std::optional( key ) ); - return kind + " " + hex; + return kind + " " + hex16( key ); } // ─── THE LEGACY-ACK BACKFILL — what a row written before provenance existed CAN still get back ───────── @@ -7474,6 +7506,10 @@ inline std::size_t healDefectSites( Snapshot& base, const IdentityAliases& al ) { std::size_t healed = 0; std::vector addSites; + // ADD, NEVER OVERWRITE, per `to` anchor: two alias sources that converge on one current anchor (a scheme alias and a + // rename alias, CodeRabbit on #383) must not both copy their sites under it — the baseline check above only sees + // anchors the baseline already held, not ones this loop filled. The first alias in `from` order fills; the rest skip. + std::vector filledTo; for( const auto& [ from, to ] : al.toCurrent ) { const auto lo = std::lower_bound( base.defectSites.begin(), base.defectSites.end(), DefectSiteKey{ from, 0 } ); @@ -7482,6 +7518,15 @@ inline std::size_t healDefectSites( Snapshot& base, const IdentityAliases& al ) { continue; } + if( lo == base.defectSites.end() || lo->anchor != from ) + { + continue; // nothing to copy: an alias with no recorded sites must not claim the anchor + } + if( std::find( filledTo.begin(), filledTo.end(), to ) != filledTo.end() ) + { + continue; + } + filledTo.push_back( to ); for( auto it = lo; it != base.defectSites.end() && it->anchor == from; ++it ) { addSites.push_back( { to, it->site } ); diff --git a/src/serialize.h b/src/serialize.h index d1a650f1a..49feeefe9 100644 --- a/src/serialize.h +++ b/src/serialize.h @@ -26,7 +26,9 @@ #include "sarif.h" // R-E (2026-08-17): rootRelativeUri/rootPrefixOf — the same root= single-root-only // strip --grep's emitGrepReport uses, reused here so the two verbs cannot diverge #include "prconverge.h" // W2-F: RankDisclosure + the pr_iters= / pr_converged= spellings (CLI and MCP share them) +#include "historyread.h" // K51: history_unread= on every root that prints churn=/amp= #include "gitmine.h" // F3 (H2H-Graft): RecentFile — the map's rows are the churn-decay miner's own product +#include "mapinventory.h" // D7: the map scope's tier — computeMapInventory, kMapInventoryLegend #include #include // std::iota — codeFirstKeep @@ -349,7 +351,9 @@ inline std::string ctxRootOpen( std::string_view task, std::string_view routeNot { out += " route_scrubbed=\"1\""; } + out += renderHistoryUnread( DiscloseAs::XmlAttrs ); // K51: absent unless the amp=/churn= walk could not be read; shared by --for/--pack-task/--from-trace out += ">"; + out += renderHistoryUnread( DiscloseAs::LegendComment ); // its own definition (historyread.h) return out; } @@ -377,6 +381,7 @@ inline std::string ctxRootJsonScrubKeys( std::string_view task, std::string_view { keys += ",\"route_xml_scrubbed\":true"; } + keys += renderHistoryUnread( DiscloseAs::JsonKeys ); // K51: this is the one JSON root writer every ctx-family document shares (--for, --pack-task, --from-trace) return keys; } @@ -1074,12 +1079,20 @@ inline FileTail computeFileTail( const IngestResult& ing, const std::vector& esc ) +// `cutNextAttr` (a ready ` next="…"` from nextAttrXml, or empty) rides the open tag ONLY when the tail is cut +// (shown < total): the call that serves the files left out. Empty (every caller but the --for explicit-budget +// regime) ⇒ byte-identical to before. +inline std::string renderFileTailXml( const FileTail& t, std::size_t shownCap, std::vector& esc, std::string_view cutNextAttr = {} ) { const std::size_t shown = std::min( shownCap, t.paths.size() ); std::string x = ""; + + "\" capped=\"" + ( shown < t.total ? "1" : "0" ) + "\""; + if( shown < t.total ) + { + x += cutNextAttr; + } + x += ">"; for( std::size_t i = 0; i < shown; ++i ) { x += ""; @@ -1128,11 +1141,11 @@ inline constexpr std::string_view kForHdrLegend = // Explicit-budget row fit: the largest shown count whose rendered XML fits `budgetBytes` (0 rows always // "fits" — the shell is reserved by the caller). Walks down from the collected count; deterministic. -inline std::size_t fileTailShownForBudget( const FileTail& t, std::size_t budgetBytes, std::vector& esc ) +inline std::size_t fileTailShownForBudget( const FileTail& t, std::size_t budgetBytes, std::vector& esc, std::string_view cutNextAttr = {} ) { for( std::size_t shown = t.paths.size(); shown > 0; --shown ) { - if( renderFileTailXml( t, shown, esc ).size() <= budgetBytes ) + if( renderFileTailXml( t, shown, esc, cutNextAttr ).size() <= budgetBytes ) { return shown; } @@ -1953,6 +1966,11 @@ struct MapAnnotations // FE-B: the compact posture strips the map's via="name" comment (compactlegend.h, the via/x rows restate it), so the // map does not write it at all — and the --max-tokens fit, which measures this render, prices the delivered bytes. bool viaLegendStripped = false; + // D7 MAP-INVENTORY (mapinventory.h): the map scope's inventory tier — after the ranked rows, naming or counting + // every indexed file they did not show, plus the program entries. Set where codeFirstRows is (the plain CLI map, MCP + // analyze on a clean working set, MCP rank_by=pagerank) except under --max-tokens (mapInventoryWanted). Filled by + // assignment, like the trailing fields above. + bool inventoryTier = false; }; // ── the code-first row pick: data_sections_cut= / next= (docs/EVALS.md "Map data Sections never crowd code out of the @@ -2743,6 +2761,65 @@ inline bool hopSlotHasProvenEdge( const std::vector& outOff, cons []( std::uint8_t bit ) { return bit != 0; } ); } +// D7 MAP-INVENTORY: whether THIS map carries the inventory tier (mapinventory.h), decided once for both dialects. The +// default map scope only (MapAnnotations::inventoryTier); never under in=DIR's stub (no ranked rows to complement), +// multi-root (the roots table already partitions the paths, and a per-root inventory is a different answer), or +// --max-tokens (its fit binary-searches top-k over the rendered bytes; an inventory that GROWS as top-k shrinks would +// make that search non-monotone). +inline bool mapInventoryWanted( const IngestResult& ing, const MapAnnotations& ann ) noexcept +{ + return ann.inventoryTier && !ann.stubSymbols && ing.rootLabels.size() < 2 && ann.maxTokensFit == nullptr; +} + +// The XML element: `……`. Entry callees go +// through writeMapCalleeRows, the ranked rows' own writer, so prov=/via="name"/x= read exactly as they do above. +template +inline void writeMapInventoryXml( XmlWriter& w, const IngestResult& ing, const MapInventory& inv, const PathRel& pathRel, + const std::vector& outOff, const std::vector& outTargets, + const std::vector* outProv, const std::vector* outNameOnly, + std::vector& esc, std::vector& scratch ) +{ + EXPECTS( inv.active, "the caller emits the tier only when it has a cut to inventory" ); + w.write( " inv.entries.size() ) + { + w.write( " entries_capped=\"1\" entries_total=\"" ); w.write( std::to_string( inv.entryTotal ) ); w.write( "\"" ); + w.write( nextAttrXml( inv.entriesNext, "entries_next" ) ); // rule 5: the cut is recoverable + } + if( inv.namesCapped ) + { + w.write( " names_capped=\"1\" names_total=\"" ); w.write( std::to_string( inv.namesTotal ) ); w.write( "\"" ); + w.write( nextAttrXml( inv.namesNext, "names_next" ) ); + } + w.write( ">" ); + for( NodeId id : inv.entries ) + { + const Symbol& s = ing.symbols[ id ]; + w.write( "" ); + writeMapCalleeRows( w, ing, outOff[ id ], outOff[ id + 1 ], outTargets, outProv, outNameOnly, esc, scratch ); + w.write( "" ); + } + for( const MapInventoryDir& d : inv.dirs ) + { + w.write( "" ); + } + w.write( "" ); +} + inline void serialize( std::FILE* out, const IngestResult& ing, const std::vector& rank, const std::vector& outOff, const std::vector& outTargets, int topK, bool mostImportantLast = false, @@ -2854,6 +2931,9 @@ inline void serialize( std::FILE* out, const IngestResult& ing, const std::vecto if( !seen[f] ) { seen[f] = 1; fileOrder.push_back( f ); } buckets[f].push_back( id ); } + // D7: the inventory over the files the kept rows did NOT reach (`seen` is exactly the printed groups) + const MapInventory inventory = ( mapInventoryWanted( ing, ann ) && !stubbed ) + ? computeMapInventory( ing, outOff, outTargets, seen, pathRel ) : MapInventory{}; // T1/§H7: the byte MODEL. It keeps two jobs and loses one. It still decides the T3 emit order below // (which has to be decided before any byte exists — see §H7 at estimateTokens) and it still supplies @@ -3029,6 +3109,10 @@ inline void serialize( std::FILE* out, const IngestResult& ing, const std::vecto { legend += kDataSectionsCutLegend; // charged to the map that carries data_sections_cut= } + if( inventory.active ) + { + legend += kMapInventoryLegend; // D7: charged to the map that carries + } // W2-F: the pr_iters= / pr_converged= definition, charged to the maps that carry the attributes — empty // for a lexical or HITS ordering, and the prose half only on the map whose iteration stopped short. legend += renderDisclosure( ann.prDisclosure, DiscloseAs::LegendComment ); @@ -3247,6 +3331,7 @@ inline void serialize( std::FILE* out, const IngestResult& ing, const std::vecto // L3 follow-up (CodeRabbit 4053600616): TRULY last, past every pre-existing attribute — same placement // rule as lens= just above, so no attribute-adjacency assertion in test/ can break on it. if( ann.notesDegraded ) { h += notes::kNotesDegradedAttr; } + h += renderHistoryUnread( DiscloseAs::XmlAttrs ); // K51: absent unless the amp=/churn= walk could not be read // The code-first pick's swaps, last of all (the same placement rule as the two above): absent when nothing was swapped. if( dataSecCut.cut > 0 ) { @@ -3254,6 +3339,7 @@ inline void serialize( std::FILE* out, const IngestResult& ing, const std::vecto h += nextAttrXml( dataSectionsNext( dataSecCut ) ); } h += ">"; + h += renderHistoryUnread( DiscloseAs::LegendComment ); // K51: its own definition, right behind the root it marks return h; }; const auto buildTail = [ & ]( std::size_t estTokens ) -> std::string @@ -3335,6 +3421,21 @@ inline void serialize( std::FILE* out, const IngestResult& ing, const std::vecto stub += "/>"; w.write( stub ); } + // D7: the inventory tier rides the LOW-importance end of the document — after the rows, or before them when the + // map is emitted important-last (an explicit flag or the fill-aware auto flip), so the top-ranked rows keep the + // end position that order promises. + const auto writeInventory = [ & ]() + { + if( inventory.active ) + { + writeMapInventoryXml( w, ing, inventory, pathRel, outOff, outTargets, outProv, outNameOnly, esc, hedgedScratch ); + } + }; + const bool inventoryFirst = !stable && effImportantLast; // --stable's path order wins, as it does for the rows + if( inventoryFirst ) + { + writeInventory(); + } static const std::vector kNoFiles; for( std::uint32_t f : ann.stubSymbols ? kNoFiles : fileOrder ) { @@ -3553,6 +3654,10 @@ inline void serialize( std::FILE* out, const IngestResult& ing, const std::vecto } w.write( "" ); } + if( !inventoryFirst ) + { + writeInventory(); + } w.write( "" ); }; // DEGRADE, one path for both failures: the head goes out FIRST carrying the MODELLED estimate (the pre-§H7 number, so @@ -5343,9 +5448,14 @@ inline void packSignatures( std::FILE* out, const IngestResult& ing, const std:: // for the caller's legend splices. Lens path only; zeroed // (nothing cut) on every other path. knob-honesty-068: its // continuationRequest is an INPUT, read before the reset. - SigRowSpelling spelling = {} ) // lean-answers lane: the --for lens's zero cx=/ccx=/in= omitted + SigRowSpelling spelling = {}, // lean-answers lane: the --for lens's zero cx=/ccx=/in= omitted // (SigRowFacts::elideZero); its legend says absent = 0. Default ⇒ // byte-identical (pack-task, from-trace and the map stay as they were). + std::uint32_t rankBase = 0 ) // d1/paging (#294): 1-based GLOBAL candidate-rank base for the windowed + // serving (0 = the un-windowed default, byte-identical) — a continuation + // page's rows carry their GLOBAL r= (page 2's first row is r=201, not + // r=1: the rank-adaptive doc/sig tiers and the r=1 row's next= hint key + // on GLOBAL rank), so pages merge without duplicate r= { const SigsCutContinuation* const cutNext = cutOut != nullptr ? cutOut->continuationRequest : nullptr; if( cutOut ) @@ -5413,7 +5523,7 @@ inline void packSignatures( std::FILE* out, const IngestResult& ing, const std:: globalRankOf.assign( S, 0 ); for( std::size_t k = 0; k < keep; ++k ) { - globalRankOf[order[k]] = std::uint32_t( k + 1 ); + globalRankOf[order[k]] = rankBase + std::uint32_t( k + 1 ); // + rankBase: the WINDOWED serving's global base } } @@ -5754,7 +5864,11 @@ inline void packSignatures( std::FILE* out, const IngestResult& ing, const std:: { continue; } - pushShownSigId( shownIdsOut, order, e.globalRank ); // lane 2 + // lane 2 — the LOCAL rank under rankBase: globalRank carries the WINDOWED serving's + // global offset (#294 review), and order[] holds the WINDOW's rows in its head, so the + // id mapping must subtract rankBase or it reads past the window into the masked tail + // (the #362 review round's next_offset=2^32 seam). + pushShownSigId( shownIdsOut, order, std::uint32_t( e.globalRank - rankBase ) ); writeSigHeadWithEnd( w, e.head, e.endLine ); // e= after l=, when the lens asked and the extent is known if( !e.doc.empty() ) { w.write( "" ); w.write( escapeXml( e.doc, esc ) ); w.write( "" ); } w.write( escapeXml( e.sig, esc ) ); @@ -6327,24 +6441,38 @@ struct CalleeCallsSink // FE-B: set once the block's caller has been charged the via="name" legend comment it will write (packBodies/packHops // write it ahead of the rows, after the walk) — so the budget the walk honours already holds those bytes bool* viaLegendCharged = nullptr; + // owner-hop (verbs_for.h THE QUESTION'S OWNER): an owner row lists its callees up to a RUNAWAY GUARD instead of the + // sixteen a ranked row shows, and a cut listing carries the pasteable full list. Defaulted to the ranked shape, so + // every pre-existing caller is byte-identical. `cappedNext` is already XML-escaped and written only when cut. + std::uint32_t rowCap = 16; + std::string_view cappedNext; }; // The wrapper, written in front of the rows it describes (they have to be walked before `shown` // is known). pageview.h THE TRUNCATION VOCABULARY rule 3: shown=/capped= are added ONLY when the 16-row // cap or the byte budget actually cut the list, and they are added TOGETHER — never shown="T" capped="0" // on a complete listing. -inline void appendCallsBlock( std::string& out, std::uint32_t total, int shown, const std::string& rows ) +// `nextEscaped` (owner rows only): the continuation that lists the whole set, written beside capped="1" and only then. +inline void appendCallsBlock( std::string& out, std::uint32_t total, int shown, const std::string& rows, std::string_view nextEscaped = {} ) { char callsHdr[ 64 ]; - if( static_cast( shown ) < total ) + const bool cut = static_cast( shown ) < total; + if( cut ) { - rw::formatTo( callsHdr, sizeof( callsHdr ), "", total, shown ); + rw::formatTo( callsHdr, sizeof( callsHdr ), "", total ); + rw::formatTo( callsHdr, sizeof( callsHdr ), " nameRows; // names-only rendering: collected, merged by name, written after the walk int shown = 0; - for( std::uint32_t k = outOff[id]; k < outOff[id + 1] && shown < 16 && used < budgetBytes; ++k ) + for( std::uint32_t k = outOff[id]; k < outOff[id + 1] && shown < int( sink.rowCap ) && used < budgetBytes; ++k ) { const NodeId cid = walk[ k - outOff[id] ]; if( cid >= ing.symbols.size() ) @@ -6597,7 +6725,7 @@ inline void emitCalleeCallsBlock( std::string& out, NodeId id, const std::vector } } appendMergedCalleeNameRows( callsBody, nameRows, esc ); // no-op on the signature rendering (nameRows stays empty) - appendCallsBlock( out, total, shown, callsBody ); + appendCallsBlock( out, total, shown, callsBody, sink.cappedNext ); } // THE DISCLOSURE (§B8.3). This was the one budgeted section element carrying NO attributes at all: @@ -7348,6 +7476,137 @@ inline void packBodies( std::FILE* out, const IngestResult& ing, const std::vect w.flush(); } +// ── THE QUESTION'S OWNER ROWS (lane owner-hop-068; the selection is verbs_for.h forOwnerHopPlan) ────────────────── +// The first `rows` nodes handed to packHops are OWNER rows: a ranked callable definition whose whole name equals a word +// of the question. Each prints `qword=""`, lists its callees up to kForOwnerHopNameGuard (a runaway guard, the +// kMaxExpandSibs precedent — never a target) and, when that listing is cut, says where the whole of it is +// (``). Owners the plan could not seat (the row cap) or the budget could not fund are counted +// in ``, the continuation naming the first of them. +inline constexpr std::uint32_t kForOwnerHopNameGuard = 100; +struct HopOwners +{ + std::size_t rows = 0; // the first `rows` nodes are owner rows, in slot order + std::vector words; // the folded question word of each owner row (one per row) + std::size_t budgetBytes = 0; // 0: ride ON TOP of the hop budget (default regime, guard only) + bool fundFirst = false; // explicit ceiling: owners are funded first and the ranked rows get + // what they leave of budgetBytes (never more than their own budget) + std::size_t capCut = 0; // owners with a proven edge past the row cap (never seated) + NodeId firstCapCut = kNoNode; +}; + +// The continuation that lists one owner's callees: the canonical id when the symbol has a scope (two same-named methods +// of one file are two entities, and FILE:NAME would merge them), else FILE:NAME. Escaped for an attribute value. +inline std::string ownerCalleesNext( const IngestResult& ing, const Symbol& s, std::string_view pathRel, std::string_view rootArg, + std::vector& esc ) +{ + std::string target = s.scope.empty() ? std::string( pathRel ) + ":" + std::string( s.name ) : canonicalIdForEmit( ing, s, rootArg ); + return "--callees=" + std::string( escapeXml( target, esc ) ); +} + +// What the owner walk hands back to packHops: the rows, how many nodes they consumed, and the budget and via-legend charge +// the RANKED rows then run on, plus the cut disclosure (`cutNext` already escaped; empty when nothing was cut). +struct OwnerHopRows +{ + std::string xml; + std::size_t rows = 0; // leading nodes that were owner rows (served or budget-cut) + std::size_t shown = 0; + std::size_t rankedBudget = SIZE_MAX; // the ranked rows' budget: unchanged, or (explicit ceiling) what the owners left + bool rankedVia = false; // the ranked rows' via-legend charge state + std::size_t cut = 0; // qword_cut=: cap cuts + budget cuts + std::string cutNext; +}; + +// The owner walk (packHops' first rows). Default regime: their own byte room, unlimited, the name guard the only stop, and +// their own via-legend charge — so the ranked rows are charged exactly what they were charged without owners. Explicit +// ceiling (fundFirst): funded first; the ranked rows get the rest. A row that cannot pay for its identity and its next= is +// not started: it is counted as cut, so an owner row never drops its names without saying where the rest are. +template +inline OwnerHopRows packOwnerHopRows( const IngestResult& ing, const std::vector& nodes, const HopOwners* owners, + const std::vector& outOff, const std::vector& outTargets, + const std::vector& outNameOnly, RedactCounts* redact, const std::vector* rank, + std::string_view rootArg, std::size_t rankedBudget, bool rankedVia, + PathRelFn&& pathRel, ContentOfFn&& contentOf, std::vector& esc ) +{ + OwnerHopRows out; + out.rankedBudget = rankedBudget; + out.rankedVia = rankedVia; + if( owners == nullptr ) + { + return out; + } + EXPECTS( owners->words.size() == owners->rows, "one question word per owner row" ); + out.rows = std::min( owners->rows, nodes.size() ); + out.cut = owners->capCut; + std::size_t used = 0; + const std::size_t budget = owners->budgetBytes > 0 ? owners->budgetBytes : SIZE_MAX; + bool via = rankedVia; + NodeId firstCut = kNoNode; // the first owner the BUDGET cut (it ranks before every cap cut) + for( std::size_t i = 0; i < out.rows; ++i ) + { + const NodeId id = nodes[ i ]; + // forOwnerHopPlan seats only indexed symbols, each with a proven callee edge (hopSlotHasProvenEdge, read there) + ASSUME( id < ing.symbols.size(), "the owner plan seats only indexed symbols (forOwnerHopPlan)" ); + const Symbol& s = ing.symbols[ id ]; + const std::string next = ownerCalleesNext( ing, s, pathRel( s.fileId ), rootArg, esc ); + std::string row = "words[ i ], esc ); + row += "\">"; + const std::size_t rowFixed = row.size() + next.size() + 8 + 16; // + ' next=""' + '' + if( used + rowFixed >= budget ) + { + firstCut = firstCut == kNoNode ? id : firstCut; + ++out.cut; + continue; + } + used += rowFixed; + emitCalleeCallsBlock( row, id, outOff, outTargets, ing, contentOf, esc, used, budget, + CalleeCallsSink{ redact, /*recorded=*/nullptr, /*namesOnly=*/true, rank, &outNameOnly, &via, + kForOwnerHopNameGuard, next } ); + out.xml += row; + out.xml += ""; + ++out.shown; + } + if( owners->fundFirst ) + { + out.rankedBudget = std::min( rankedBudget, owners->budgetBytes > used ? owners->budgetBytes - used : 0 ); + out.rankedVia = rankedVia || via; + } + const NodeId first = firstCut != kNoNode ? firstCut : owners->firstCapCut; + if( out.cut > 0 ) + { + ASSUME( first < ing.symbols.size(), "a counted owner cut names an indexed symbol (forOwnerHopPlan / the walk above)" ); + out.cutNext = ownerCalleesNext( ing, ing.symbols[ first ], pathRel( ing.symbols[ first ].fileId ), rootArg, esc ); + } + return out; +} + +// The open tag: the truncation triple, noedge= when non-zero, and the owner cut disclosure when there is one. +inline std::string hopsOpenTag( std::size_t shown, std::size_t requested, std::size_t noEdge, const OwnerHopRows& own ) +{ + char open[ 128 ]; + if( noEdge > 0 ) + { + rw::formatTo( open, sizeof( open ), " 0 ) + { + tag += " qword_cut=\"" + std::to_string( own.cut ) + "\" next=\"" + own.cutNext + "\""; // DISCLOSE the owners not served + } + tag += ">"; + return tag; +} + // ── THE SECTION: one-hop edge context without the body text ──────────────────────────────────── // (pre-registered: docs/EVALS.md, the T3 route-narrowing round; the allowance is // kForCompactHopBudgetBytes above, with its derivation.) @@ -7367,8 +7626,9 @@ inline void packBodies( std::FILE* out, const IngestResult& ing, const std::vect // group for, and walking `nodes` in the order the caller ranked them spends the budget on the // best-ranked symbols first. // -// A candidate with no resolved out-edges gets NO row and is counted in `noedge=` instead — see the loop -// for why that is the honest form rather than the cheap one. Call edges are name-based, so dynamic +// A candidate with no resolved out-edges gets NO row and is counted in `noedge=` instead — and so does an OWNER of the +// question (THE QUESTION'S OWNER ROWS above) with none, exactly once whether or not it was also a ranked candidate — see +// the loop for why that is the honest form rather than the cheap one. Call edges are name-based, so dynamic // dispatch, callbacks and macros can be missing: `noedge=` means none FOUND, never none exists // (CONTRIBUTING #3), and it is deliberately a separate number from `capped=`, which says only that the // byte budget stopped before the remaining candidates. @@ -7377,10 +7637,10 @@ inline void packHops( std::FILE* out, const IngestResult& ing, const std::vector const std::vector& outOff, const std::vector& outTargets, const std::vector& outNameOnly, // FE-B: Graph::outNameOnly — the rows' via="name" RedactCounts* redact = nullptr, - std::size_t* outShown = nullptr, // rows actually emitted; nullptr ⇒ not recorded const std::vector* rank = nullptr, // query relevance, for ordering a CUT callee listing std::string_view rootArg = {}, // R-E: same single-root-only root= every verb takes - bool viaLegendInHead = false ) // FE-B: packBodies' parameter of the same name + bool viaLegendInHead = false, // FE-B: packBodies' parameter of the same name + const HopOwners* owners = nullptr ) // owner-hop: the leading owner rows; nullptr ⇒ none (byte-identical) { bool viaLegendCharged = viaLegendInHead; // FE-B: the via="name" legend has been charged against the budget (CalleeCallsSink) if( budgetBytes == 0 ) // 0 ⇒ UNLIMITED, packBodies' own convention @@ -7424,12 +7684,19 @@ inline void packHops( std::FILE* out, const IngestResult& ing, const std::vector return contents.emplace( fid, std::move( s ) ).first->second; }; - std::string children; - std::size_t requestedCount = 0; - std::size_t shownCount = 0; + // owner-hop: the leading owner rows first (THE QUESTION'S OWNER ROWS above; nothing when `owners` is null), then the + // ranked rows exactly as before, from the node the owners end at, on the budget and via charge the owners leave them + const OwnerHopRows own = packOwnerHopRows( ing, nodes, owners, outOff, outTargets, outNameOnly, redact, rank, rootArg, + budgetBytes, viaLegendCharged, pathRel, contentOf, esc ); + std::string children = own.xml; + std::size_t requestedCount = own.shown; + std::size_t shownCount = own.shown; std::size_t noEdgeCount = 0; - for( NodeId id : nodes ) + budgetBytes = own.rankedBudget; + viaLegendCharged = own.rankedVia; + for( auto it = nodes.begin() + std::ptrdiff_t( own.rows ); it != nodes.end(); ++it ) { + const NodeId id = *it; if( id >= ing.symbols.size() ) { continue; // never a request this function could answer @@ -7482,29 +7749,15 @@ inline void packHops( std::FILE* out, const IngestResult& ing, const std::vector // capped= is the BUDGET bit only: the candidates neither printed nor accounted by noedge=. Emitted // as the 0|1 boolean the vocabulary requires either way; noedge= follows the house silence rule and // appears only when it is non-zero. - char open[ 128 ]; - if( noEdgeCount > 0 ) - { - rw::formatTo( open, sizeof( open ), "", - shownCount, requestedCount, shownCount + noEdgeCount < requestedCount ? 1 : 0, noEdgeCount ); - } - else - { - rw::formatTo( open, sizeof( open ), "", - shownCount, requestedCount, shownCount < requestedCount ? 1 : 0 ); - } + const std::string openTag = hopsOpenTag( shownCount, requestedCount, noEdgeCount, own ); if( !viaLegendInHead && children.find( " via=\"name\"" ) != std::string::npos ) { w.write( viaNameLegendComment() ); // FE-B: exactly when a hop's row carries via="name" } - w.write( open ); + w.write( openTag ); w.write( children ); w.write( "" ); w.flush(); - if( outShown ) - { - *outShown = shownCount; - } } // ── M6 (density audit 2026-08-08): the WHOLE-FILE form a bare --expand can serve ───────────────────── @@ -7868,8 +8121,10 @@ inline std::string outlineSkeleton( const std::string& src, const Symbol& s, boo // , byte-identical). total= counts the valid requests with a body to outline (a module scope has // none, the packBodies bodyless rule); a skeleton whose span is unreadable is not shown and so also cuts. inline void packOutline( std::FILE* out, const IngestResult& ing, const std::vector& nodes, std::size_t budgetBytes, bool compress, RedactCounts* redact, - std::string_view rootArg = {} ) // R-E (2026-08-17): same single-root-only root + std::string_view rootArg = {}, // R-E (2026-08-17): same single-root-only root // argument serialize() takes — see its comment. + std::vector* shownIdsOut = nullptr ) // expand-lean-k64: the ids this call EMITTED an for + // (nullptr ⇒ not recorded; output unchanged either way) { XmlWriter w( out ); std::vector esc; @@ -7934,6 +8189,10 @@ inline void packOutline( std::FILE* out, const IngestResult& ing, const std::vec piece += safe; piece += "]]>"; used += safe.size(); ++shownCount; + if( shownIdsOut != nullptr ) + { + shownIdsOut->push_back( id ); + } pieces.push_back( PackedBodyPiece{ slot, id, SIZE_MAX, false, std::move( piece ) } ); } std::stable_sort( pieces.begin(), pieces.end(), []( const PackedBodyPiece& x, const PackedBodyPiece& y ) { return x.fileSlot < y.fileSlot; } ); @@ -8169,6 +8428,82 @@ inline SectionStubPricing priceSectionStub( const char* tag, std::size_t preCapT return p; } +// One stubbable --for section ( or ) as the stub plan reads it. +struct ForStubbableSection +{ + bool hasContent; // the section would render non-empty + std::size_t preCapTotal; // its own pre-cap row count: the stub's total= + bool rendered; // its bytes were measured (false = the open_memstream degrade path, never collapsed) + std::size_t renderedBytes; // the measured full render (0 when !rendered) +}; + +// The / stub decision of a ranked --for answer: which section collapses (L2 / R2-L2', priceSectionStub) and +// the ONE restoring spelling. One function for the CLI lens (verbs_for.h runForLens) and its MCP twin (mcpverbs.h +// forTaskText), for both the final decision and the charge (forSectionsSigCharge below). +struct ForSectionStubPlan +{ + std::string nextInvocation; // empty when neither section is a candidate + SectionStubPricing lego{}; + SectionStubPricing compose{}; + bool legoWillStub = false; + bool composeWillStub = false; + + // the bytes the two sections occupy as served: a collapsing section at its stub, any other at its measured render. + // kForSectionStubLegend is not in it: a disclosure clause, exempt from the charge like the e=/owner/tail clauses. + [[nodiscard]] std::size_t servedBytes( const ForStubbableSection& legoIn, const ForStubbableSection& composeIn ) const noexcept + { + return ( legoWillStub ? lego.stubXml.size() : legoIn.renderedBytes ) + ( composeWillStub ? compose.stubXml.size() : composeIn.renderedBytes ); + } +}; + +inline ForSectionStubPlan planForSectionStubs( std::string_view sections, std::string_view forTask, const ForStubbableSection& legoIn, + const ForStubbableSection& composeIn ) +{ + ForSectionStubPlan p; + // a CANDIDATE has content and was not opted back in by --sections=; it collapses only when its stub is cheaper + const bool legoCandidate = legoIn.hasContent && !sectionsWant( sections, "lego" ); + const bool composeCandidate = composeIn.hasContent && !sectionsWant( sections, "compose" ); + if( legoCandidate || composeCandidate ) + { + p.nextInvocation = nextFlag( "--for=", forTask ); + p.nextInvocation += ' '; + p.nextInvocation += nextFlag( "--sections=", "lego,compose" ); + } + if( legoCandidate ) + { + p.lego = priceSectionStub( "lego", legoIn.preCapTotal, p.nextInvocation, legoIn.rendered, legoIn.renderedBytes ); + } + if( composeCandidate ) + { + p.compose = priceSectionStub( "compose", composeIn.preCapTotal, p.nextInvocation, composeIn.rendered, composeIn.renderedBytes ); + } + p.legoWillStub = legoCandidate && p.lego.collapse; + p.composeWillStub = composeCandidate && p.compose.collapse; + const std::size_t served = p.servedBytes( legoIn, composeIn ); + ENSURES( served <= legoIn.renderedBytes + composeIn.renderedBytes, + "planForSectionStubs: a collapse made the two sections bigger than their measured renders" ); + return p; +} + +// lego-charge (K18 root cause): what / cost the budget, computed BEFORE the sigs render. It used to be +// the full pre-rendered bytes, while the default answer serves a collapsing section as a ~120 B stub: textual's 6.5 KB lego +// left 71 B for (4 rows shipped in a 4.6 KB answer under a 7.5 KB ceiling). Now a section is charged as it is SERVED. +// Two rules decide what "served" means here: +// * Without an explicit ceiling, the charge is the DEFAULT answer's, also under --sections=: the stub's next= promises the +// sections back "byte-identically in one call" (kForSectionStubLegend), and only a call whose match the stubbed +// answer's renders them so (the lego is narrowed to the rendered sigs' files, §P3קP4). The restored sections then ride +// on top of that answer, priced in est_tokens=; charging them in full would serve FEWER rows than the stub promised. +// * Under an explicit ceiling (--token-budget / MCP budget_tokens), a section --sections= opts into is +// charged at its full render: the ceiling is a hard bound, and the stub's next= never carries one. +// The charge is taken before the §P3קP4 narrow, on the pre-narrow render and count. The narrow only empties implementor +// lists, so a served stub's total= can only shrink (runForLens ENSURES it). A section served whole was charged at its +// pre-narrow render, as it was before this change. +inline ForSectionStubPlan forSectionsSigChargePlan( std::string_view sections, bool explicitCeiling, std::string_view forTask, + const ForStubbableSection& legoIn, const ForStubbableSection& composeIn ) +{ + return planForSectionStubs( explicitCeiling ? sections : std::string_view(), forTask, legoIn, composeIn ); +} + // B6.3 HTTP-route cross-service view: for a set of relevant symbols, emit the synthesized route USE→DEF // edges (client call → server handler) as a block. ONLY called from --for and --around (NOT the // default map) — see model.h RouteEdge / graph.h buildGraph's B6.3 section for how these are matched. @@ -9406,6 +9741,7 @@ inline void writeJsonMapHeader( JsonWriter& w, std::string& esc, const JsonMapHe // (every MCP client) reading a truncated ranking with no key to tell it so is the exact defect the XML // side just closed. Same slot, same absent-means-converged rule. w.write( renderDisclosure( h.ann->prDisclosure, DiscloseAs::JsonKeys ) ); + w.write( renderHistoryUnread( DiscloseAs::JsonKeys ) ); // K51: the JSON twin of the root's history_unread= (absent unless the amp= walk was unread) // The JSON twin of the XML root's data_sections_cut= / next= (codeFirstKeep): same keys, same absent-at-zero rule. if( h.dataSectionsCut.cut > 0 ) @@ -9438,6 +9774,64 @@ inline void writeJsonMapHeader( JsonWriter& w, std::string& esc, const JsonMapHe w.write( "]" ); } +// D7: the JSON twin of writeMapInventoryXml — `,"inv":{"listed":L,"unlisted":U,"entry":[{"p":…,"n":…,"c":[…]}], +// "ls":[{"p":…,"n":N,"f":[…]}]}` after the "r" array. The callee objects spell exactly what a "r" row's "c" array does. +template +inline void writeMapInventoryJson( JsonWriter& w, const IngestResult& ing, const MapInventory& inv, const PathRel& pathRel, + const std::vector& outOff, const std::vector& outTargets, + const std::vector* outProv, const std::vector* outNameOnly, + std::string& esc ) +{ + EXPECTS( inv.active, "the caller emits the tier only when it has a cut to inventory" ); + w.write( ",\"inv\":{\"listed\":" ); w.write( std::to_string( inv.listed ) ); + w.write( ",\"unlisted\":" ); w.write( std::to_string( inv.unlisted ) ); + // each guard with its recovery argv (rule 5), as the XML twin prints them + if( inv.entryTotal > inv.entries.size() ) { w.write( ",\"entries_capped\":true,\"entries_total\":" + std::to_string( inv.entryTotal ) + ",\"entries_next\":" ); writeJsonStr( w, inv.entriesNext, esc ); } + if( inv.namesCapped ) { w.write( ",\"names_capped\":true,\"names_total\":" + std::to_string( inv.namesTotal ) + ",\"names_next\":" ); writeJsonStr( w, inv.namesNext, esc ); } + w.write( ",\"entry\":[" ); + for( std::size_t i = 0; i < inv.entries.size(); ++i ) + { + const NodeId id = inv.entries[ i ]; + const Symbol& s = ing.symbols[ id ]; + w.write( i ? ",{\"p\":" : "{\"p\":" ); writeJsonStr( w, pathRel( s.fileId ), esc ); + w.write( ",\"n\":" ); writeJsonStr( w, s.name, esc ); + w.write( ",\"c\":[" ); + for( std::uint32_t e = outOff[ id ]; e < outOff[ id + 1 ]; ++e ) + { + w.write( e > outOff[ id ] ? ",{\"n\":" : "{\"n\":" ); writeJsonStr( w, ing.symbols[ outTargets[ e ] ].name, esc ); + if( outProv && e < outProv->size() && ( *outProv )[ e ] ) + { + w.write( ",\"prov\":" ); writeJsonStr( w, provLabel( ( *outProv )[ e ] ), esc ); + } + if( outNameOnly && e < outNameOnly->size() && ( *outNameOnly )[ e ] != 0 ) + { + w.write( ",\"via\":\"name\"" ); + } + w.write( "}" ); + } + w.write( "]}" ); + } + w.write( "],\"ls\":[" ); + for( std::size_t i = 0; i < inv.dirs.size(); ++i ) + { + const MapInventoryDir& d = inv.dirs[ i ]; + w.write( i ? ",{\"p\":" : "{\"p\":" ); writeJsonStr( w, d.dir, esc ); + w.write( ",\"n\":" ); w.write( std::to_string( d.total ) ); + if( !d.named.empty() ) + { + w.write( ",\"f\":[" ); + for( std::size_t j = 0; j < d.named.size(); ++j ) + { + if( j ) { w.write( "," ); } + writeJsonStr( w, namesplit::afterLast( pathRel( d.named[ j ] ), "/" ), esc ); + } + w.write( "]" ); + } + w.write( "}" ); + } + w.write( "]}" ); +} + // The default-map JSON sibling of serialize() — same rank/order/bucket logic (kept a deliberate, mechanical // duplication per the L2 seam decision: a sibling emitter, not a shared-emission refactor of the XML path, // so the G5 byte-identical-default contract carries zero risk from this addition). Scope: the common single- @@ -9510,6 +9904,9 @@ inline void serializeJson( std::FILE* out, const IngestResult& ing, const std::v if( !seen[f] ) { seen[f] = 1; fileOrder.push_back( f ); } buckets[f].push_back( id ); } + // D7: the XML twin's inventory, the same selection (mapinventory.h) over the same printed-file set + const MapInventory inventory = mapInventoryWanted( ing, ann ) ? computeMapInventory( ing, outOff, outTargets, seen, pathRel ) + : MapInventory{}; // T1/§H7: SAME byte-model the XML sibling uses, in the same two roles — the fill-order oracle and the // rate source. The REPORTED est_tokens is measured from the emitted bytes in PHASE 2 below, exactly as @@ -9670,7 +10067,12 @@ inline void serializeJson( std::FILE* out, const IngestResult& ing, const std::v } w.write( "]}" ); } - w.write( "]}" ); + w.write( "]" ); + if( inventory.active ) + { + writeMapInventoryJson( w, ing, inventory, pathRel, outOff, outTargets, outProv, outNameOnly, esc ); + } + w.write( "}" ); }; // DEGRADE, one path for both failures: the header FIRST with the MODELLED estimate, the array streamed behind it. const auto emitModelled = [ & ]() diff --git a/src/taskroute.h b/src/taskroute.h index 85f6435db..820df9003 100644 --- a/src/taskroute.h +++ b/src/taskroute.h @@ -15,6 +15,7 @@ #include "infra/jsonesc.h" // shSingleQuote — the repository's canonical POSIX argv quoting #include "model.h" // IngestResult +#include "queryshape.h" // detail::lowerAscii — the one ASCII lowercaser (K50) #include "query.h" // isKnownLayerWord — the layer vocabulary --verify enforces at evaluation #include "sarif.h" // rootRelativeUri / rootPrefixOf — the ONE root-relative path rule the map emits with #include "verify.h" // parseClaim — the SHIPPED claim grammar; the router never re-implements it @@ -62,16 +63,10 @@ struct TaskRouteResult int margin = 0; }; -inline std::string lowerAscii( std::string_view text ) -{ - std::string out; - out.reserve( text.size() ); - for( const unsigned char c : text ) - { - out.push_back( char( std::tolower( c ) ) ); - } - return out; -} +// K50: ONE ASCII lowercaser for the tree's task-text readers. This file used to spell its own loop (`for( const unsigned char c : +// text )`, an implicit sign change on every byte >= 0x80 that aborted the sanitizer build on a non-ASCII prompt), and the fix for it +// made that loop byte-identical to queryshape.h's, which --quality-delta flagged as a clone of the reused helper. It is that one now. +using queryshape::detail::lowerAscii; // "is this word one of these?" — the shape four tables' predicates were each writing out by hand, which is // how a five-line any_of becomes a duplication finding against its own neighbours. @@ -447,8 +442,9 @@ inline SymbolMention symbolMention( std::string_view task, std::string_view lowe // fall through to the weak tier, where Section-kind symbols are never weak evidence (weakEvidenceKind), // so the name could not resolve even when explicitly backtick-marked in the task text (backticks are // step 1 of identifierMentionShape, itself unreachable without this). - const bool hasMark = std::any_of( name.begin(), name.end(), []( const unsigned char c ) + const bool hasMark = std::any_of( name.begin(), name.end(), []( const char ch ) { + const unsigned char c = static_cast( ch ); // explicit narrowing (K50): a byte >= 0x80 must not convert implicitly return std::isupper( c ) || c == '_' || c == ':' || c == '.' || c == '$'; } ); if( hasMark ) @@ -626,7 +622,7 @@ inline std::string firstFileLineToken( std::string_view task ) const std::string_view lineDigits = token.substr( colon + 1 ); const std::string_view fileGuess = token.substr( 0, colon ); const bool allDigits = !lineDigits.empty() && std::all_of( lineDigits.begin(), lineDigits.end(), - []( const unsigned char c ) { return std::isdigit( c ) != 0; } ); + []( const char c ) { return std::isdigit( static_cast( c ) ) != 0; } ); // explicit narrowing (K50) if( allDigits && looksLikeFileToken( fileGuess ) ) { return std::string( fileGuess ) + ":" + std::string( lineDigits ); diff --git a/src/verbs_change.h b/src/verbs_change.h index b8d3afe04..06143e0c0 100644 --- a/src/verbs_change.h +++ b/src/verbs_change.h @@ -688,6 +688,13 @@ inline constexpr int kRunTraceExitCommandFailed = 4; // H7 (capture-audit 2026-09-04; hosts added at the wave-3 close): the ONE sentence for a --stray-content=SUBSTR // filter that selected no ref NAME — spoken by the bare verb ("" host) and by its two hosts ("--plan: ", "--abi: "). // refs="0" at exit 0 would read as "no branch carries stray work", the most reassuring answer these verbs give. +// K75: the blob batch behind a cross-branch verb never started or died part-way. The refusal names the cause where the reader +// looks (stderr, exit 1) — the other answer was merged="1" / "no break" for every branch, a measured-looking zero made of nothing. +inline void printBlobsUnreadRefusal( const char* hostPrefix ) +{ + DISCLOSE( Diagnostics::answerRefused, "the blob batch behind the cross-branch sweep did not serve every blob, so no verdict is printed - stderr names the cause, exit 1" ); + rw::emitTo( stderr, "ripwire: {}{} - refusing rather than print verdicts nothing measured\n", hostPrefix, rw::crossref::blobsUnreadCause( rw::crossref::UnreadBlobBatch::Branches ) ); +} inline void printStrayFilterNoMatch( const char* hostPrefix, std::string_view filter ) { rw::emitTo( stderr, "ripwire: {}--stray-content={} matches no local ref — a zero here would be a failure, not a " @@ -1416,6 +1423,10 @@ int runAbiCheck( const MainDispatch& d ) { printStrayFilterNoMatch( "--abi: ", d.cfg.strayFilter ); // H7, the host's spelling (wave-3 close) } + else if( result.blobsUnread ) + { + printBlobsUnreadRefusal( "--abi: " ); + } else { rw::emitTo( stderr, "ripwire: --abi: more than {} refs match — narrow it with --stray-content=SUBSTR\n", crossref::kMaxRefs ); @@ -1465,6 +1476,10 @@ std::optional runCrossRef( const MainDispatch& d ) { printStrayFilterNoMatch( "--plan: ", cfg.strayFilter ); // H7, the host's spelling (wave-3 close) } + else if( result.blobsUnread ) + { + printBlobsUnreadRefusal( "--plan: " ); + } else { rw::emitTo( stderr, "ripwire: --plan: more than {} refs match — narrow it with --stray-content=SUBSTR\n", crossref::kMaxRefs ); @@ -1499,6 +1514,10 @@ std::optional runCrossRef( const MainDispatch& d ) // branch carries stray work" — from a sweep that matched no branch NAME at all. printStrayFilterNoMatch( "", cfg.strayFilter ); } + else if( result.blobsUnread ) + { + printBlobsUnreadRefusal( "--stray-content: " ); // K75 + } else { rw::emitTo( stderr, "ripwire: --stray-content: more than {} refs match — narrow it with --stray-content=SUBSTR\n", crossref::kMaxRefs ); @@ -1517,6 +1536,11 @@ std::optional runCrossRef( const MainDispatch& d ) const crossref::EvalReport rep = crossref::evalStray( root, std::string( cfg.evalStray ) ); if( !rep.ok ) { + if( rep.blobsUnread ) + { + printBlobsUnreadRefusal( "--eval-stray: " ); // K75 + return 1; + } if( !rep.badRefs.empty() ) { // H13: the file read fine and this IS a git repo — the refusal is that one or more @@ -1605,13 +1629,22 @@ std::optional runCrossRef( const MainDispatch& d ) // §A7: HEAD's rows are documented as the PARSED answer, so hand the tree scan what the index knows. const std::vector indexDefs = crossref::whereisIndexDefSites( d.ing, whereisSel, root ); + // --detail lifts both caps of this verb: the row cap below and the runaway guard on the other refs' blob parse. + const crossref::WhereisParseBudget parseBudget = cfg.detail ? crossref::WhereisParseBudget{ SIZE_MAX, SIZE_MAX } : crossref::WhereisParseBudget{}; crossref::WhereResult result = crossref::computeWhereis( root, whereisSel, cfg.strayFilter, - crossref::WhereisEvidence{ cfg.withHistory ? &history : nullptr, indexDefs } ); + crossref::WhereisEvidence{ cfg.withHistory ? &history : nullptr, indexDefs }, parseBudget ); if( !result.ok ) { rw::emitTo( stderr, "ripwire: --whereis: {} is not a git repository (or has no HEAD commit) — no refs to search\n", root.c_str() ); return 1; } + if( result.blobsUnread ) + { + DISCLOSE( Diagnostics::answerRefused, "whereis: the blob stream did not serve every blob, so no hit count is printed — stderr names the cause, exit 1" ); + rw::emitTo( stderr, "ripwire: --whereis: {} — refusing rather than print a hit count nothing measured\n", + crossref::blobsUnreadCause( crossref::UnreadBlobBatch::Trees ) ); // K75's one renderer: the sweep's words, this verb's noun + return 1; + } result.seedSpec = std::move( whereisSeed ); // The tree zero stays an answer; the near-miss only says WHICH zero it is (a name this repo never // had, or a keystroke away from one it has). Computed only on the zero, so a real hit list is diff --git a/src/verbs_for.h b/src/verbs_for.h index ddcfa33fc..73f237310 100644 --- a/src/verbs_for.h +++ b/src/verbs_for.h @@ -556,6 +556,11 @@ struct ForLensHeaderParts bool viaPresent = false; // FE-B: may a row of this bundle carry via="name"? Some head row // (the same row head modScopePresent reads) has a name-only out-edge // (serialize.h namesOnlyOutAny). Over-approximates the 16-per-symbol cap. + bool ownerPresent = false; // owner-hop: the compact carries a qword= row or a qword_cut= disclosure + // (forOwnerHopPlan, decided before the header, cleared with the section) + bool viaByOwnerOnly = false; // owner-hop: viaPresent holds ONLY because the owner rows joined the via question + // (the six ranked hop ids alone answer no) — the via clause is then the owner + // rows' disclosure, exempt from the sig trim exactly as the qword clause is // ── THE DROPPABLE LEGEND, as ONE bit ────────────────────────────────────────────────────────────── // confidenceNote / tailLegend / idRouteLegend moved in lock step at every read and every write, and the @@ -785,6 +790,41 @@ inline constexpr std::string_view kForCompactLegendConfidence = "; confidence=/margin_pct= head score drop (low=flat)"; inline constexpr std::string_view kForCompactLegendHops = "; h l= p= n=, c n= l= (joined for same-named callees, shown= counts them), noedge= no callee resolved"; +// owner-hop: present-only, right after the hops clause, on the same plan the rows come from. Says it is BY NAME (a lookup in +// the ranked head, never a resolution), widens noedge= to the owners, and defines the two disclosures an owner cut carries. +// What qword= does NOT mean: that the row is the only, or the true, implementation of the question's subject. +inline constexpr std::string_view kForCompactLegendOwner = + "; h qword= its name (case-folded, leading _#$@ dropped) equals that question word (by name, not resolved, not the only one), first, calls to 100 (capped=1 next= all); " + "qword_cut= owners not so served, next= the first"; +// owner-hop: the full dialect's reading of the same two attributes (present-only, after the clause above; no "--": a comment) +inline constexpr std::string_view kForOwnerHopLegend = + "; hops row qword= the question word that row's NAME equals (ASCII case-folded, leading _ # $ @ dropped; by name in the " + "ranked head, a lookup, not a resolution; it " + "does not mean the symbol is the only or the true implementation): such an owner row comes first (a word on a short " + "common-verb list such as get, set, run or write ranks after the other owner words) and its calls child " + "lists up to 100 callee names (capped=1 with next= naming the full list when the guard or the budget cut it); an owner " + "with no resolved callee is counted in noedge= like any ranked symbol. hops qword_cut=N: owners not served as a qword " + "row (the three-row cap or the budget; one of the six ranked candidates still keeps its plain row), next= the first"; +// …and the one question both dialects ask: the clause in this dialect, or nothing (ownerPresent is set only on the compact +// route and cleared with the section, so it never rides without the hops clause in front of it). +inline std::string_view ownerHopLegend( const ForLensHeaderParts& p, bool compact ) noexcept +{ + if( !p.ownerPresent ) + { + return {}; + } + return compact ? kForCompactLegendOwner : kForOwnerHopLegend; +} +// …and what the sig-trim ledger exempts for the owner rows in this dialect: the qword clause, plus the via="name" clause when +// the owner rows alone switched it on (a base answer without them never carried it, so charging it cost a row: +// fastify-13 Context r23, pdoc AstInfo r38). ownerPresent and viaPresent are cleared with their sections, so neither part +// is exempted for a clause the header did not write. +inline std::size_t ownerLegendExemptBytes( const ForLensHeaderParts& p, bool compact ) +{ + const bool viaByOwner = p.viaPresent && p.viaByOwnerOnly; + const std::size_t viaBytes = !viaByOwner ? 0u : ( compact ? rw::kForCompactViaNameClause.size() : rw::forViaNameClause().size() ); + return ownerHopLegend( p, compact ).size() + viaBytes; +} inline constexpr std::string_view kForCompactLegendBodies = "; b t= n= p= l= full bodies, c n= l= callee signatures"; inline constexpr std::string_view kForCompactLegendTail = @@ -894,6 +934,7 @@ inline void appendCompactForLegend( std::string& h, const ForLensHeaderParts& p, { h += kForCompactLegendBodies; } + h += ownerHopLegend( p, /*compact=*/true ); // owner-hop: present-only, right after the hops clause // The unconditionally-droppable-free tail of this dialect: five clauses that ride exactly when the bit beside // them is set, in THIS order, which is the contract a reader and every byte pin depend on. A table rather than // five identical ifs — the list has grown once per lane (hdr, compose, then train 9's lego and layer) and the @@ -1042,6 +1083,7 @@ inline std::string forLensHeaderText( const ForLensHeaderParts& p, bool withRout { h.append( kForAutoBundleLegend ); // T3: present whenever auto mode is on, whatever the fit outcome — it explains bodies="0" too } + h.append( ownerHopLegend( p, /*compact=*/false ) ); // owner-hop: present-only, right after the bundle clause if( p.tailLegend ) { h.append( rw::kForFileTailLegend ); // deep-tail: defines r= and the element (sigs-charge-exempt, serialize.h) @@ -1934,6 +1976,52 @@ std::size_t forSigSideCeiling( bool autoBundleMode, int packTopN, std::size_t bu return bundleBudget; } +// ── THE TRAILING-SECTION RESERVE under an explicit ceiling (lego-charge fix round 4, estchargecheck #11 A7) ── +// §F1 charged --with-graph's block and the --detail bodies, but only AFTER had already claimed the whole +// sig-side ceiling: the graph block (a FIXED cost, no budget knob) and the first --detail body (packBodies serves +// it whole or head-cut even at a 1-byte budget, with its section wrapper and the omitted-bodies comment) then rode +// past the stated budget. MEASURED on base with `src --for --token-budget=2000 --detail=20 --with-graph`: 6177 B +// ("parse the command line flags") and 6167 B ("emit the legend") against a 5428 B allowance, both over_ceiling="1" +// with the ladder's last-rung note claiming the HEADER floor left no payload to trim — the trimmable payload was +// , sized as if the two trailing sections cost nothing. The fix is the forSigSideCeiling idea applied to these +// two sections: under an explicit --token-budget the sig side's claim is capped so the graph block and a first-body +// floor fit inside the bundle budget. The default regime (no --token-budget) and a budgeted run with neither flag +// pass reserveBytes=0, so the claim is the unchanged sigSideCeiling (byte-identical by construction). The cap only +// binds when the bundle cannot hold sigSideCeiling AND the reserve, so a wide explicit ceiling in auto-bundle mode +// (sig side already frozen at the default share) is untouched too — forbudgetmonotoncheck's invariant holds. +std::size_t forSigClaimWithTrailingReserve( std::size_t sigSideCeiling, std::size_t bundleBudget, std::size_t reserveBytes ) noexcept +{ + if( reserveBytes == 0 ) + { + return sigSideCeiling; + } + const std::size_t room = bundleBudget > reserveBytes ? bundleBudget - reserveBytes : 0; + const std::size_t claim = std::min( sigSideCeiling, room ); + ENSURES( claim <= sigSideCeiling && claim + reserveBytes <= std::max( bundleBudget, reserveBytes ), + "forSigClaimWithTrailingReserve: the sig claim grew past its ceiling or left the reserve no room" ); + return claim; +} + +// The FIRST-BODY FLOOR's byte budget: the first --detail body served whole when it is at most a quarter of the +// bundle budget, else head-cut at that quarter (packBodies' truncateOversizedFirst path, which carries lines= and +// the next= that serves the rest — a disclosed, recoverable cut). Only the FIRST body is floored: the rest of +// --detail still takes whatever the ceiling leaves, exactly as before. 0 when there is no servable first body. +inline constexpr std::size_t kForDetailFloorShareDivisor = 4; // the floor's cap = bundleBudget / this +std::size_t forDetailFirstBodyFloor( const rw::IngestResult& ing, const std::vector& detailIds, std::size_t bundleBudget ) +{ + for( const rw::NodeId id : detailIds ) // packBodies' own skip rule: an invalid id or a module scope is no body + { + if( id >= ing.symbols.size() || ing.symbols[id].kind == rw::SymKind::ModuleScope ) + { + continue; + } + const rw::Symbol& s = ing.symbols[id]; + const std::size_t raw = s.endByte > s.sigStartByte ? s.endByte - s.sigStartByte : 0; + return std::min( raw, bundleBudget / kForDetailFloorShareDivisor ); + } + return 0; +} + // The auto-body candidate head: the top-kPackTaskBodyCandidates positive-score rows of the ranked // surface — the same "top heads with a positive score" rule packTaskBundleText applies, on the same // (score desc, id asc) order sigs selected with. @@ -2209,6 +2297,233 @@ inline std::vector forCompactHopIds( const std::vector& } return hopIds; } +// ── THE QUESTION'S OWNER (lane owner-hop-068) ───────────────────────────────────────────────────────────────────── +// The hop candidates above are the six best-RANKED rows, and a question that names the function it is about ("How does +// the JWT middleware verify a token?" -> jwt(), rank 12 behind a dozen symbols that repeat the question's other words) +// never got that function's callees — the graded gold items of two round-1 rows (getCookie, HTTPException, set; +// validateAsync*). An OWNER is a symbol of the ranked pool (the first kForLensDefaultTopN rows with a positive score) that +// * is a callable DEFINITION — a function or method with a body (a C prototype and its body are one entity: the +// body's row is the owner, the prototype is no second one) — and not a test symbol (rw::isTestSymbol); +// * has a name that, with leading _ # $ @ removed and ASCII case folded, EQUALS a whole word of the question (words +// split on every byte that is not a letter, digit, '_' or a non-ASCII byte; length >= 3; not a route stopword) — +// whole name only: "log" never makes catalog or dialog owners; `__init__` folds to init__ and matches no word; +// * names a word with at least one such definition inside the head tier (rank <= kForDocExcerptRankCount). The gate +// is per WORD: once a word qualifies, EVERY same-named definition in the pool is an owner, each its own row (two +// classes' _notify are two entities, never one picked silently). +// This is a NAME lookup over the ranked head, not a resolution: qword= says the name equals the word, never that the +// symbol is the only or the true implementation of the subject. The case fold matches the question's word only; it never +// merges two entities. Language-neutral by construction: a name, a kind, a body span and the test partition are what +// every indexed language already sets. +// SLOTS: owners with a proven callee edge (the hop-slot rule, rw::hopSlotHasProvenEdge) take up to kForOwnerHopRows rows +// in rank order — except that a word on kOwnerCommonWords (verbs and function words a question USES: get, then, write …) +// ranks after every other word, so "how does the gateway get and then write the token" seats gateway first (gate (V)). +// The rest are disclosed (qword_cut= + next=). An owner with no proven edge never takes a slot; it is counted in noedge= +// once (it is appended to the hop nodes only when it is not already one of the six). RIPWIRE_NO_OWNER_HOP=1 turns the +// whole plan off — the A/B handle, the RIPWIRE_NO_DOCS_AFTER_CODE precedent. +inline constexpr std::size_t kForOwnerHopRows = 3; +inline constexpr std::string_view kOwnerCommonWords[] = { + "all", "call", "each", "from", "get", "has", "into", "new", "put", "read", "run", "set", "that", "then", "this", + "use", "when", "with", "write" }; + +struct ForOwnerPlan +{ + std::vector rowIds; // the owner rows, slot order (<= kForOwnerHopRows) + std::vector rowWords; // the folded question word of each row + std::vector noEdgeIds; // owners with no proven edge that are NOT among the six candidates + std::size_t capCut = 0; // owners with a proven edge past the row cap + rw::NodeId firstCapCut = rw::kNoNode; + bool any() const noexcept { return !rowIds.empty() || capCut > 0; } // a qword row or a qword_cut= disclosure (the legend's bit) + bool changesNodes() const noexcept { return any() || !noEdgeIds.empty(); } // …or an edgeless owner to count in noedge= +}; + +// What the --for header already decided for the enrichment section it introduces: FE-B's "the header defines +// via=\"name\"" (ForLensHeaderParts::viaPresent) and the owner plan its qword clause read. One aggregate, so the section +// builders render exactly what the header promised without a parameter per promise. +struct ForSectionPromises +{ + bool viaLegendInHead = false; + const ForOwnerPlan* owners = nullptr; // nullptr ⇒ no owner rows (an empty plan reads the same) +}; + +// ASCII fold of one name or word, leading _ # $ @ removed (the owner match key) +inline std::string ownerFoldName( std::string_view n ) +{ + std::size_t i = 0; + while( i < n.size() && ( n[i] == '_' || n[i] == '#' || n[i] == '$' || n[i] == '@' ) ) + { + ++i; + } + std::string out( n.substr( i ) ); + for( char& c : out ) + { + c = ( c >= 'A' && c <= 'Z' ) ? char( c - 'A' + 'a' ) : c; + } + return out; +} + +// the question's distinct owner words, sorted (binary-searched below — any question length, no per-word scan) +inline std::vector ownerQuestionWords( std::string_view task ) +{ + std::vector words; + const auto isWordByte = []( unsigned char c ) noexcept + { return ( c >= 'a' && c <= 'z' ) || ( c >= 'A' && c <= 'Z' ) || ( c >= '0' && c <= '9' ) || c == '_' || c >= 0x80; }; + std::size_t i = 0; + while( i < task.size() ) + { + if( !isWordByte( static_cast( task[i] ) ) ) + { + ++i; + continue; + } + const std::size_t start = i; + while( i < task.size() && isWordByte( static_cast( task[i] ) ) ) + { + ++i; + } + std::string w = ownerFoldName( task.substr( start, i - start ) ); + if( w.size() >= 3 && !rw::isRouteStopword( w ) ) + { + words.push_back( std::move( w ) ); + } + } + std::sort( words.begin(), words.end() ); + words.erase( std::unique( words.begin(), words.end() ), words.end() ); + return words; +} + +static_assert( std::ranges::is_sorted( kOwnerCommonWords, rw::sortutil::svLess ), "kOwnerCommonWords is binary-searched: keep it sorted" ); + +inline ForOwnerPlan forOwnerHopPlan( const rw::IngestResult& ing, const rw::Graph& g, std::string_view task, + const std::vector& lensSurfaceIds, const std::vector& lensRank, + const std::vector& hopIds ) +{ + ForOwnerPlan plan; + if( std::getenv( "RIPWIRE_NO_OWNER_HOP" ) != nullptr ) + { + return plan; + } + const std::vector words = ownerQuestionWords( task ); + if( words.empty() ) + { + return plan; + } + struct Owner + { + rw::NodeId id; + std::size_t rank; // 1-based position in the ranked surface + std::string word; + }; + std::vector found; + const std::size_t pool = std::min( lensSurfaceIds.size(), std::size_t( rw::kForLensDefaultTopN ) ); + for( std::size_t i = 0; i < pool; ++i ) + { + const rw::NodeId id = lensSurfaceIds[i]; + if( id >= ing.symbols.size() || lensRank[id] <= 0.0f ) + { + continue; + } + const rw::Symbol& s = ing.symbols[id]; + if( ( s.kind != rw::SymKind::Function && s.kind != rw::SymKind::Method ) || !rw::isDefinitionNotDeclaration( s ) || rw::isTestSymbol( ing, id ) ) + { + continue; + } + std::string key = ownerFoldName( s.name ); + if( std::binary_search( words.begin(), words.end(), key ) ) + { + found.push_back( Owner{ id, i + 1, std::move( key ) } ); + } + } + // the head-tier gate, per WORD: a word qualifies when one of its definitions ranks inside the head tier + std::vector headWords; + for( const Owner& o : found ) + { + if( o.rank <= rw::kForDocExcerptRankCount ) + { + headWords.push_back( o.word ); + } + } + std::sort( headWords.begin(), headWords.end() ); + std::erase_if( found, [ & ]( const Owner& o ) { return !std::binary_search( headWords.begin(), headWords.end(), o.word ); } ); + // rank order, common words after the rest (stable: rank order inside each group) + std::stable_partition( found.begin(), found.end(), + []( const Owner& o ) { return !std::ranges::binary_search( kOwnerCommonWords, std::string_view( o.word ), rw::sortutil::svLess ); } ); // svLess: #345 + for( Owner& o : found ) + { + if( !rw::hopSlotHasProvenEdge( g.outOff, g.outNameOnly, o.id ) ) + { + if( std::ranges::find( hopIds, o.id ) == hopIds.end() ) + { + plan.noEdgeIds.push_back( o.id ); // counted in noedge= once; a candidate already is + } + continue; + } + if( plan.rowIds.size() < kForOwnerHopRows ) + { + plan.rowIds.push_back( o.id ); + plan.rowWords.push_back( std::move( o.word ) ); + continue; + } + if( plan.capCut++ == 0 ) + { + plan.firstCapCut = o.id; + } + } + ENSURES( plan.rowIds.size() == plan.rowWords.size() && plan.rowIds.size() <= kForOwnerHopRows, "one word per seated owner, at most the cap" ); + return plan; +} + +// The node list with the owner plan applied: owner rows first, then the six candidates minus the owners MOVED out +// of them (moved, never duplicated), then the edgeless owners that are not candidates (each counted in noedge= once). +inline std::vector forOwnerHopNodes( const ForOwnerPlan& plan, const std::vector& hopIds ) +{ + std::vector nodes = plan.rowIds; + for( rw::NodeId id : hopIds ) + { + if( std::ranges::find( plan.rowIds, id ) == plan.rowIds.end() ) + { + nodes.push_back( id ); + } + } + nodes.insert( nodes.end(), plan.noEdgeIds.begin(), plan.noEdgeIds.end() ); + return nodes; +} + +// packHops' owner description of a plan. Default regime: the owner rows ride ON TOP of the hop budget (completeness first — +// every ranked row keeps the budget it had, and the name guard is the only stop). Explicit ceiling: the owners are funded +// first from what the ceiling left, minus the widest qword_cut/next= disclosure the walk could write (the envelope is the +// ranked rows' own reserve, kCompactFixedBytes, and is not charged twice). +inline rw::HopOwners forHopOwners( const rw::Config& cfg, const rw::IngestResult& ing, const ForOwnerPlan& plan, + std::size_t leftBytes, std::string_view rootArg ) +{ + rw::HopOwners owners; + owners.rows = plan.rowIds.size(); + owners.words.assign( plan.rowWords.begin(), plan.rowWords.end() ); + owners.capCut = plan.capCut; + owners.firstCapCut = plan.firstCapCut; + if( cfg.tokenBudget == 0 || !plan.any() ) + { + return owners; + } + std::vector esc; + const std::string rootPrefix = rootArg.empty() ? std::string() : rw::sarif::rootPrefixOf( rootArg ); + std::size_t cutReserve = 0; // ' qword_cut="NNN" next="--callees=' + the longest target + '"' + std::vector cuttable = plan.rowIds; + if( plan.firstCapCut != rw::kNoNode ) + { + cuttable.push_back( plan.firstCapCut ); + } + for( rw::NodeId id : cuttable ) + { + const rw::Symbol& s = ing.symbols[ id ]; + const std::string_view rel = rootArg.empty() ? std::string_view( ing.files[ s.fileId ] ) + : rw::sarif::rootRelativeUri( ing.files[ s.fileId ], rootPrefix ); // packHops' pathRel + cutReserve = std::max( cutReserve, rw::ownerCalleesNext( ing, s, rel, rootArg, esc ).size() + 28 ); + } + owners.budgetBytes = leftBytes > cutReserve ? leftBytes - cutReserve : 1; // 1, never 0 (0 reads as "ride on top") + owners.fundFirst = true; + return owners; +} + // --detail=N's bodies: the top min(N, forTopN) symbols by (score desc, id asc) — the order the sigs use. One spelling, read // by the --detail section and by the header's via="name" question (FE-B). inline std::vector forDetailIds( const rw::IngestResult& ing, const std::vector& lensRank, int detail, int forTopN ) @@ -2224,11 +2539,79 @@ inline std::vector forDetailIds( const rw::IngestResult& ing, const return ids; } +// fix round 4 (rule 5, review rv-wrap-recipe-k29 F1): the compact section an exhausted explicit ceiling +// leaves out, as a counted marker instead of nothing — ``. +// total= counts the CANDIDATE head the section covers (ids the ingest knows) — not the owner rows: at a spent ceiling the +// answer must stay byte-identical with and without the owner-hop plan (ownerhopcheck (B2), owner-hop's signed contract: +// "owner rows add nothing"), and the next= answer serves those owner rows too. How many candidates have proven edges +// (noedge=) is not known without the render, so the marker does not claim it. next= is the same question without +// --token-budget, whose own compact allowance serves the hop rows. No candidates => no marker (nothing was cut). Markup +// bytes, charged through buildForEnrichment's markupBytes like the rendered section. +inline constexpr std::string_view kForHopsBudgetCutLegend = + "; hops shown=0 capped=1: this budget left out the call-hop rows (one hop of callees for each of total= top rows); next= serves them"; +// the hops cut's continuation (` next="--for=TASK"`) — one spelling for the marker, the partial-cut splice and the reserve +inline std::string forHopsCutNextAttr( const rw::Config& cfg ) +{ + return rw::nextAttrXml( rw::nextFlag( "--for=", cfg.forTask ) ); +} +// the bytes an explicit ceiling reserves out of the sig claim so the compact hops' cut disclosure always fits: the widest +// marker (``, total <= kPackTaskBodyCandidates, two digits) plus its +// present-only legend clause — the kCompactAttrReserve / kForFileTailShellReserve pattern. +inline std::size_t forHopsCutReserve( const rw::Config& cfg ) +{ + constexpr std::size_t kMarkerShell = std::string_view( "" ).size(); + return kMarkerShell + forHopsCutNextAttr( cfg ).size() + kForHopsBudgetCutLegend.size(); +} + +rw::ChargedSection forHopsBudgetCutMarker( const rw::Config& cfg, const rw::IngestResult& ing, const std::vector& hopIds ) +{ + rw::ChargedSection marker; + std::size_t total = 0; + for( const rw::NodeId id : hopIds ) + { + total += id < ing.symbols.size() ? 1u : 0u; + } + if( total == 0 ) + { + return marker; + } + marker.xml = ""; + marker.tokens = rw::tokensForEmittedBytes( marker.xml.size(), rw::kBytesPerTokenDefault ); + marker.isRendered = true; + return marker; +} + +// fix round 4 (rule 5): an explicit-budget the budget CUT names the call that serves the rest. A cut with no +// next= of its own (the qword_cut= disclosure already carries one) gets ` next=` spliced into the open tag; the bytes +// were held back from the rows' budget (buildForCompactHops' rowRoom), so the section stays inside what it was given. +// A rendered section with shown="0" capped="1" is spliced the same way and then reads as the exhausted-ceiling marker +// does (counted, continued). An uncut section (capped="0") is untouched. +void forHopsCutDisclose( rw::ChargedSection& section, std::string_view cutNext ) +{ + std::string& x = section.xml; + const std::size_t end = x.find( '>' ); + if( !section.isRendered || end == std::string::npos || x.compare( 0, 5, " carrying the count and the call that serves them (the same question without the + // budget, whose compact allowance is its own). The bytes are this section's and the ladder prices them. + out.section = forHopsBudgetCutMarker( cfg, ing, hopIds ); return out; } // the surface allowance MINUS this surface's own fixed disclosure — see kForCompactSurfaceBudgetBytes. constexpr std::size_t kCompactFixedBytes = kForCompactBundleLegend.size() + kCompactAttrReserve + kCompactWrapReserve; - const std::size_t hopBudget = std::min( leftBytes, rw::kForCompactSurfaceBudgetBytes > kCompactFixedBytes - ? rw::kForCompactSurfaceBudgetBytes - kCompactFixedBytes - : std::size_t( 1 ) ); + // fix round 4: under an explicit ceiling a cut carries next= (forHopsCutNextAttr), so its bytes come out of the + // rows' budget first; the default regime is unchanged (cutNext empty) + const std::string cutNext = cfg.tokenBudget > 0 ? forHopsCutNextAttr( cfg ) : std::string(); + const std::size_t rowRoom = leftBytes > cutNext.size() ? leftBytes - cutNext.size() : 1u; + const std::size_t hopBudget = std::min( rowRoom, rw::kForCompactSurfaceBudgetBytes > kCompactFixedBytes + ? rw::kForCompactSurfaceBudgetBytes - kCompactFixedBytes + : std::size_t( 1 ) ); + + // THE QUESTION'S OWNER (forOwnerHopPlan): owner rows first, the candidates they were not, the edgeless owners last; + // an empty plan leaves both exactly as they were (forOwnerHopNodes / forHopOwners). + const std::vector hopNodes = forOwnerHopNodes( ownerPlan, hopIds ); + const rw::HopOwners owners = forHopOwners( cfg, ing, ownerPlan, leftBytes, fcRootArg ); out.section = rw::chargeSection( [ & ]( std::FILE* f ) - { rw::packHops( f, ing, hopIds, hopBudget, g.outOff, g.outTargets, g.outNameOnly, redactPtr, /*outShown=*/nullptr, &lensRank, fcRootArg, - viaLegendInHead ); }, // g.outNameOnly also feeds the hop-slot rule (serialize.h noProvenEdge) + { rw::packHops( f, ing, hopNodes, hopBudget, g.outOff, g.outTargets, g.outNameOnly, redactPtr, &lensRank, fcRootArg, + viaLegendInHead, &owners ); }, // g.outNameOnly also feeds the hop-slot rule (serialize.h noProvenEdge) // MARKUP rate, not the body rate — and this is an honesty choice, not a copy-paste slip. The body // rate (3.80 B/tok) prices SOURCE TEXT; the compact section contains none, only tags, identifiers // and line numbers, which tokenize like the rest of the bundle. Charging structured markup at the @@ -2284,8 +2680,12 @@ ForAutoBodiesResult buildForCompactHops( const rw::Config& cfg, const rw::Ingest out.section = rw::ChargedSection{}; return out; } - out.attr = hopIds.empty() ? " bundle=\"compact\" bodies=\"0\" reason=\"no_candidates\"" + out.attr = hopNodes.empty() ? " bundle=\"compact\" bodies=\"0\" reason=\"no_candidates\"" : " bundle=\"compact\" bodies=\"0\" reason=\"compact-route\""; + if( !cutNext.empty() ) + { + forHopsCutDisclose( out.section, cutNext ); + } return out; } @@ -2300,12 +2700,12 @@ ForAutoBodiesResult buildForEnrichment( const rw::Config& cfg, const rw::IngestR const std::vector& lensSurfaceIds, const std::vector& lensRank, const ForEnrichmentPlan& plan, const std::vector& anchorDefs, rw::RedactCounts* redactPtr, std::size_t committedBytes, std::size_t bundleBudget, - bool viaLegendInHead ) // FE-B: the header defines via="name" (ForLensHeaderParts::viaPresent) + const ForSectionPromises& promised ) // FE-B via="name" in the head + the owner plan { const std::size_t committed = committedBytes + plan.attrReserve; ForAutoBodiesResult out = plan.compact - ? buildForCompactHops( cfg, ing, g, lensSurfaceIds, lensRank, committed, bundleBudget, redactPtr, viaLegendInHead ) - : buildForAutoBodies( cfg, ing, g, lensSurfaceIds, lensRank, committed, bundleBudget, redactPtr, anchorDefs, viaLegendInHead ); + ? buildForCompactHops( cfg, ing, g, lensSurfaceIds, lensRank, committed, bundleBudget, redactPtr, promised ) + : buildForAutoBodies( cfg, ing, g, lensSurfaceIds, lensRank, committed, bundleBudget, redactPtr, anchorDefs, promised.viaLegendInHead ); if( plan.compact ) { out.markupBytes = out.section.xml.size(); @@ -2322,8 +2722,11 @@ ForAutoBodiesResult buildForEnrichment( const rw::Config& cfg, const rw::IngestR // regime: the full row cap, bytes riding on top (est_tokens measures them). Explicit regime: rows fit the // residual the rendered bundle actually left; the shell's bytes were reserved ahead of the body walk // (kForFileTailShellReserve inside the enrichment's committed sum), so the disclosure always fits. +// fix round 4 (rule 5, review rv-wrap-recipe-k29 F1): under an explicit ceiling the cut tail carries `cutNextAttr` — +// the --for file page (forTailCutNextAttr) that lists every positive-score file — and the shell reserve the caller +// took (forTailShellReserve) includes it, so the disclosure still always fits. inline std::string renderForFileTailXml( const rw::FileTail& tail, std::size_t tokenBudget, - std::size_t bundleBudget, std::size_t spentBytes ) + std::size_t bundleBudget, std::size_t spentBytes, std::string_view cutNextAttr = {} ) { std::vector esc; if( tokenBudget == 0 ) @@ -2331,8 +2734,16 @@ inline std::string renderForFileTailXml( const rw::FileTail& tail, std::size_t t return rw::renderFileTailXml( tail, rw::kForFileTailShownCap, esc ); } const std::size_t tailAllowed = std::max( bundleBudget > spentBytes ? bundleBudget - spentBytes : 0u, - rw::kForFileTailShellReserve ); - return rw::renderFileTailXml( tail, rw::fileTailShownForBudget( tail, tailAllowed, esc ), esc ); + rw::kForFileTailShellReserve + cutNextAttr.size() ); + return rw::renderFileTailXml( tail, rw::fileTailShownForBudget( tail, tailAllowed, esc, cutNextAttr ), esc, cutNextAttr ); +} + +// The explicit-budget tail's cut continuation: the --for file page (one row per positive-score file, pageable), the +// one call that lists the files a budget left out of . Empty without --token-budget: the default regime's tail is +// unchanged (a named deferral — adding it there moves every default --for answer). +inline std::string forTailCutNextAttr( const rw::Config& cfg ) +{ + return cfg.tokenBudget > 0 ? rw::nextAttrXml( rw::forWidenNext( cfg.forTask ) ) : std::string(); } // R2-AF (round 2, S4): render the task's named-file/decl-impl-partner rows (rw::forNamedHeaderRows, @@ -2371,13 +2782,58 @@ inline std::string renderForHdrRowsXml( const rw::IngestResult& ing, const std:: return x; } +// the CLI half of the shared page (forpage.h owns the doc): write it to stdout. One implementation, +// three dialects — the CLI bundle verbs and the MCP twins all build the SAME document. +inline int emitForCandidatePage( const rw::IngestResult& ing, const std::vector& lensRank, const rw::AdaptiveCut& forCut, + const rw::ForCandidatePageReq& req ) +{ + // the CLI half of the shared page (forpage.h owns the doc): write it to stdout. One + // implementation, three dialects — the CLI bundle verbs and the MCP twins all build the SAME + // document. rw::emitRaw (#362 review's ask): the repo's own stdout writer, not the raw stream call. + const std::string doc = rw::forCandidatePageDoc( ing, lensRank, forCut, req ); + rw::emitRaw( stdout, doc.c_str() ); + return 0; +} + +// B2 (#362 review round 4): the argv fragment of every accepted flag that changes the lens ranking, +// echoed into the page's next= so a pasted continuation re-ranks identically. The bundle-shaping +// set (--adaptive/--anchor/--signatures-only/…) is REFUSED beside a window (cli.h), so it never +// appears here; these four are accepted because they select the ranking, not the bundle's shape. +inline std::string forPageRankFlagArgv( const rw::Config& cfg ) +{ + std::string s; + if( cfg.noRoute ) { s += " --no-route"; } + if( cfg.noMentionBoost ) { s += " --no-mention-boost"; } + if( cfg.noDocMention ) { s += " --no-doc-mention"; } + if( cfg.cochangeBoost ) { s += " --cochange-boost"; } + return s; +} + +// #362 review round 4, follow-up (quality bar): the windowed-page predicate, named once — the +// inline spelling appeared twice (runForLens's bundle path, runPackTask's B1 gate) and the second +// read pushed runPackTask past its complexity bar. +inline bool candidatePageWindowed( const rw::Config& cfg ) +{ + return cfg.tokenBudget != 0 && ( cfg.pageLimit > 0 || cfg.pageOffset > 0 ); +} + // knob-honesty-068 (orchestrator ruling C3, completed 2026-10-08): one RENDER of the --for lens over the ranking runForLens // computed once (`lr`). `sigsNextPayFromRows` is the charged continuation's payment mode (serialize.h // SigsCutContinuation::payFromRows); with `sigsNextTryOther` set, a run whose capped, charged document lands past its ceiling // writes NOTHING to stdout and asks for the run in the other payment mode (runForLens below decides which answer ships). -std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bool sigsNextPayFromRows, bool* sigsNextTryOther ) +// `forAtStamp` (train 26c, CodeRabbit on #383): the at= stamp, read ONCE per run by runForLens — every render shares it. +struct ForLensPassMode +{ + bool sigsNextPayFromRows = false; + bool* sigsNextTryOther = nullptr; + std::string_view forAtStamp; +}; +std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, const ForLensPassMode& mode ) { using namespace rw; + const bool sigsNextPayFromRows = mode.sigsNextPayFromRows; + bool* const sigsNextTryOther = mode.sigsNextTryOther; + const std::string forAtStamp( mode.forAtStamp ); const Config& cfg = d.cfg; const IngestResult& ing = d.ing; const Graph& g = d.g; @@ -2450,7 +2906,10 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo // positive-score file, ranked file-first (forpage.h says how), the follow-up the routing-loop ladder // showed completes answers where a body cannot. It owns its own root, so it bypasses the whole // bundle below; cli.h refuses every bundle-shaping flag beside it rather than ignoring one. - if( cfg.pageLimit > 0 || cfg.pageOffset > 0 ) + // PAGING-POC (issue #294): under --token-budget an explicit window is NOT the file page — it is + // the BUDGETED-BUNDLE CANDIDATE PAGE, served by the shared inline below. The file + // page keeps every windowless-budget call exactly as forwidencheck pins it. + if( ( cfg.pageLimit > 0 || cfg.pageOffset > 0 ) && cfg.tokenBudget == 0 ) { const ForFilePage page = computeForFilePage( ing, lensRank, lr.evidence ); const std::string pageXml = renderForFilePageXml( ing, page, ForPageRenderParts{ cfg.forTask, ctxRootOpen( cfg.forTask, routeNoteRaw, flRootArg ), @@ -2593,6 +3052,22 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo } ForConfidence forConf = deriveForConfidence( forCut, forTopN, forHomonymDecline ); + + // ── THE BUDGETED-BUNDLE CANDIDATE PAGE (paging PoC, issue #294): budget + explicit window ⇒ the + // resumable candidate page (shared inline; the un-paged bundle below stays byte-identical). + if( cfg.tokenBudget != 0 && ( cfg.pageLimit > 0 || cfg.pageOffset > 0 ) ) + { + // W3-N1/#294 review: the page redacts through packSignatures — the tally must be reported, + // exactly like the --json early return below (a redacted credential must never be silent). + const std::string rankFlags = forPageRankFlagArgv( cfg ); + const int pageRc = emitForCandidatePage( ing, lensRank, forCut, rw::ForCandidatePageReq{ cfg.forTask, "--for=", rankFlags, + /*pasteHandle=*/true, /*compactLegend=*/cfg.legend != "full", + routeNoteRaw, flRootArg, redactPtr, + flSingleRoot ? std::string_view( gitstamp::stampAt( std::string( root ) ) ) : std::string_view(), + cfg.tokenBudget, cfg.pageLimit, cfg.pageOffset } ); + reportRedactions( stderr, redactCounts ); // W3-N1: the page redacts — the tally reports + return pageRc; + } // L-W: coverage= rides the SAME sentence and the SAME byte exemption as confidence=/margin_pct= — but ONLY // on a THIN answer (owner decision 2026-09-12: present-only). The thin verdict is decided HERE, from the // resolved surface above and the top symbol's term share, and it drives three things at once: the @@ -2677,9 +3152,9 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo // flRootArg itself uses): "" on multi-root or a non-git root, same silence-means-omitted convention // every other stamped verb follows. ALWAYS present when non-empty, like confidence= — a root fact, // never budget-dropped, which is why it rides the SAME byte-exempt reserve as confidence= below. - // Computed once as the raw stamp (both dialects need it: XML wraps it as an attribute, JSON quotes - // it bare) so a single --token-budget run pays exactly one extra git subprocess, not two. - const std::string forAtStamp = flSingleRoot ? gitstamp::stampAt( root ) : std::string(); + // Computed once per RUN, in runForLens (both dialects need it: XML wraps it as an attribute, JSON quotes + // it bare), so a --token-budget run that renders in up to three payment modes pays the stamp's two git + // subprocesses once, not once per render (CodeRabbit on #383, verbs_for.h:3843; test/forrankordercheck.sh (10c)). const std::string forAtAttrStr = forAtStamp.empty() ? std::string() : ( " at=\"" + forAtStamp + "\"" ); // R2-AF (round 2, S4): resolved once, ahead of the header build below, so its legend clause can be @@ -2794,16 +3269,28 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo // node set is known before the header: the compact route's hop ids, T3's auto-body candidates, or --detail's head // (autoBundleMode excludes --detail). Their name-only out-edges decide it — over-approximating only the byte ladder // and the 16-per-symbol cap inside those sections, never the rows the header itself lists. + // owner-hop: THE QUESTION'S OWNER plan (forOwnerHopPlan), decided here, before the header, because the header's + // qword clause and its via="name" question both read it; the compact route's renders exactly this plan. + const ForOwnerPlan ownerPlan = ( autoBundleMode && plan.compact ) + ? forOwnerHopPlan( ing, g, cfg.forTask, lensSurfaceIds, lensRank, forCompactHopIds( lensSurfaceIds, lensRank ) ) + : ForOwnerPlan{}; bool forViaPresent = false; + bool forViaByOwnerOnly = false; // owner-hop: the six ranked hop ids alone answer no, the owner rows switch it on if( !g.outNameOnly.empty() ) { std::vector viaIds; if( autoBundleMode && plan.compact ) { - viaIds = forCompactHopIds( lensSurfaceIds, lensRank ); // only the hops packHops gives a slot can carry a row (the hop-slot rule drops a hop whose every callee // edge is name-only, so its via="name" rows never print) - std::erase_if( viaIds, [ & ]( rw::NodeId id ) { return !rw::hopSlotHasProvenEdge( g.outOff, g.outNameOnly, id ); } ); + const auto unslotted = [ & ]( rw::NodeId id ) { return !rw::hopSlotHasProvenEdge( g.outOff, g.outNameOnly, id ); }; + viaIds = forCompactHopIds( lensSurfaceIds, lensRank ); + std::erase_if( viaIds, unslotted ); + const bool rankedVia = rw::namesOnlyOutAny( g.outOff, g.outNameOnly, viaIds ); // the question without owners + std::vector ownerViaIds = ownerPlan.rowIds; // owner rows carry too + std::erase_if( ownerViaIds, unslotted ); + forViaByOwnerOnly = !rankedVia && rw::namesOnlyOutAny( g.outOff, g.outNameOnly, ownerViaIds ); + viaIds.insert( viaIds.end(), ownerViaIds.begin(), ownerViaIds.end() ); } else if( autoBundleMode && plan.autoBodies ) { @@ -2821,7 +3308,8 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo cfg.anchor, plan.autoBodies, plan.compact, cfg.legend == "compact", /*tailLegend=*/true, /*idRouteLegend=*/true, /*legendDropped=*/false, flRootArg, /*hdrLegend=*/!forHdrRows.empty(), forScPresent, forComposePresent, - forLegoPresent, forLayerPresent, forModScopePresent, forEndLinePresent, forViaPresent }; + forLegoPresent, forLayerPresent, forModScopePresent, forEndLinePresent, forViaPresent, + ownerPlan.any(), forViaByOwnerOnly }; const auto buildForHeader = [ & ]( bool withRouteAttr, bool withTaskEcho, std::string_view extraNotes ) { return forLensHeaderText( headerParts, withRouteAttr, withTaskEcho, extraNotes ); }; std::string headerStr = buildForHeader( /*withRouteAttr=*/true, /*withTaskEcho=*/true, {} ); @@ -3093,8 +3581,12 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo // admits are the ones it admitted without it (owner ruling: e= is exempt from the row budget). const std::size_t endLineLegendEmitted = !headerParts.endLinePresent ? 0u : ( compactLegendOn ? rw::kForCompactEndLineLegend.size() : rw::kForEndLineLegend.size() ); + // owner-hop: the qword clause — and the via="name" clause when only the owner rows switched it on — are disclosures + // on the same contract: exempt, so is byte-identical with and without the owner rows (the rows ride on top of + // the hop budget; their reading never costs a signature row). Gate: ownerhopcheck (S). + const std::size_t ownerLegendEmitted = ownerLegendExemptBytes( headerParts, compactLegendOn ); const std::size_t exemptBytes = adaptiveNote.size() + autoLegendBytes + confidenceExemptBytes + tailLegendEmitted + idRouteLegendEmitted - + endLineLegendEmitted; + + endLineLegendEmitted + ownerLegendEmitted; if( exemptBytes > headerStr.size() ) { DISCLOSE( "runForLens: header exemptions exceed the emitted header — the sig ledger would underflow; charging the header whole" ); @@ -3118,7 +3610,8 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo const std::size_t fullExempt = adaptiveNote.size() + enrichmentLegendBytesEmitted( plan, false ) + confidenceEarlyAttrsBytes + confidenceEarlyNoteBytes + forAtAttrStr.size() + rw::kForFileTailLegend.size() + idRouteParts.bytes() - + ( headerParts.endLinePresent ? rw::kForEndLineLegend.size() : 0u ); + + ( headerParts.endLinePresent ? rw::kForEndLineLegend.size() : 0u ) + + ownerLegendExemptBytes( headerParts, /*compact=*/false ); std::size_t fullCharged = fullExempt > fullHeaderBytes ? fullHeaderBytes : fullHeaderBytes - fullExempt; fullCharged -= std::min( fullCharged, rw::forZeroNoteBytes( fullHeader ) ); // lean-answers: exempt, as above chargedHeaderBytes = std::min( chargedHeaderBytes, fullCharged ); @@ -3126,12 +3619,71 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo // BOTH headers, so it measures only the sig room the compact dialect was given beyond its honest cost compactLedgerGapBytes = fullCharged - chargedHeaderBytes; // >= 0 by the min above } - const std::size_t fixedBytes = chargedHeaderBytes + legoStr.size() + composeStr.size() + routeStr.size() + 6; // + "" + // lego-charge: / cost what they are SERVED as — a collapsing section its stub, not its full render + // (rw::forSectionsSigChargePlan states both rules and why --sections= keeps the default's charge without a ceiling). + // A section the degrade path will stream unmeasured has no bytes here (legoStr/composeStr empty): blockCharge discloses it. + const bool explicitForCeiling = cfg.tokenBudget > 0; // the lens's only ceiling (--max-tokens bounds --detail bodies) + const rw::ForStubbableSection legoChargeIn{ .hasContent = !legoStr.empty(), .preCapTotal = legoPreCapCount, .rendered = legoPreRendered, + .renderedBytes = legoStr.size() }; + const rw::ForStubbableSection composeChargeIn{ .hasContent = !composeStr.empty(), .preCapTotal = composePreCapCount, + .rendered = composePreRendered, .renderedBytes = composeStr.size() }; + const rw::ForSectionStubPlan sigChargePlan = rw::forSectionsSigChargePlan( cfg.sections, explicitForCeiling, cfg.forTask, legoChargeIn, composeChargeIn ); + const std::size_t fixedBytes = chargedHeaderBytes + sigChargePlan.servedBytes( legoChargeIn, composeChargeIn ) + routeStr.size() + 6; // + "" // the auto bundle's SECTION SPLIT — the sig side's claim is capped so an explicit ceiling wider // than the default cannot re-inflate the trimmed sig tail at the bodies' expense (the rule, its // measured defect and the invariant: forSigSideCeiling above; gate: forbudgetmonotoncheck). const std::size_t sigSideCeiling = forSigSideCeiling( autoBundleMode, cfg.packTopN, bundleBudget ); - const std::size_t sigsBudget = sigSideCeiling > fixedBytes ? sigSideCeiling - fixedBytes : 1; // ≥1: 0 would mean "no budget" + + // §F1's graph block, rendered HERE (before ) rather than beside the bodies: it reads only lensRank, so the + // bytes are the same wherever it renders, and an explicit ceiling must know them before the sig side claims its + // share (fix round 4: forSigClaimWithTrailingReserve). Emission order is unchanged (detail, then graph). + rw::ChargedSection graphSection, detailSection; + if( cfg.withGraph ) + { + graphSection = rw::chargeSection( [ & ]( std::FILE* f ) { packGraphBlock( f, ing, lensRank, g.outOff, g.outTargets ); }, + rw::kBytesPerTokenDefault ); + if( !graphSection.isRendered ) + { + // the block streams uncharged below: the lens's own contract omits an est_tokens that left it out + DISCLOSE( blockCharge, ForLensBlockCharge::DisclosureWhy::SiblingBlockUnmeasured, "runForLens: the graph block streams uncharged — est_tokens omitted" ); + } + } + // fix round 4 (estchargecheck #11 A7): under an EXPLICIT ceiling the graph block and a first --detail body floor are + // reserved out of the sig side's claim, so cannot spend the bytes the two trailing sections will ship. The + // floor is MEASURED — the --detail section rendered at the floor budget (wrapper, the first body, the omitted-bodies + // comment) — not estimated. No explicit ceiling, or neither flag: reserve 0, claim unchanged (byte-identical). + const std::vector detailIds = cfg.detail > 0 ? forDetailIds( ing, lensRank, cfg.detail, forTopN ) : std::vector{}; // (score desc, id asc) — same order as the sigs + // + // THE PRE-REGISTERED KILL (fix round 4, reports/lego-charge.md): with BOTH --detail and --with-graph the reserve is OFF. + // Measured on the 32 round-1 questions at --token-budget=2000 --detail=3 --with-graph, reserving both sections cut the + // gold-relevant owner Reactive._set (textual-12, r=8) out of ; header + graph + rows through r=8 + any real first + // body exceed the allowance there, so no floor keeps the row AND meets the budget. That shape keeps the ranked rows' + // claim, the first body still gets its floor (head-cut with next=, recoverable), and the overshoot is disclosed by the + // root's over_ceiling="1" — every cut named, nothing reserved away. Each section alone keeps its reserve (no gold row lost). + const bool reserveTrailing = explicitForCeiling && !( cfg.detail > 0 && cfg.withGraph ); + const std::size_t detailFloorBudget = explicitForCeiling ? forDetailFirstBodyFloor( ing, detailIds, bundleBudget ) : 0u; + std::size_t trailingReserve = reserveTrailing ? graphSection.xml.size() : 0u; + if( reserveTrailing && detailFloorBudget > 0 ) + { + // a MEASURE, not a rendering the reader gets: it redacts the same text into a scratch tally, so a secret the + // real render redacts is still counted once + std::optional floorRedact = redactPtr != nullptr ? std::optional( *redactPtr ) : std::nullopt; + trailingReserve += rw::chargeSection( [ & ]( std::FILE* f ) + { packBodies( f, ing, detailIds, detailFloorBudget, g.outOff, g.outTargets, g.outNameOnly, cfg.compress, floorRedact ? &*floorRedact : nullptr, + /*ranges=*/nullptr, notesPtr, /*outEmitted=*/nullptr, /*truncateOversizedFirst=*/true, + /*withFileContext=*/false, flRootArg, &lensRank, headerParts.viaPresent ); }, + rw::kBytesPerTokenBody ).xml.size(); // a degraded measure is 0 bytes: the reserve falls back to the graph alone + } + const std::string forTailNext = forTailCutNextAttr( cfg ); // fix round 4: the explicit-budget tail's cut next= (empty otherwise) + if( explicitForCeiling ) + { + // fix round 4 (rule 5): the two cut disclosures an explicit ceiling owes — the tail's next= and, on the compact + // route, the hops cut (marker or spliced next= + its clause) — are reserved like the trailing sections, so + // disclosing a cut never pushes the answer over the budget it was cut for + trailingReserve += forTailNext.size() + ( plan.compact ? forHopsCutReserve( cfg ) : 0u ); + } + const std::size_t sigClaim = forSigClaimWithTrailingReserve( sigSideCeiling, bundleBudget, trailingReserve ); + const std::size_t sigsBudget = sigClaim > fixedBytes ? sigClaim - fixedBytes : 1; // ≥1: 0 would mean "no budget" // The two attributes SPLICED into the header AFTER the ceiling ladder has chosen a rung — est_tokens // (" est_tokens=\"NNNNNNNN\"", bounded well under 24 B: 8 digits covers ~100M tokens) and, when the @@ -3346,45 +3898,34 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo // whichever state they are already in (post-narrow on the happy path, since narrowing already ran // above; un-narrowed on the degrade path, since narrowing never runs there either) — the same state // each path's own render (buffered or direct-to-stdout) would use. - const bool sectionsWantLego = rw::sectionsWant( cfg.sections, "lego" ); - const bool sectionsWantCompose = rw::sectionsWant( cfg.sections, "compose" ); const std::size_t legoStubTotal = legoPreRendered ? legoPreCapCount : rw::legoPreCapRowCount( ing, legoScoped ); const std::size_t composeStubTotal = composePreRendered ? composePreCapCount : rw::composePreCapRowCount( ing, g.composeEdges, lensSurfaceIds ); - const bool legoHasContent = legoPreRendered ? !legoStr.empty() : legoStubTotal > 0; - const bool composeHasContent = composePreRendered ? !composeStr.empty() : composeStubTotal > 0; - // CANDIDATE for collapse — has content, and --sections did not already opt it back in. Round 1 - // stubbed every candidate unconditionally; round 2 (below) additionally PRICES it. - const bool legoCandidate = legoHasContent && !sectionsWantLego; - const bool composeCandidate = composeHasContent && !sectionsWantCompose; - // the ONE restoring invocation, built once and used at every site (the size-gate probe below, the - // buffered substitution, and the degrade-path fallback further down) that needs it — never a second, - // differently-spelled build. - std::string sectionsNextInvocation; - if( legoCandidate || composeCandidate ) - { - sectionsNextInvocation = rw::nextFlag( "--for=", cfg.forTask ); - sectionsNextInvocation += ' '; - sectionsNextInvocation += rw::nextFlag( "--sections=", "lego,compose" ); - } - // R2-L2' (round-2, priced re-registration of L2/B1, rv-prereg2 Amendment 1 R4): a candidate collapses - // ONLY WHEN CHEAPER — see rw::priceSectionStub (serialize.h) for the shared rule (posture-independent - // by construction) and the DEGRADE path it documents: no rendered bytes to gate on there, so the - // section never collapses and is streamed whole below. Pulled into one small function so this - // already-long lens carries none of the pricing branching itself. - const rw::SectionStubPricing legoPricing = legoCandidate - ? rw::priceSectionStub( "lego", legoStubTotal, sectionsNextInvocation, legoPreRendered, legoStr.size() ) - : rw::SectionStubPricing{}; - const rw::SectionStubPricing composePricing = composeCandidate - ? rw::priceSectionStub( "compose", composeStubTotal, sectionsNextInvocation, composePreRendered, composeStr.size() ) - : rw::SectionStubPricing{}; - const bool legoWillStub = legoCandidate && legoPricing.collapse; - const bool composeWillStub = composeCandidate && composePricing.collapse; + // R2-L2' (round-2, priced re-registration of L2/B1, rv-prereg2 Amendment 1 R4): a CANDIDATE (has content, not opted + // back in by --sections=) collapses ONLY WHEN CHEAPER — rw::planForSectionStubs / priceSectionStub (serialize.h) hold + // the one rule both surfaces share, its ONE restoring invocation (used at every site below: the buffered substitution + // and the degrade-path fallback), and the DEGRADE path it documents: no rendered bytes to gate on there, so the + // section never collapses and is streamed whole below. + const rw::ForSectionStubPlan stubPlan = rw::planForSectionStubs( + cfg.sections, cfg.forTask, + rw::ForStubbableSection{ .hasContent = legoPreRendered ? !legoStr.empty() : legoStubTotal > 0, .preCapTotal = legoStubTotal, + .rendered = legoPreRendered, .renderedBytes = legoStr.size() }, + rw::ForStubbableSection{ .hasContent = composePreRendered ? !composeStr.empty() : composeStubTotal > 0, .preCapTotal = composeStubTotal, + .rendered = composePreRendered, .renderedBytes = composeStr.size() } ); + const std::string& sectionsNextInvocation = stubPlan.nextInvocation; + const rw::SectionStubPricing& legoPricing = stubPlan.lego; + const rw::SectionStubPricing& composePricing = stubPlan.compose; + const bool legoWillStub = stubPlan.legoWillStub; + const bool composeWillStub = stubPlan.composeWillStub; // postcondition of the rule itself: whichever section actually collapses is, by construction, smaller // than what it replaced (the WHOLE point of pricing it) — never a stub that grew the answer. ENSURES( !( legoPreRendered && legoWillStub ) || legoPricing.stubXml.size() < legoStr.size(), "R2-L2': a lego stub collapsed without being smaller than the section it replaced" ); ENSURES( !( composePreRendered && composeWillStub ) || composePricing.stubXml.size() < composeStr.size(), "R2-L2': a compose stub collapsed without being smaller than the section it replaced" ); + // lego-charge: the budget priced a collapsing lego at its PRE-narrow stub; the narrow only empties implementor + // lists, so the stub served now is never longer than the one charged (total= can only shrink). + ENSURES( !( legoWillStub && sigChargePlan.legoWillStub ) || legoPricing.stubXml.size() <= sigChargePlan.lego.stubXml.size(), + "lego-charge: the served lego stub is longer than the stub the budget was charged for" ); std::string sectionsStubNote; // present-only legend clause (kForSectionStubLegend), spliced below if( legoWillStub || composeWillStub ) { @@ -3420,22 +3961,11 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo // budget knob, so its size is a FIXED cost; the bodies are the one section with a byte budget, so they // are the section that absorbs whatever the ceiling has left. Pricing the fixed cost first is what lets // the bodies' budget be exact. - rw::ChargedSection graphSection, detailSection; - if( cfg.withGraph ) - { - graphSection = rw::chargeSection( [ & ]( std::FILE* f ) { packGraphBlock( f, ing, lensRank, g.outOff, g.outTargets ); }, - rw::kBytesPerTokenDefault ); - if( !graphSection.isRendered ) - { - // the block streams uncharged below: the lens's own contract omits an est_tokens that left it out - DISCLOSE( blockCharge, ForLensBlockCharge::DisclosureWhy::SiblingBlockUnmeasured, "runForLens: the graph block streams uncharged — est_tokens omitted" ); - } - } + // (graphSection is rendered above, before — fix round 4 reserves its bytes out of the sig side.) // both kept alive past the render so the isRendered=false degrade path below re-emits the SAME set at - // the SAME budget (the map path's emitSection lambda has the identical contract) - std::vector detailIds; - std::size_t detailBodyBudget = 0; + // the SAME budget (the map path's emitSection lambda has the identical contract); detailIds is computed above + std::size_t detailBodyBudget = 0; // --detail=N (lever 3): importance-weighted detail — spend FULL bodies on only // the top-N ranked symbols (the head the rank identifies), leaving the rest as the signatures emitted @@ -3444,7 +3974,6 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo // the lens. N=0 emits nothing → byte-identical to a run without --detail. if( cfg.detail > 0 ) { - detailIds = forDetailIds( ing, lensRank, cfg.detail, forTopN ); // (score desc, id asc) — same order as the sigs // Composes with --max-tokens: when set, it bounds the body byte budget (same conservative rate the // map path uses). §F1: --token-budget SHAPES this lens (D10 — trims to fit, always exit 0), so it // has to bound the bodies as well; before this it bounded ONLY and the bodies rode along on @@ -3459,9 +3988,11 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo { const std::size_t spentBytes = headerStr.size() + forHdrXml.size() + sigsStr.size() + legoStr.size() + composeStr.size() + routeStr.size() + graphSection.xml.size() + 6 + headerSpliceReserve + droppedPositiveSpliceReserve + sectionsStubSpliceReserve - + rw::kForFileTailShellReserve; // deep-tail: the shell's reserved bytes (explicit regime only — this branch) + + rw::kForFileTailShellReserve + forTailNext.size(); // deep-tail: the shell's reserved bytes + its cut next= (explicit regime only — this branch) const std::size_t leftBytes = bundleBudget > spentBytes ? bundleBudget - spentBytes : 1; - detailBodyBudget = std::min( detailBodyBudget, leftBytes ); + // fix round 4: never below the first-body floor the sig side reserved (forDetailFirstBodyFloor) — the header's + // exempt disclosures are paid by this side, and they must not cut the one body the reserve was taken for + detailBodyBudget = std::min( detailBodyBudget, std::max( leftBytes, detailFloorBudget ) ); } detailSection = rw::chargeSection( [ & ]( std::FILE* f ) { packBodies( f, ing, detailIds, detailBodyBudget, g.outOff, g.outTargets, g.outNameOnly, cfg.compress, redactPtr, @@ -3488,16 +4019,17 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo enrich = buildForEnrichment( cfg, ing, g, lensSurfaceIds, lensRank, plan, routeAnchorDefs, redactPtr, headerStr.size() + forHdrXml.size() + sigsStr.size() + legoStr.size() + composeStr.size() + routeStr.size() + graphSection.xml.size() + 6 + headerSpliceReserve + droppedPositiveSpliceReserve + sectionsStubSpliceReserve - + ( cfg.tokenBudget > 0 ? rw::kForFileTailShellReserve : 0u ), + + ( cfg.tokenBudget > 0 ? rw::kForFileTailShellReserve + forTailNext.size() : 0u ), // deep-tail: under an explicit ceiling the tail SHELL's bytes are // reserved ahead of the body walk (the kAutoAttrReserve pattern) so // the disclosure always fits; the DEFAULT regime reserves nothing — // the tail rides on top there and the bodies stay byte-identical. - bundleBudget, headerParts.viaPresent ); + bundleBudget, ForSectionPromises{ headerParts.viaPresent, &ownerPlan } ); if( enrich.surfaceOff || enrich.legendOff ) { headerParts.autoBundle = headerParts.compactBundle = false; headerParts.viaPresent = false; // FE-B: the section that could carry it is gone (autoBundleMode excludes --detail) + headerParts.ownerPresent = false; // owner-hop: the same section headerStr = buildForHeader( /*withRouteAttr=*/true, /*withTaskEcho=*/true, {} ); } } @@ -3512,6 +4044,12 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo } const rw::ChargedSection& autoSection = enrich.section; const std::string& autoAttr = enrich.attr; + if( enrich.legendOff && !autoSection.xml.empty() ) + { + // fix round 4: the hops budget-cut marker rides a header whose compact legend was just dropped — its one + // present-only clause says what the empty element is (priced by the ladder through rootFinish) + sectionsStubNote += kForHopsBudgetCutLegend; + } // ── DEEP-TAIL d2: render the file-grain tail, funded LAST (see serialize.h kForFileTailShownCap) ── // Rendered after the bodies decision so the explicit regime spends only the RESIDUAL the rendered @@ -3523,7 +4061,8 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo const std::string tailStr = renderForFileTailXml( forFileTailShown, cfg.tokenBudget, bundleBudget, headerStr.size() + forHdrXml.size() + sigsStr.size() + legoStr.size() + composeStr.size() + routeStr.size() + graphSection.xml.size() + detailSection.xml.size() - + autoSection.xml.size() + autoAttr.size() + 6 + headerSpliceReserve + droppedPositiveSpliceReserve + sectionsStubSpliceReserve ); + + autoSection.xml.size() + autoAttr.size() + 6 + headerSpliceReserve + droppedPositiveSpliceReserve + sectionsStubSpliceReserve, + forTailNext ); // N1: the last rung's note DEFINES the root attribute it accompanies (over_ceiling= …), so the attribute is // never on a document whose legend does not explain it; the bracket spelling stays. It is PROSE ONLY now — @@ -3533,6 +4072,18 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo " [task_echo: dropped (ceiling)]", " [task_echo + route_attr: dropped (ceiling)]", " [over_ceiling= is 1 on the root: the header floor (verbatim task echo + fixed legend) exceeds this budget" " - no payload left to trim]" }; + // fix round 5 (owner ruling 2026-10-09, option B; reports/lego-charge.md): the both-trailing-sections shape under an + // explicit ceiling KEEPS its ranked rows (reserveTrailing above is off) and goes over the budget on purpose — answers + // first, a budget never silently removes an answer. There the header floor is NOT what overshoots (measured: a ~1.8 KB + // header under a 5.4 KB allowance), so blaming it was false. This note names what really rides past the budget. It + // is chosen only when the header as built fits the allowance by itself; a header floor that alone exceeds it keeps + // the note above (true for this shape: estchargecheck #11 A7B-N). Single-flag shapes still get the note above even + // where their header fits — pre-existing, a named follow-up. Spelled without double hyphens: it rides inside the + // header XML comment. + static constexpr rw::CeilingLadderNotes kNotesKeptRows{ + kNotes.echoDropped, kNotes.echoAndRouteDropped, + " [over_ceiling= is 1 on the root: the ranked rows are kept, not cut for the trailing sections, and with the" + " with-graph block and the first detail body at its floor they exceed this budget - kept, not trimmed]" }; // PR #135: the priced root this header will carry — every late splice and the est_tokens fixpoint, in // finishForLensHeader (above runForLens, with each splice's rationale). Assembled BEFORE the ladder so the @@ -3684,8 +4235,11 @@ std::optional runForLensPass( const MainDispatch& d, rw::LensRanking lr, bo } // M3: the ladder hands back the rung it took. That value — never a search of the emitted text — is // what puts over_ceiling="1" on the root below. + // fix round 5: the last rung's reason must be TRUE — the header floor only when the header alone overruns + const bool keptTrailingOverrun = explicitForCeiling && !reserveTrailing && headerStr.size() + 6 <= ladderCeiling; rw::CeilingLadderChoice chosen = rw::climbCeilingLadderBy( buildForHeader, headerStr, fitsExactCeiling, fitsCeiling, - /*hasRouteAttr=*/!routeNoteRaw.empty(), kNotes ); + /*hasRouteAttr=*/!routeNoteRaw.empty(), + keptTrailingOverrun ? kNotesKeptRows : kNotes ); headerStr = std::move( chosen.header ); rootFinish.lastRungFired = ( chosen.rung == rw::CeilingRung::OverCeiling ); // knob-honesty-068 (orchestrator rulings 2026-10-07): the recovery handle ALWAYS ships. Read off the FINISHED @@ -3817,14 +4371,17 @@ std::optional runForLens( const MainDispatch& d ) } rw::LensRanking lr = computeLensRanking( d, cfg.forTask, forCompactPosture( cfg ), /*fullDistribution=*/!cfg.candidates ); // deep-tail: the bundle serves the file-grain tail; candidates has no tail and keeps the H2 pruning + // the at= stamp (gitstamp.h): one read per run — every payment-mode render below shares it, like the ranking + const bool forSingleRoot = d.ing.realPaths.empty() && cfg.roots.size() == 1; + const std::string forAtStamp = forSingleRoot ? rw::gitstamp::stampAt( d.root ) : std::string(); // only an explicit --token-budget charges the handle (forSigsNextCharged), so only then can a render hand back if( cfg.tokenBudget == 0 ) { - return runForLensPass( d, std::move( lr ), /*sigsNextPayFromRows=*/true, nullptr ); + return runForLensPass( d, std::move( lr ), ForLensPassMode{ .sigsNextPayFromRows = true, .forAtStamp = forAtStamp } ); } const rw::RedactCounts tallyAtEntry = d.redactCounts; bool unpaidOver = false; - std::optional rc = runForLensPass( d, lr, /*sigsNextPayFromRows=*/false, &unpaidOver ); + std::optional rc = runForLensPass( d, lr, ForLensPassMode{ .sigsNextPayFromRows = false, .sigsNextTryOther = &unpaidOver, .forAtStamp = forAtStamp } ); if( !unpaidOver ) { return rc; @@ -3832,14 +4389,14 @@ std::optional runForLens( const MainDispatch& d ) ASSUME( !rc ); // the unpaid render that handed back wrote nothing d.redactCounts = tallyAtEntry; bool paidOver = false; - rc = runForLensPass( d, lr, /*sigsNextPayFromRows=*/true, &paidOver ); + rc = runForLensPass( d, lr, ForLensPassMode{ .sigsNextPayFromRows = true, .sigsNextTryOther = &paidOver, .forAtStamp = forAtStamp } ); if( !paidOver ) { return rc; } ASSUME( !rc ); // the paid render that handed back wrote nothing d.redactCounts = tallyAtEntry; - return runForLensPass( d, std::move( lr ), /*sigsNextPayFromRows=*/false, nullptr ); + return runForLensPass( d, std::move( lr ), ForLensPassMode{ .sigsNextPayFromRows = false, .forAtStamp = forAtStamp } ); } std::optional runTargetedViews( const MainDispatch& d ) @@ -4035,6 +4592,31 @@ std::optional runTargetedViews( const MainDispatch& d ) // kPackTask* constants) now lives in packtask.h (L4) as packTaskBundleText() — shared verbatim with the MCP // explore/pack_task verb (mcpverbs.h's packTaskText()). This handler only resolves CLI-specific inputs // (flags, MainDispatch pointers) and hands them to that ONE assembler. +// (#362 review, item 2): the budgeted-window branch, extracted — runPackTask's own complexity was +// the quality gate's finding (19 -> 28 as the paging round grew the branch inline). One helper, one +// job: refuse a partitioned window, serve the candidate page, report the redaction tally. +inline std::optional runPackTaskPage( const MainDispatch& d, const rw::LensRanking& lr, const std::string& task, + const rw::PackTaskInputs& in ) +{ + using namespace rw; + const Config& cfg = d.cfg; + const IngestResult& ing = d.ing; + if( cfg.partitionCount > 0 ) + { + rw::emitRaw( stderr, "ripwire: --partition and a --limit/--offset window are mutually exclusive — a partitioned bundle has N+1 slices and no single candidate order to page; drop one\n" ); + return 1; + } + const AdaptiveCut packCut = adaptiveCut( lr.rank, 5, std::size_t( cfg.packTopN > 0 ? cfg.packTopN : kForLensDefaultTopN ), true ); + const std::string packAtStamp = in.rootArg.empty() ? std::string() : gitstamp::stampAt( std::string( in.rootArg ) ); + const std::string packRankFlags = forPageRankFlagArgv( cfg ); + const int packPageRc = emitForCandidatePage( ing, lr.rank, packCut, rw::ForCandidatePageReq{ task, "--pack-task=", packRankFlags, + /*pasteHandle=*/true, /*compactLegend=*/cfg.legend != "full", + lr.routeNote, in.rootArg, + d.redactPtr, packAtStamp, cfg.tokenBudget, cfg.pageLimit, cfg.pageOffset } ); + reportRedactions( stderr, d.redactCounts ); // W3-N1: the page redacts — the tally reports + return packPageRc; +} + std::optional runPackTask( const MainDispatch& d ) { using namespace rw; @@ -4054,7 +4636,12 @@ std::optional runPackTask( const MainDispatch& d ) const std::string task( cfg.packTask ); // ── the routed+anchored lens ranking, shared verbatim with --for (all existing boosts apply) ─────────── - LensRanking lr = computeLensRanking( d, task ); + // B1 (#362 review round 4): a window pages the WHOLE positive-score distribution — a pruned ranking is + // exact only for its top-K, so total= and the far pages would depend on a performance optimisation + // (measured: 931 vs 1232 candidates on src/). The rule --for's bundle path already follows. The un-paged + // bundle keeps the H2 pruning: its consumers read only the head. + const bool packWindowPages = candidatePageWindowed( cfg ); + LensRanking lr = computeLensRanking( d, task, /*compactCandidate=*/false, /*fullDistribution=*/packWindowPages ); // Q3 per-file churn (mirror --for): only mined on the --for git pass, so here it stays empty/zero when // --for wasn't also given → the churn= attr is simply omitted by packSignatures (nullptr-safe). @@ -4080,6 +4667,16 @@ std::optional runPackTask( const MainDispatch& d ) // R-E (2026-08-17 harvest): same single-root condition every other verb's root= uses (sarif.h). in.rootArg = ( ing.realPaths.empty() && cfg.roots.size() == 1 ) ? cfg.roots[0] : std::string_view(); + // ── THE BUDGETED-BUNDLE CANDIDATE PAGE (paging PoC, issue #294): --pack-task under --token-budget + // with an explicit window pages its ranked candidates, mirroring --for (the shared inline). The cliff + // statistic is computed here ONCE (the same call --for's disclosure uses); --partition and a window + // refuse together — a partitioned bundle has N+1 slices and no single candidate order to page. + // (#362 review, item 2): the branch body lives in runPackTaskPage — this stays the one-line gate + if( cfg.tokenBudget != 0 && ( cfg.pageLimit > 0 || cfg.pageOffset > 0 ) ) + { + return runPackTaskPage( d, lr, task, in ); + } + // --partition=N: the FAN-OUT form. Same lens ranking, same PackTaskInputs, same // assembler; partition.h only decides WHICH slice each of the N+1 bundles is masked to and how the // per-agent budget divides. Handled before the single-bundle emission below because it replaces the diff --git a/src/verbs_grep.h b/src/verbs_grep.h index ee9285689..19bbed7f6 100644 --- a/src/verbs_grep.h +++ b/src/verbs_grep.h @@ -913,7 +913,7 @@ int emitGrepReport( const rw::Config& cfg, const rw::IngestResult& ing, const rw // R-H: the tier disclosure (helper above) — empty when nothing was held back. const std::string tierAttr = grepTierAttrs( tierReport, /*floorAlreadyEmitted=*/hitsCapped != 0 ); // N2: tier_budget= floors the root too // §R-J: unindexed_files_scanned=/unindexed_files_skipped=/unindexed_candidates_capped= (helper above). - const std::string auxAttr = grepUnindexedAttrs( aux ); + const std::string auxAttr = grepUnindexedAttrs( aux ) + renderHistoryUnread( DiscloseAs::XmlAttrs ); // K51: absent unless the amp= walk (--metrics) could not be read const std::string regexSkipAttr = grepRegexSkipAttrs( cfg, found, aux, /*floorAlreadyEmitted=*/hitsCapped != 0 || tierReport.budgetHit != nullptr ); std::string scanShortAttr; if( found.unreadableFiles != 0 ) @@ -954,6 +954,7 @@ int emitGrepReport( const rw::Config& cfg, const rw::IngestResult& ing, const rw /*collectionCapped=*/ hitsCapped != 0 ), // H8: the cap hits_capped= names floors the root hitsCapped, completeAttr, tierAttr.c_str(), corpusAttr.c_str(), auxAttr.c_str(), regexSkipAttr.c_str(), scanShortAttr.c_str(), rw::nextAttrXml( grepNext ).c_str() ); + rw::emitTo( stdout, "{}", renderHistoryUnread( DiscloseAs::LegendComment ) ); // K51: its own definition, behind the root it marks // G1 (2026-08-15 harvest): hits GROUP by file under , root-relative when this is a single-root // run (report-memgraph §F6: the absolute root prefix alone was 42.5% of a real --grep payload; the // repeated-per-hit path was report-octocode §F1's 31.4%). Byte-identical text within one file's group diff --git a/src/wrap.h b/src/wrap.h index dfcdbf1ab..1210fcf27 100644 --- a/src/wrap.h +++ b/src/wrap.h @@ -705,7 +705,7 @@ inline void wrapEmitCliFirst( const AgentTarget& row, const std::string& token, rw::emitTo( stdout, "# RECOMMENDED — this agent can run shell commands, so call the CLI directly. It costs\n" "# nothing until you invoke it, and it reads {}, so the paste block below IS the wiring:\n" - "{} . --for=\"\" --token-budget=2000\n" + "{} . --for=\"\"\n" "#\n" "# ...every follow-up call answers with the compact legend by default (terse definitions of only the\n" "# attributes the answer carries; the payload is byte-identical either way). Add --legend=full when a\n" @@ -808,7 +808,7 @@ inline void wrapEmitAgent( const std::string_view agent, const WrapListing& list { rw::emitRaw( stdout, "# ripwire -> aider (no MCP; feed a ranked repo map as read-only context)\n" - "ripwire . --for=\"\" --token-budget=2000 > .ripwire-map.txt\n" + "ripwire . --for=\"\" > .ripwire-map.txt\n" "aider --read .ripwire-map.txt\n" "# re-run the first line when the tree changes; the warm cache makes it ~instant.\n" ); } diff --git a/test/budgetpolicycheck.sh b/test/budgetpolicycheck.sh index 3cadcf38d..81c0c46af 100755 --- a/test/budgetpolicycheck.sh +++ b/test/budgetpolicycheck.sh @@ -355,5 +355,61 @@ head -1 "$TMP/rtb.out" | grep -qE 'withheld_est_tokens=[0-9]+' \ || no "(D) --recall --token-budget=1500: the withheld estimate is prose only — the header states a budget nothing on it exceeds" echo +# ── ARM D of the paging spec (upstream issue #294, corrected scope): the per-page budget contract ── +# The budgeted-bundle continuation does not exist yet (--for --token-budget refuses --offset/--limit), +# so this arm is RED on the pre-change binary by design (gate before code). Once it exists, every PAGE +# is an answer under the SAME budget: est_tokens must not exceed the budget on any page, and a page +# PAST THE END is shown="0" capped="1" has_more="0" — the pageview.h past-the-end contract — never an +# error, never a silent empty success (an agent's has_more loop terminates on the truth, not on a crash). +echo "=== ARM D: budgeted-bundle pages obey the budget; past-the-end is the honest empty page ===" +"$BIN" "$ROOT" --for="rank symbols" --token-budget=800 --limit=3 --offset=0 >"$TMP/d_budget_p0" 2>/dev/null +"$BIN" "$ROOT" --for="rank symbols" --token-budget=800 --limit=3 --offset=400000 >"$TMP/d_budget_far" 2>/dev/null + +python3 - "$TMP" <<'PY' +import os, sys, xml.etree.ElementTree as ET + +tmp = sys.argv[1] +problems = [] + +def load(name): + path = os.path.join(tmp, name) + if not os.path.exists(path) or os.path.getsize(path) == 0: + problems.append(f"{name}: no output — the budgeted-bundle page does not exist yet " + "(today --token-budget refuses --offset/--limit; that refusal IS this arm's recorded red)") + return None + raw = open(path, encoding="utf-8", errors="replace").read() + try: + return ET.fromstring(raw), raw + except ET.ParseError as e: + problems.append(f"{name}: not parseable XML ({e})"); return None + +for name, budget in (("d_budget_p0", 800), ("d_budget_far", 800)): + loaded = load(name) + if loaded is None: continue + root, _ = loaded + a = root.attrib + est = a.get("est_tokens") + if est is not None and int(est) > budget: + problems.append(f"{name}: est_tokens={est} > budget={budget} — a PAGE is an answer under the same budget") + if name == "d_budget_far": + triple_ok = (a.get("shown") == "0" and a.get("capped") == "1" and a.get("has_more") == "0") + if not triple_ok: + problems.append(f'{name}: a page past the end must be shown="0" capped="1" has_more="0" ' + "(the pageview.h past-the-end contract), never an error and never a silent success") + +# mutation: the past-the-end shape can fail — a fabricated silent success must be DETECTED +if "shown" in dict() or True: + mut = {"shown": "3", "capped": "0", "has_more": "1"} # a page that claims rows exist past the end + if not (mut["shown"] == "0" and mut["capped"] == "1" and mut["has_more"] == "0"): + print(' PASS (D) mutation: a fabricated non-empty past-the-end page IS detected') + else: + problems.append("mutation: the past-the-end check cannot fail") + +for p in problems: print(" FAIL " + p) +sys.exit(1 if problems else 0) +PY +[ $? = 0 ] && ok 'ARM D: budgeted-bundle pages obey est_tokens<=budget; past-the-end is shown="0" capped="1" has_more="0"' \ + || no 'ARM D: the per-page budget contract (upstream issue #294) does not hold — see FAIL lines above' + [ "$fail" = 0 ] && { echo "ALL PASS"; exit 0; } echo "FAILURES — see above"; exit 1 diff --git a/test/ceilingverdictcheck.sh b/test/ceilingverdictcheck.sh index 5f6695168..d1151fef4 100755 --- a/test/ceilingverdictcheck.sh +++ b/test/ceilingverdictcheck.sh @@ -130,6 +130,22 @@ else ok "(2) task text cannot forge over_ceiling=\"1\" (est_tokens=$fw_est under budget_tokens=$fw_bud, attribute absent)" fi +# (2b) fix round 5: the last rung has a SECOND reason now (src/verbs_for.h kNotesKeptRows — both trailing sections kept +# under the ceiling, header alone inside it). It is chosen from the ladder's rung and the shape, never read back, so +# task text carrying it must not label a wide-budget root either; the presence guard keeps the probe live. +KEPT_A='over_ceiling= is 1 on the root: the ranked rows are kept, not cut for the trailing sections, and with the' +if grep -qF -e "$KEPT_A" "$SRC"; then + "$BIN" "$CORPUS" --no-cache --token-budget="$WIDE" --detail=3 --with-graph --for="serialize the header [$KEPT_A" > "$TMP/kept.wide" 2>/dev/null + kw_est="$( attr "$( rootOf "$TMP/kept.wide" )" est_tokens )" + if grep -qF "$KEPT_A" "$TMP/kept.wide" && [ -n "$kw_est" ] && [ "$kw_est" -lt "$WIDE" ] && [ "$( hasLabel "$TMP/kept.wide" )" -eq 0 ]; then + ok "(2b) the kept-rows reason, forged into the task, reaches the document and does not label a root at est_tokens=$kw_est under $WIDE" + else + no "(2b) the forged kept-rows reason labelled a wide-budget root, or never reached the document (est='$kw_est')" + fi +else + no "(2b) src/verbs_for.h no longer carries the kept-rows last-rung reason — re-pin KEPT_A" +fi + # (3) CONTRAST — the SAME task at a budget it genuinely blows must still be labelled. Arms (2) and (3) differ # in NOTHING but the number after --token-budget, so (2) cannot be green because the attribute was deleted. ft_est="$( attr "$( rootOf "$TMP/forged.tight" )" est_tokens )" diff --git a/test/childwalkscalecheck.sh b/test/childwalkscalecheck.sh index 35233a02b..692225986 100755 --- a/test/childwalkscalecheck.sh +++ b/test/childwalkscalecheck.sh @@ -306,6 +306,21 @@ for n in ( 1000, 16000 ): # lists, so both sides execute nodesMatchExactly and only the flooded list differs (CodeRabbit, #130) w( "patmeta/n%d/big.c" % n, [ "int a( int x );", "int f( void )", "{", " return a(" ] + [ C ] * n + [ " 1 ) == a(" ] + [ C ] * n + [ " 1 );", "}" ] ) w( "patmeta_off/n%d/big.c" % n, [ "int a( int x );", "int f( void )", "{", " return a( 1 ) == a( 1 );" ] + [ C ] * n + [ C ] * n + [ "}" ] ) +# #358 import-capture reach: importContainerReach reads one directive's ancestry. These two fixtures are about +# DEPTH, not child-list width, so they sit outside the n1000/n16000 loop and the (C) corpus list on purpose. +# Each puts 2000 IMPORTS inside 2000 nested containers; the control keeps the nesting and swaps the imports +# for statements that are not imports, so both sides run the same tags pass over the same tree and only the +# import emission differs. A reach read that costs a descent PER HOP (an upward ts_node_parent walk) is +# 60-230x its control here; one descent from the root is under 3x. +D = 2000 +w( "reachjs/big.js", [ "{" * D ] + [ 'require( "./m" );' ] * D + [ "}" * D ] ) +w( "reachjs_off/big.js", [ "{" * D ] + [ "noop();" ] * D + [ "}" * D ] ) +w( "reachc/big.c", [ "#ifdef G%d" % i for i in range( D ) ] + [ '#include "m.h"' ] * D + [ "#endif" ] * D ) +w( "reachc_off/big.c", [ '#include "m.h"' ] * D ) +for d in ( "reachjs", "reachjs_off" ): + w( d + "/m.js", [ "int x;" ] ) +for d in ( "reachc", "reachc_off" ): + w( d + "/m.h", [ "int x;" ] ) PY # user-CPU seconds (user+sys) of one cold run of "$@" against corpus $1 @@ -528,6 +543,14 @@ b_pm_off="$( usercpu "$TMP/patmeta_off/n16000" "$BIN" --pattern='$X == $X' )" b_pm_walk="$( usercpu "$TMP/patmeta/n16000" "$BIN" --pattern='$X == $X' )" arm "(B36) nodesMatchExactly" "$b_pm_off" "$b_pm_walk" 8 0.30 "--pattern='\$X == \$X' over two 16000-comment argument lists vs the same pattern with the floods outside both lists" +# #358 — importContainerReach (src/ingest_importcap.h): ancestry read in ONE descent, not k ts_node_parent hops +b_rjs_off="$( usercpu "$TMP/reachjs_off" "$BIN" --top-k=100000 )" +b_rjs_on="$( usercpu "$TMP/reachjs" "$BIN" --top-k=100000 )" +arm "(B37) importContainerReach/js" "$b_rjs_off" "$b_rjs_on" 8 0.30 "2000 require() calls inside 2000 nested blocks vs the same nesting holding 2000 non-import calls" +b_rc_off="$( usercpu "$TMP/reachc_off" "$BIN" --top-k=100000 )" +b_rc_on="$( usercpu "$TMP/reachc" "$BIN" --top-k=100000 )" +arm "(B38) importContainerReach/c" "$b_rc_off" "$b_rc_on" 8 0.30 "2000 #include lines inside 2000 nested #ifdef vs the same 2000 lines at file scope" + # ── (C) byte-identical against a reference binary ──────────────────────────────────────────────────── echo echo "=== (C) byte-identical output vs RIPWIRE_REF_BIN ===" diff --git a/test/churnjsonstampcheck.sh b/test/churnjsonstampcheck.sh index e3ec111ba..ef7eda89c 100755 --- a/test/churnjsonstampcheck.sh +++ b/test/churnjsonstampcheck.sh @@ -40,6 +40,14 @@ python3 - "$TMP/churn.json" "$TMP/pagerank.json" > "$TMP/verdict.txt" <<'PY' import json, sys churn = json.load( open( sys.argv[1] ) ) plain = json.load( open( sys.argv[2] ) ) +# D7 (2026-10-08): "inv" is PAYLOAD, not header — the JSON twin of the map scope's element, which rides the +# pagerank map whose ranked rows left files out and never a churn-ranked map (not the map scope). It sits beside "r" +# and is compared as payload: the header keysets below exclude it, and INV_* says where it rode. +PAYLOAD = { "inv" } +print( "INV_PLAIN=" + str( int( "inv" in plain ) ) ) +print( "INV_CHURN=" + str( int( "inv" in churn ) ) ) +churn = { k: v for k, v in churn.items() if k not in PAYLOAD } +plain = { k: v for k, v in plain.items() if k not in PAYLOAD } extra = sorted( set( churn ) - set( plain ) ) missing = sorted( set( plain ) - set( churn ) ) print( "EXTRA=" + ",".join( extra ) ) @@ -62,6 +70,9 @@ else fi [ -z "$MISSING" ] && ok "churn JSON header drops no key the pagerank header carries" \ || no "churn JSON header is MISSING keys the pagerank header has: $MISSING" +# D7: the inventory payload rides the map scope (pagerank, top-k=3 over src/ leaves files out) and never churn +[ "$INV_PLAIN" = 1 ] && [ "$INV_CHURN" = 0 ] && ok "\"inv\" (payload) rides the pagerank map and not the churn map" \ + || no "\"inv\" placement: pagerank=$INV_PLAIN churn=$INV_CHURN (want 1/0) — the payload exclusion above would hide a header key" # (2) the values carry MEANING (not pinned to a sha/window that moves with the tree) if [ "$RANK_BY" = "churn" ]; then ok "rank_by == \"churn\""; else no "rank_by is '$RANK_BY', want \"churn\""; fi diff --git a/test/compactlegendcheck.sh b/test/compactlegendcheck.sh index 7e575a652..f35b4b9e9 100755 --- a/test/compactlegendcheck.sh +++ b/test/compactlegendcheck.sh @@ -551,9 +551,18 @@ probeFor() # the defs page whenever it lists MORE definitions than the all page under the row cap, whatever its bytes. Rides only a # page that carries listing=; nothing else moved. The OLD path keeps its pin: the whereis-listing=all twin holds the # whole-list page's legend at 700 B (measured 692, unchanged). +# RE-PINNED 2026-10-08 (lane/map-inventory-d7, D7), measured on this gate's fixture: ripwire.map/v1 892->1129, pin +# 910->1140 (measured + 10 rounded up to 10). The --top-k=3 probe leaves out three of the fixture's six files, so the map +# now carries the inventory tier (, ; no on this fixture) and its two +# present-only readings (+237 B with separators). Nothing else moved. The OLD path keeps its pin: the --no-inventory twin +# after the universe loop holds the same probe without the tier at 910 B. +# RE-PINNED 2026-10-08 (lane/expand-lean-k64): ripwire.expand-file/v1 360 -> 510 (measured 508, the --expand=distance probe). +# A name with several definitions now serves lean (no ride-along map), so the whole-file answer this probe reaches carries +# topk_default="0" (+112 B reading) and map_next= (+56 B reading) on its root, both present-only (compactlegend.h). The +# (U-twin-expand) arm below keeps the OLD 360 B pin for everything else in that legend. PIN_TABLE=' ripwire.edit-check/v1 610 595 -ripwire.map/v1 910 892 +ripwire.map/v1 1140 1129 ripwire.map-diff/v1 900 885 ripwire.pack-signatures/v1 770 759 ripwire.metrics/v1 1040 1021 @@ -618,7 +627,7 @@ ripwire.layout/v1 1220 1203 ripwire.pack-task/v1 990 974 ripwire.pack-top-n/v1 760 745 ripwire.expand/v1 280 265 -ripwire.expand-file/v1 360 342 +ripwire.expand-file/v1 510 508 ripwire.for/v1 775 761 ' pinFor() @@ -658,6 +667,39 @@ else fi ;; esac fi +# (U-twin-expand) the pre-2026-10-08 ripwire.expand-file/v1 pin (360 B), kept (PROCESS rule 4): the same --expand=distance probe +# with exactly the two lean-default readings cut out of a copy of the compact answer must still fit the OLD pin. The cut is +# taken only when BOTH readings occur once; any other premise FAILs (a cut that removes nothing proves nothing). +( cd "$REPO" && "$BIN" . --expand=distance --legend=compact >"$TMP/xft.c" 2>/dev/null "$TMP/xft.full" 2>/dev/null /dev/null && [ "$xftRest" -le 360 ]; then + ok "(U-twin-expand) expand-file compact prose minus the two lean-default readings ($xftCut B) is $xftRest B <= 360 B (the pre-lean pin)" + else + no "(U-twin-expand) expand-file compact prose minus the two lean-default readings ($xftCut B) is ${xftRest:-?} B > 360 B — the rest re-inflated" + fi ;; + esac +fi nXml=0; nXmlBad=0; nRefuse=0; nSkip=0; loopBytes=0; xmlVerbs=""; nDefBad=0 # (UG) rv-r1-L1-2: EVERY XML verb the flag universe reaches, EVERY instance of every attribute its DEFAULT answer carries, # defined `name=` in that answer's own legend (legendcoveragecheck (G)'s predicate, on this gate's fixture). No floor. @@ -796,6 +838,18 @@ if [ "$rcWall" -eq 0 ] && [ "$( leg schema "$TMP/wall.c" )" = "ripwire.whereis/v else no "(U-TWIN) --whereis-listing=all compact PROSE legend is '${wallLb:-none}' B (pin 700; exit $rcWall): $( leg legend "$TMP/wall.c" | head -c 200 )" fi +# (U-TWIN) the map without its inventory tier keeps the pre-D7 pin: the --top-k=3 probe's pin rose to 1140 B for the +# readings, and without this twin the rest of the map legend could grow inside that headroom unseen. --no-inventory serves +# the same ranked rows with no tier (mapinventorycheck (N)). 910 = the pre-D7 pin (measured 892). +( cd "$REPO" && "$BIN" . --top-k=3 --no-inventory >"$TMP/mnoinv.c" 2>/dev/null "$TMP/mnoinv.f" 2>/dev/null /dev/null && [ "$mniLb" -le 910 ]; then + ok "(U-TWIN) --top-k=3 --no-inventory compact PROSE legend is $mniLb B (<= the pre-D7 910 B pin; the tiered probe's pin is 1140)" +else + no "(U-TWIN) --top-k=3 --no-inventory compact PROSE legend is '${mniLb:-none}' B (pin 910; exit $rcMni): $( leg legend "$TMP/mnoinv.c" | head -c 200 )" +fi echo echo "=== (F) the DEFAULT is the default posture; --legend=full restores MORE legend over the SAME payload (L1) ===" @@ -1061,8 +1115,12 @@ done # unpaid clause defining it, est_tokens pricing the bytes delivered (P3's rule), and the P4 claim itself (compact no further # over than full). Near miss at 750: paying is what makes it fit — est_tokens <= budget in both dialects, next= rides, no # over_ceiling= and no unpaid clause (a label riding every capped answer would pass the 500 half alone). +# RE-PINNED 2026-10-10 (train 26c): the near miss moves 750 -> 900. lego-charge charges / at their served size +# and #362's rides the bundle, so this fixture's 4-row floor answer prices est_tokens=918 compact / 963 full +# (was 814 / 859): at 750 both dialects are still at the floor, unpaid and labelled; the paid next= first fits in both +# dialects at 850 (784 / 829) and at 900 with margin (855 / 868). The 500 half is unchanged (4 rows, over, the clause). p4floor_bad=0 -for tb in 500 750; do +for tb in 500 900; do for lg in compact full; do ( cd "$P4" && "$BIN" . --for="widget routine dispatcher" --token-budget=$tb --legend=$lg 2>/dev/null ) >"$TMP/p4f.$lg" done @@ -1092,7 +1150,7 @@ for tb in 500 750; do p4floor_bad=$(( p4floor_bad + 1 )); no "(P4 floor twin) --token-budget=$tb: compact est_tokens=${pc:-?} over the budget AND over the full answer's ${pf:-?}" fi done -[ "$p4floor_bad" -eq 0 ] && ok "(P4 floor twin) at 500 both dialects ship next= unpaid at the rank 1..4 floor with over_ceiling=\"1\", the unpaid clause and an honest est_tokens, the 750 run's paid next= fits with no label, compact never further over than full" +[ "$p4floor_bad" -eq 0 ] && ok "(P4 floor twin) at 500 both dialects ship next= unpaid at the rank 1..4 floor with over_ceiling=\"1\", the unpaid clause and an honest est_tokens, the 900 run's paid next= fits with no label, compact never further over than full" # (P5) the BUDGET LEDGER survives compaction (orchestrator rule, METHODOLOGY §9.3/§9.4: never cut silently). --pack-task's # full legend ends with "budget=N bytes (T-token target, ceiling C) | ranking: … | bodies: … | callers: … | notes: … | tests: diff --git a/test/countfloorcheck.sh b/test/countfloorcheck.sh index caaff53ca..145f9bf81 100644 --- a/test/countfloorcheck.sh +++ b/test/countfloorcheck.sh @@ -210,6 +210,7 @@ ROW_R2="$( enc_row "$OUT_R2" 'Walker::walk' )"; ROW_R3="$( enc_row "$OUT_R2" wal case "$ROW_R2" in *callers_floor=*) no "(R) Python self.walk() recursion is floored: $ROW_R2" ;; *) ok "(R) Python method recursing via self.walk(): no callers_floor" ;; esac case "$ROW_R3" in *callers_floor=*) no "(R) Python recursive module function is floored: $ROW_R3" ;; *) ok "(R) Python recursive module function walk_tree: no callers_floor" ;; esac R_R3="$( root_tag "$( run pyrec --safe-delete=walk_tree )" safe-delete )" +[ -n "$R_R3" ] && [ "$( attr "$R_R3" callers )" = 0 ] || no "(R) premise: walk_tree: $R_R3" # an empty answer must not read as "not floored" case "$R_R3" in *callers_floor=*|*' next='*) no "(R) --safe-delete=walk_tree is floored: $R_R3" ;; *) ok "(R) --safe-delete=walk_tree: no callers_floor, no next=" ;; esac OUT_RM="$( run crec --grep='n ? p' )"; ROW_PING="$( enc_row "$OUT_RM" pong )"; ROW_PONG="$( enc_row "$OUT_RM" ping )" [ "$( attr "$ROW_PING" callers )" = 1 ] && [ "$( attr "$ROW_PONG" callers )" = 2 ] || no "(R) premise: mutual recursion counts: [$ROW_PING] [$ROW_PONG]" @@ -410,7 +411,9 @@ for dialect in compact full; do && ok "(N/$dialect) $term defined in the legend of the $var answer" || no "(N/$dialect) $term is not defined in the $var legend" done done -case "$( run c --safe-delete=helper_dead --legend=compact )" in *callers_floor*|*uses_floor*|*sites_l*) no "(N) a legend defines a floor/sites term its answer does not carry" ;; *) ok "(N) present-only: no floor/sites term on an answer without them" ;; esac +OUT_N="$( run c --safe-delete=helper_dead --legend=compact )" +[ -n "$( root_tag "$OUT_N" safe-delete )" ] || no "(N) premise: no answer for helper_dead (an empty answer must not read as present-only)" +case "$OUT_N" in *callers_floor*|*uses_floor*|*sites_l*) no "(N) a legend defines a floor/sites term its answer does not carry" ;; *) ok "(N) present-only: no floor/sites term on an answer without them" ;; esac # ── O: determinism + xmllint ───────────────────────────────────────────────────────────────────────────── echo "=== O: determinism and well-formed XML ===" diff --git a/test/crashsweepcheck.sh b/test/crashsweepcheck.sh index d2b1cc35f..33a3b4912 100755 --- a/test/crashsweepcheck.sh +++ b/test/crashsweepcheck.sh @@ -294,6 +294,8 @@ cat > "$TMP/registry.tsv" <<'REGISTRY' ccjson.h ccCountLoc fopen 1 closes the only early exit is the failed open; nothing between open and fclose allocates clones.h findClones fopen 1 closes skips only a failed open; fclose right after the sized read clones.h findClonesType3 fopen 1 closes same shape as findClones +crossref.h catFileBatch fopen 1 closes returns only on a failed open; fclose after the list is written +crossref.h catFileBatch popen 1 closes the read loop leaves by break or its own condition only; pclose then unlink crossref.h evalStray fopen 1 closes returns only on a failed open; fclose after the read loop crossref.h streamBlobs fopen 1 closes returns only on a failed open; fclose after the list is written crossref.h streamBlobs popen 1 closes the read loop leaves by break/continue only; pclose then unlink @@ -317,7 +319,7 @@ infra/os.h spawn_sh open 1 closes in the forked child: dup2 onto stdin, close, t infra/os.h read_small open 1 closes one read, then ::close before the only return past the failed-open one (Linux /proc and /sys files) ingest_cache.h openOnce open 1 owned ReadFd's destructor closes it ingest_cache.h saveCache fdopen 1 closes adopts ExclTempFile's released fd; fclose on its own line; a failed fdopen ::closes the fd -ingest_crawl.h collectGitIgnored popen 1 closes the overflow break still reaches pclose +ingest_crawl.h runGitIgnoreProbe popen 1 closes the overflow break still reaches pclose ingest_docpass.h publishDocBridgeBlob fdopen 1 closes adopts ExclTempFile's released fd; fclose on its own line; a failed fdopen ::closes the fd lintrules.h loadLintRules fopen 1 closes skips only a failed open; fclose after the sized read main.cpp dispatchMain fopen 1 closes returns only on a failed open; fclose after the read loop diff --git a/test/crossrefcheck.sh b/test/crossrefcheck.sh index f210cde53..75c0d9a31 100755 --- a/test/crossrefcheck.sh +++ b/test/crossrefcheck.sh @@ -231,14 +231,40 @@ srcRow="$( grep -n 'engine\.' "$TMP/w2rows" | tail -1 | cut -d: -f1 )" # TWIN on the default listing (lean-answers lane): the same def rows in the same order (source before the doc), the # doc-quoting kind="ref" row not dropped but COUNTED — refs count= equals the whole list's kind="ref" rows. -"$BIN" "$R" --whereis=computeBudget >"$TMP/w2d" 2>/dev/null +# +# FOLD UPDATE (lane whereis-defs-fix): the three branches all inherit base's `int computeBudget( int frames )` at +# engine.cpp:3, so the definitions page lists that line ONCE, on the first branch by name, with refs="3" +# (foldSharedBranchDefs); the whole list above (--whereis-listing=all, the untouched twin) still prints each +# branch's own row. The arm now asserts the fold's arithmetic instead of row-for-row equality: every def row of the +# default is a def row of the whole list, in the same order, and the refs= counts (1 when absent) add up to the +# whole list's def rows — nothing is dropped, it is counted. +# +# FIX ROUND 2 (rule 4, by design): a branch row is now a PARSER's label, and a branch holding HEAD's own engine.h blob +# takes HEAD's index labels: the index lists the prototype `int computeBudget( int frames );` (engine.h:2) as a +# definition site, so it is kind="def" on the three branches as it already was on HEAD (the lexical test called the +# same bytes a ref on the branches only, because the line ends in `;`). With 8 def rows against 3 refs the defs page +# is no longer the SHORTER one (2470 B vs 2439 B), so the default serves the whole list; this arm's subject is the +# defs page's fold arithmetic, so it asks for that page by name (--whereis-listing=defs). The default's choice keeps +# its own arms (L15 and pagingsweepcheck). +"$BIN" "$R" --whereis=computeBudget --whereis-listing=defs >"$TMP/w2d" 2>/dev/null tr '<' '\n' <"$TMP/w2d" | sed -n 's/^hit ref="\([^"]*\)".* p="\([^"]*\)" l="\([0-9]*\)" kind="\([a-z]*\)".*/\1 \4 \2:\3/p' >"$TMP/w2dall" +tr '<' '\n' <"$TMP/w2d" | grep '^hit ' | grep 'kind="def"' | awk '{ n = 1; if( match( $0, / refs="[0-9]+"/ ) ) { n = substr( $0, RSTART + 7, RLENGTH - 8 ) } print n }' >"$TMP/w2dweights" W2REFS="$( grep -c ' ref ' "$TMP/w2all" )" -{ [ "$( grep ' def ' "$TMP/w2dall" )" = "$( grep ' def ' "$TMP/w2all" )" ] && ! grep -q ' ref ' "$TMP/w2dall" \ +W2DEFS_ALL="$( grep -c ' def ' "$TMP/w2all" )" +W2DEFS_SUM="$( awk '{s+=$1} END{print s+0}' "$TMP/w2dweights" )" +{ [ "$( grep ' def ' "$TMP/w2dall" )" = "$( grep ' def ' "$TMP/w2all" | awk '{ k = ( $1 == "HEAD" ) ? "HEAD" NR : $2 " " $3 } !seen[k]++' )" ] && ! grep -q ' ref ' "$TMP/w2dall" \ + && [ "$W2DEFS_SUM" = "$W2DEFS_ALL" ] && [ "$W2DEFS_ALL" -ge 2 ] \ && [ "$( sed -n 1p "$TMP/w2dall" )" = "HEAD def engine.cpp:3" ] && [ "$W2REFS" -ge 1 ] \ && grep -q "" \ - || { no "whereis (default listing twin): the default lost a def row or miscounted the refs"; cat "$TMP/w2dall"; grep -o ']*>' "$TMP/w2d"; } + && ok "whereis (default listing twin): the def rows of the whole list, source first, shared branch lines folded (refs= sums to $W2DEFS_ALL def rows); the $W2REFS kind=\"ref\" row(s), the doc row among them, counted by " \ + || { no "whereis (default listing twin): the default lost a def row, folded wrongly (refs= sum $W2DEFS_SUM vs $W2DEFS_ALL def rows) or miscounted the refs"; cat "$TMP/w2dall"; grep -o ']*>' "$TMP/w2d"; } +# The fold's witness on this fixture: the three branches' identical engine.cpp:3 line is one row, refs="3", on the +# first branch by name, and the whole list keeps all three (a count, never a drop). Fix round 2: engine.h:2 (HEAD's own +# blob on every branch, the index's prototype definition) folds the same way — two refs="3" rows. +W2FOLDED="$( tr '<' '\n' <"$TMP/w2d" | grep -c '^hit ref="feat-merged" .*kind="def" refs="3" ' )" +[ "$W2FOLDED" -eq 2 ] && [ "$( grep -c ' def engine.cpp:3' "$TMP/w2all" )" -eq 4 ] && [ "$( grep -c ' def engine.h:2' "$TMP/w2all" )" -eq 4 ] && ! grep -q ' refs="' "$TMP/w2" \ + && ok 'whereis (fold): the branches'"'"' shared engine.cpp:3 and engine.h:2 definitions are one row each, refs="3" on feat-merged; listing=all keeps all eight rows and carries no refs=' \ + || { no "whereis (fold): expected two feat-merged def rows with refs=\"3\" (got $W2FOLDED) and 4+4 engine.cpp:3/engine.h:2 def rows in the whole list"; cat "$TMP/w2dall"; } # The finding's own repro, on this repo, when it is a git tree deep enough to answer. Skipped rather # than failed on a shallow/absent checkout: this arm is a bonus over the fixture arms above, which @@ -256,6 +282,130 @@ else ok "whereis: repo root is not a git tree — real-repo arm skipped" fi +# ── 5b) --whereis on the OTHER refs: kind="def" only where a parser finds the definition, and the definitions page +# lists a line that several refs hold ONCE (refs="N") ────────────────────────────────────────────────────── +# The 26b sweep (ideas-fable-26b #9): with the definitions page listing every kind="def" row of every ref, a defs="1" +# symbol listed 2113 "definitions" — 59 of the first 60 were call sites that merely ended in `}` or `)` +# (`if( !task.empty() ) { out += escapeXml( task, esc ); }`), a string literal's `impl `, a comment's `def `, and every +# ref's own copy of the one real line. Its own fixture: three branches hold the same shapes.cpp / calls.js (shape-b +# byte-identical to shape-a, shape-c with one extra comment line so its lines are numbered differently), beside the +# call shapes of C++, Python, JS, a string literal, a shell comment and a using-declaration. HEAD defines the name in +# budget.cpp, so HEAD's label comes from the index. +# FIX ROUND 2: every branch row is now a PARSER's label too (labelBranchRowsByParse): a branch holding HEAD's own blob of +# a path takes HEAD's index labels, any other blob holding a parse-worthy line is parsed by the index's own extraction. +SH="$TMP/shapes"; mkdir -p "$SH" +gs(){ git -C "$SH" "$@" >/dev/null 2>&1; } +gs init -q -b main +gs config commit.gpgsign false +printf '#pragma once\nint computeBudget( int frames );\n' > "$SH/budget.h" +printf 'int computeBudget( int frames )\n{\n return frames * 16;\n}\n' > "$SH/budget.cpp" +gs add budget.h budget.cpp +gs commit -qm base +write_shapes(){ # $1 = extra leading comment lines for shapes.cpp (shifts every line number) + { printf '%s' "$1"; cat <<'EOF' +#include "budget.h" +static int computeBudget( int frames ) { return frames * 32; } +auto computeBudget( long frames ) -> long { return frames; } +int notADef( int total ) +{ + if( total == 0 ) { total += computeBudget( 1 ); } + if( total == 1 ) { computeBudget( 2 ); } + return computeBudget( 3 ); +} +EOF + } > "$SH/shapes.cpp" + printf 'if computeBudget( 7 ):\n pass\nx = computeBudget( 8 )\n' > "$SH/calls.py" + printf 'await computeBudget( 9 )\ncomputeBudget( 10 ).then( x => x )\nclass Budget {\n computeBudget( frames ) {\n return frames\n }\n}\n' > "$SH/calls.js" + printf 'w.write( " "$SH/w.cpp" + printf '# 0 def rows in computeBudget( 12 ) hits\n' > "$SH/note.sh" +} +gs checkout -qb shape-a +write_shapes "" +gs add shapes.cpp calls.py calls.js w.cpp note.sh +gs commit -qm "the shapes" +gs checkout -qb shape-b +printf 'unrelated\n' > "$SH/other.txt" +gs add other.txt +gs commit -qm "shape-b: the same shapes, plus an unrelated file" +gs checkout -q main +gs checkout -qb shape-c +write_shapes "// shape-c: one more line above, so every line number differs from shape-a's +" +gs add shapes.cpp calls.py calls.js w.cpp note.sh +gs commit -qm "shape-c: the same lines, numbered differently" +gs checkout -q main + +"$BIN" "$SH" --whereis=computeBudget >"$TMP/sd" 2>/dev/null; rcSd=$? +"$BIN" "$SH" --whereis=computeBudget --whereis-listing=all >"$TMP/sa" 2>/dev/null; rcSa=$? +rows_of(){ tr '<' '\n' <"$1" | sed -n 's/^hit ref="\([^"]*\)".* p="\([^"]*\)" l="\([0-9]*\)" kind="\([a-z]*\)"\( test_local="1"\)\{0,1\}\( refs="[0-9]*"\)\{0,1\} t=.*/\1 \4 \2:\3\6/p'; } +rows_of "$TMP/sd" >"$TMP/sdrows"; rows_of "$TMP/sa" >"$TMP/sarows" +if [ $rcSd -ne 0 ] || [ $rcSa -ne 0 ] || ! grep -q ' root (rc $rcSd / $rcSa)" +else + # (i) RED on 93aeab1d: the nine call shapes read kind="def" on every branch. Every branch def row is one of the + # three definition shapes — the one-line body, the trailing return and the JS method — and no other line. + # Fix round 2 (rule 4, by design): plus budget.h's prototype `int computeBudget( int frames );` — HEAD's own + # budget.h blob on every branch takes HEAD's index label, and the index lists that prototype as a definition + # site (HEAD said def and the branches said ref for the same bytes; now both say def). + DEF_T="$( tr '<' '\n' <"$TMP/sa" | grep '^hit ref="shape-' | grep 'kind="def"' | sed -n 's/.* t="\(.*\)"\/>$/\1/p' | sort -u )" + EXPECT_T="$( printf '%s\n' 'computeBudget( frames ) {' 'auto computeBudget( long frames ) -> long { return frames; }' 'static int computeBudget( int frames ) { return frames * 32; }' 'int computeBudget( int frames )' 'int computeBudget( int frames );' | sort -u )" + [ "$DEF_T" = "$EXPECT_T" ] \ + && ok 'whereis (shapes): on the branches only the definition shapes read kind="def" (base'"'"'s budget.cpp, the one-line body, the trailing return, the JS method)' \ + || { no 'whereis (shapes): a call shape reads kind="def" on a branch, or a definition shape lost its label'; printf '%s\n' "$DEF_T"; } + # (ii) each call shape is a kind="ref" row (present, labelled ref): the fix relabels, it never drops. + missing="" + for shape in 'total += computeBudget( 1 ); }' '{ computeBudget( 2 ); }' 'return computeBudget( 3 );' 'if computeBudget( 7 ):' 'x = computeBudget( 8 )' \ + 'await computeBudget( 9 )' 'computeBudget( 10 ).then' '<impl n=' '0 def rows' 'using ns::computeBudget;'; do + tr '<' '\n' <"$TMP/sa" | grep '^hit ref="shape-a"' | grep 'kind="ref"' | grep -qF "$shape" || missing="$missing [$shape]" + done + [ -z "$missing" ] \ + && ok 'whereis (shapes): the C++, Python, JS, string-literal, comment and using-declaration call shapes are kind="ref" rows on shape-a' \ + || { no "whereis (shapes): call shapes not found as kind=\"ref\" rows:$missing"; grep '^shape-a' "$TMP/sarows"; } + # (iii) the fold: the definitions page lists each shared line once, on shape-a (first by name), refs="3" — shape-c + # holds the same text at other line numbers and still folds (the key is path + text, not the line). + # The base's budget.cpp:1 is on every branch too, so four shared lines: HEAD's two index rows + four shape-a rows. + # Fix round 2: five — budget.h:2 (HEAD's index label on HEAD's own blob, see (i)) folds as well: folded="10". + [ "$( grep -c '^shape-a def .* refs="3"' "$TMP/sdrows" )" -eq 5 ] && ! grep -q '^shape-[bc] ' "$TMP/sdrows" \ + && [ "$( sed -n 1p "$TMP/sdrows" )" = "HEAD def budget.cpp:1" ] && [ "$( grep -c ' def ' "$TMP/sdrows" )" -eq 7 ] \ + && grep -q '^shape-c def shapes.cpp:3$' "$TMP/sarows" && grep -q '^shape-a def shapes.cpp:2 refs="3"' "$TMP/sdrows" \ + && grep -o ']*>' "$TMP/sd" | grep -q ' listing="defs" folded="10" ' \ + && ok 'whereis (fold): the five shared definition lines are one row each on shape-a with refs="3" (folded="10" on the root); shape-c holds one at another line number and folds by text' \ + || { no 'whereis (fold): expected HEAD + five shape-a rows refs="3" and folded="10" on the definitions page'; cat "$TMP/sdrows"; grep -o ']*>' "$TMP/sd"; } + # (iv) the whole list is the untouched twin: every branch's own row, no refs=/folded= anywhere, rows == hits=. + SA_HITS="$( grep -o ']*' "$TMP/sa" | grep -o ' hits="[0-9]*"' | grep -o '[0-9]*' )" + SA_ROWS="$( grep -c . "$TMP/sarows" )" + # Fix round 2: 17 def rows = HEAD's 2 + 3 branches x 5 (budget.h:2 included, see (i)). + [ -n "$SA_HITS" ] && [ "$SA_ROWS" = "$SA_HITS" ] && ! grep -q ' refs="' "$TMP/sa" && ! grep -q ' folded="' "$TMP/sa" && [ "$( grep -c ' def ' "$TMP/sarows" )" -eq 17 ] \ + && ok "whereis (fold twin): --whereis-listing=all prints every row ($SA_ROWS == hits=$SA_HITS), 17 def rows, no refs= or folded=" \ + || { no "whereis (fold twin): rows $SA_ROWS vs hits=$SA_HITS, def rows $( grep -c ' def ' "$TMP/sarows" ) (want 17), or refs=/folded= leaked into the whole list"; grep -o ']*>' "$TMP/sa"; } + # (v) the counts agree across the two pages: hits= identical, and on the default equals the whole list's ref rows. + SD_HITS="$( grep -o ']*' "$TMP/sd" | grep -o ' hits="[0-9]*"' | grep -o '[0-9]*' )" + [ "$SD_HITS" = "$SA_HITS" ] && grep -q " equals the whole list's kind=\"ref\" rows" \ + || { no "whereis (fold): hits= or disagree between the pages"; grep -o ']*>' "$TMP/sd"; grep -o ']*>' "$TMP/sd"; } + # (vi) the legend defines refs= where it rides (full prose and compact dictionary), and the full page stays well-formed XML + # with the SHARED DEFINITIONS clause spliced into its comment (checklist 25: no `--` inside it). + "$BIN" "$SH" --whereis=computeBudget --legend=full >"$TMP/sdf" 2>/dev/null + "$BIN" "$SH" --whereis=computeBudget --legend=compact >"$TMP/sdc" 2>/dev/null + if command -v xmllint >/dev/null 2>&1; then + xmllint --noout "$TMP/sdf" 2>/dev/null && grep -q 'SHARED DEFINITIONS: refs="N"' "$TMP/sdf" \ + && ok 'whereis (fold legend): the full legend defines refs= (SHARED DEFINITIONS) and the page is well-formed XML' \ + || no 'whereis (fold legend): the full page is malformed or its legend does not define refs=' + else + printf ' SKIP whereis (fold legend): xmllint absent — the full page'"'"'s well-formedness is not checked\n' # rv N2: a missing tool is a named SKIP + grep -q 'SHARED DEFINITIONS: refs="N"' "$TMP/sdf" && ok 'whereis (fold legend): the full legend defines refs= (SHARED DEFINITIONS)' \ + || no 'whereis (fold legend): the full legend does not define refs=' + fi + grep -q 'hit refs=N' "$TMP/sdc" \ + && ok 'whereis (fold legend): the compact dictionary defines hit refs=N on the page that carries it' \ + || no 'whereis (fold legend): the compact dictionary lacks the hit refs=N reading' + "$BIN" "$SH" --whereis=computeBudget --whereis-listing=all --legend=compact >"$TMP/sac" 2>/dev/null; rcSac=$? + # rv N1: an ABSENCE is read only off a run that succeeded and printed its root. + [ $rcSac -eq 0 ] && grep -q '/dev/null 2>&1; [ $? -eq 1 ]; then ok "bare --whereis refuses loudly (exit 1)"; else no "bare --whereis did not exit 1"; fi mkdir -p "$TMP/plain"; printf 'int main(){return 0;}\n' > "$TMP/plain/m.cpp" @@ -367,16 +517,25 @@ else no "§B8.2 whereis: --limit=1 produced no to check (fixture has too few hits)" fi # TWIN on the default listing: the same arithmetic, the counted refs included — shown + more + refs count == hits. -W1D="$( "$BIN" "$R" --whereis=computeBudget --limit=1 2>/dev/null )" +# Fix round 2 (by design, see the default listing twin in 5b): on this fixture the defs page is no longer the shorter +# one, so the default serves the whole list; the arm names the defs page (--whereis-listing=defs) whose arithmetic it pins. +W1D="$( "$BIN" "$R" --whereis=computeBudget --whereis-listing=defs --limit=1 2>/dev/null )" WD_SHOWN="$( printf '%s' "$W1D" | grep -oE ']*' | grep -oE 'shown="[0-9]+"' | grep -oE '[0-9]+' )" WD_HITS="$( printf '%s' "$W1D" | grep -oE ']*' | grep -oE ' hits="[0-9]+"' | grep -oE '[0-9]+' )" WD_MORE="$( printf '%s' "$W1D" | grep -oE '//' | grep -oE '/dev/null )" +# FOLD UPDATE (lane whereis-defs-fix): the branches' shared definition line is one row (refs="3"), and the root's folded= +# counts the two rows it stands for — shown + more + folded + refs count == hits. The whole-list arm above is the +# untouched twin (no folded= there: rows == hits). +WD_FOLDED="$( printf '%s' "$W1D" | grep -oE ']*' | grep -oE ' folded="[0-9]+"' | grep -oE '[0-9]+' )" +{ [ -n "$WD_MORE" ] && [ -n "$WD_REFS" ] && [ "$(( WD_SHOWN + WD_MORE + ${WD_FOLDED:-0} + WD_REFS ))" = "$WD_HITS" ] && [ "$WD_ROWS" = "$WD_SHOWN" ]; } \ + && ok "§B8.2 whereis (default listing twin): shown($WD_SHOWN) + more($WD_MORE) + folded(${WD_FOLDED:-0}) + refs count($WD_REFS) == hits($WD_HITS)" \ + || no "§B8.2 whereis (default listing twin): shown=$WD_SHOWN more=$WD_MORE folded=${WD_FOLDED:-0} refs=$WD_REFS hits=$WD_HITS rows=$WD_ROWS" +[ "$WD_FOLDED" = "4" ] \ + && ok '§B8.2 whereis (default listing twin): folded="4" on the root — the two branch copies each of engine.cpp:3 and engine.h:2 behind the two refs="3" rows' \ + || no "§B8.2 whereis (default listing twin): folded=\"${WD_FOLDED:-absent}\" on the root, want 4" +WDEND="$( "$BIN" "$R" --whereis=computeBudget --whereis-listing=defs --limit=1 --offset="$WD_HITS" 2>/dev/null )" if ! printf '%s' "$WDEND" | grep -q ' root" elif printf '%s' "$WDEND" | grep -q '/dev/null; } -# (L15) the cap bites: HEAD defines zqCap once and calls it 70 times; 64 branches sit at the previous commit (same files) -# and HEAD adds one unrelated commit, so every branch row carries tip=/date=. The all page lists HEAD's definition and 59 -# references (60 rows), the defs page 60 definitions, and the defs page is the LONGER one in bytes — the default is defs. +# (L15) the cap bites: HEAD defines zqCap once and calls it 70 times; 64 branches each hold their OWN definition line +# (`return x + N`: 64 distinct lines, so none folds into another — foldSharedBranchDefs lists a line several refs share +# once, which is the lane whereis-defs-fix twin below), and HEAD adds one unrelated commit, so every branch row carries +# tip=/date=. The all page lists HEAD's definition and 59 references (60 rows), the defs page 60 definitions, and the +# defs page is the LONGER one in bytes — the default is defs. D15="$TMP/cap15"; mkdir -p "$D15"; git -C "$D15" init -q -b main >/dev/null 2>&1; git -C "$D15" config commit.gpgsign false printf 'int zqCap( int x ) { return x; }\n' >"$D15/def.c" { printf 'int useAll( void )\n{\n int s = 0;\n'; i=0; while [ $i -lt 70 ]; do printf ' s += zqCap( %d );\n' $i; i=$((i+1)); done; printf ' return s;\n}\n'; } >"$D15/use.c" printf '// head\n' >"$D15/other.c" git -C "$D15" add -A >/dev/null 2>&1; git -C "$D15" commit -qm cap1 >/dev/null 2>&1 -i=0; while [ $i -lt 64 ]; do git -C "$D15" branch "b$i" >/dev/null 2>&1; i=$((i+1)); done +i=0; while [ $i -lt 64 ]; do + git -C "$D15" checkout -qb "b$i" >/dev/null 2>&1 + printf 'int zqCap( int x ) { return x + %d; }\n' $i >"$D15/def.c" + git -C "$D15" commit -qam "b$i" >/dev/null 2>&1 + git -C "$D15" checkout -q main >/dev/null 2>&1 + i=$((i+1)) +done +printf 'int zqCap( int x ) { return x; }\n' >"$D15/def.c" printf '// head, later\n' >"$D15/other.c"; git -C "$D15" add -A >/dev/null 2>&1 GIT_COMMITTER_DATE="2026-02-02T00:00:00Z" GIT_AUTHOR_DATE="2026-02-02T00:00:00Z" git -C "$D15" commit -qm cap2 >/dev/null 2>&1 C_D="$( dw "$D15" zqCap )"; C_A="$( dw "$D15" zqCap --whereis-listing=all )"; C_F="$( dw "$D15" zqCap --whereis-listing=defs )" @@ -711,6 +879,24 @@ if [ "$CD_F" -gt "$CD_A" ] 2>/dev/null && [ "$( nbytes "$C_F" )" -gt "$( nbytes else no "LEAN (L15): premises not met: defs shown $CD_F vs all $CD_A, bytes $( nbytes "$C_F" ) vs $( nbytes "$C_A" ), all capped?" fi +# (L15-FOLD, lane whereis-defs-fix) the same shape with the 64 branches at ONE commit: their 64 copies of the definition +# line are one row refs="64" (folded="63" on the root), the defs page is now the SHORTER page (2 rows) and is still the +# default — it lists more definitions (65 represented) than the capped all page (HEAD's one). RED on 93aeab1d: 60 rows, +# every one the same line, 4 more behind . +D15F="$TMP/cap15fold"; mkdir -p "$D15F"; git -C "$D15F" init -q -b main >/dev/null 2>&1; git -C "$D15F" config commit.gpgsign false +cp "$D15/def.c" "$D15/use.c" "$D15F/"; printf '// head\n' >"$D15F/other.c" +git -C "$D15F" add -A >/dev/null 2>&1; git -C "$D15F" commit -qm cap1 >/dev/null 2>&1 +i=0; while [ $i -lt 64 ]; do git -C "$D15F" branch "b$i" >/dev/null 2>&1; i=$((i+1)); done +printf '// head, later\n' >"$D15F/other.c"; git -C "$D15F" add -A >/dev/null 2>&1 +GIT_COMMITTER_DATE="2026-02-02T00:00:00Z" GIT_AUTHOR_DATE="2026-02-02T00:00:00Z" git -C "$D15F" commit -qm cap2 >/dev/null 2>&1 +C_FD="$( dw "$D15F" zqCap )" +if printf '%s' "$C_FD" | lroot | grep -q ' listing="defs" folded="63" ' && [ "$( printf '%s' "$C_FD" | defs_in )" = "2" ] \ + && [ "$( printf '%s' "$C_FD" | tr '<' '\n' | grep -c '^hit ref="b0" .*kind="def" refs="64" ' )" = "1" ] \ + && ! printf '%s' "$C_FD" | grep -q '' +else + no 'LEAN (L15-FOLD): the shared definition line is not folded into one refs="64" row on a 2-row defs page'; printf '%s\n' "$C_FD" | lroot; printf '%s' "$C_FD" | tr '<' '\n' | grep -c '^hit ' +fi # (L15-MCP) the same symbol over MCP (no `listing`): byte-identical to the CLI defs page. if command -v python3 >/dev/null 2>&1; then { [ -n "$C_F" ] && [ "$( mcpq "$D15" zqCap '' )" = "$C_F" ] && [ "$( mcpq "$D15" zqCap ',"listing":"all"' )" = "$C_A" ]; } \ @@ -745,6 +931,533 @@ else printf ' SKIP LEAN (L16-MCP) MCP twin (no python3)\n' fi +# ── 5c) PARSED BRANCH LABELS (lane whereis-defs-fix, fix round 2; crossref.h labelBranchRowsByParse) ────────────── +# A row of another ref is kind="def" only where a PARSER finds the definition: a branch holding HEAD's own blob of a path +# takes HEAD's index labels; any other blob holding a parse-worthy line is parsed by the index's own extraction +# (ingest.h definitionLinesInBlobs). The lexical shape test only CHOOSES blobs. One well-formed file per language family +# under head/ on main; byte-identical copies under sib/ on branch `sib` (another path: those blobs are PARSED, not +# mirrored); and under c1/ the definition shapes the round-1 lexical test lost (rv-whereis-defs-fix C1). Every line naming +# probeName carries @D (a definition) or @R (a reference). Two claims per marked line: the branch row says what the +# marker says, and it says what HEAD's index row says for the same bytes. RED on 93aeab1d (18 branch rows wrong: the C2 +# call shapes read def) and on f1f11a7b (22: the C1 definition shapes read ref). +PB="$TMP/parsedlabels"; mkdir -p "$PB" +gp(){ git -C "$PB" "$@" >/dev/null 2>&1; } +mkp(){ mkdir -p "$( dirname "$PB/$1" )"; cat >"$PB/$1"; } +gp init -q -b main; gp config commit.gpgsign false +mkp head/a.c <<'EOF' +#define probeName( a ) ( (a) + 1 ) /* @D */ +int caller( int a ) +{ + if( a ) { a = probeName( a ); } /* @R */ + return probeName( a ); /* @R */ +} +EOF +mkp head/b.cpp <<'EOF' +#include +struct T { int probeName( int a ) const; }; // @D +int T::probeName( int a ) const { return a; } // @D +void run( T& t, int a ) +{ + t.probeName( a ); // @R + if( a ) { a = t.probeName( a ); } // @R + auto f = [ & ]( int x ) { return t.probeName( x ); }; // @R + // probeName( a ) { // @R +} +EOF +mkp head/c.m <<'EOF' +@interface Box +- (int)probeName:(int)a; // @R (an @interface declaration its same-file @implementation shadows: one symbol, the definition) +@end +@implementation Box +- (int)probeName:(int)a { return a; } // @D +- (void)run { [self probeName:1]; } // @R +@end +EOF +mkp head/d.py <<'EOF' +def probeName(a): # @D + return a + + +class K: + async def other(self): + return probeName(1) # @R + + +@probeName # @R +def deco(): + x = probeName(2) # @R + print("probeName(x)") # @R +EOF +mkp head/e.ts <<'EOF' +export function probeName(a: number): number { // @D + return a; +} +export class Svc { + static probeName(a: number) { return a; } // @D + run(a: number) { + probeName(a).toString(); // @R + setTimeout(() => probeName(a), 0); // @R + } +} +EOF +mkp head/f.js <<'EOF' +function probeName(a) { // @D + return a; +} +describe('x', () => { + probeName('x', () => { // @R + return 1; + }); +}); +EOF +mkp head/G.java <<'EOF' +public class G { + public static int probeName(int a) { // @D + return a; + } + int run(int a) { + return probeName(a); // @R + } +} +EOF +mkp head/h.rb <<'EOF' +class H + def probeName(a) # @D + a + end + def run(a) + puts probeName(a) # @R + probeName(a) do |x| # @R + x + end + end +end +EOF +mkp head/i.go <<'EOF' +package p + +func probeName(a int) int { // @D + return a +} + +func run(a int) { + go probeName(a) // @R + defer probeName(a) // @R + if v := probeName(a); v > 0 { // @R + } +} +EOF +mkp head/j.rs <<'EOF' +pub fn probe_other() {} +pub fn probeName(a: i32) -> i32 { // @D + a +} +fn run(a: i32) -> i32 { + let v = probeName(a); // @R + probeName(v) // @R +} +EOF +mkp head/k.swift <<'EOF' +func probeName(_ a: Int) throws -> Int { // @D + return a +} +func run() throws { + let v = try probeName(1) // @R + _ = v +} +EOF +mkp head/l.cs <<'EOF' +public class L +{ + public static int probeName(int a) { return a; } // @D + public int Run(int a) + { + return probeName(a); // @R + } +} +EOF +mkp head/m.sh <<'EOF' +probeName() { # @D + echo "$1" +} +run() { + probeName x # @R +} +EOF +mkp head/n.cpp <<'EOF' +struct N { int probeName; }; // @R (a field: not a symbol in the index, so never a def site on HEAD either) +EOF +mkp head/o.md <<'EOF' +# probeName + +Call `probeName()` from the loop. +EOF +# The C2 call shapes and the review's checklist-29 near-misses (rv-whereis-defs-fix2 X3), each beside a true definition +# of the same name in a well-formed file of its language, so each file reads both ways: the definition def, the call ref. +# The call lines carry NO marker (an unmarked line wants ref): a trailing comment would change the shape a lexical reading +# sees, and the premise below is the bare shape. +# RED on 93aeab1d (its lexical reading calls several of these calls definitions). +mkp head/q.py <<'EOF' +def probeName(n): # @D + def wrap(f): + return f + return wrap + + +@probeName(1) +def other(): + return 2 +EOF +mkp head/P.java <<'EOF' +public class P { + public static int probeName(int a) { // @D + return a; + } + @probeName(value = 1) + int run(int a) { + return a; + } +} +EOF +mkp head/t.c <<'EOF' +#define probeName( x ) ( (x) + 1 ) /* @D */ +#define WRAP( x ) probeName( x ) +int useA( int y ) +{ + if( probeName( y ) > 2 ) { + return WRAP( y ); + } + return 0; +} +EOF +mkp head/u.cpp <<'EOF' +template int probeName( int a, F f ) { return f( a ); } // @D +/** + * @see probeName( a ) + */ +int run( int y ) +{ + return probeName( y, [&]( int x ) { + return x; + } ); +} +EOF +mkp head/v.cpp <<'EOF' +struct probeName { int v; }; // @D +class Foo : public probeName { +}; +EOF +mkp head/w.js <<'EOF' +export function probeName(a) { // @D + return a; +} +function suite() { + probeName(function () { + return 1; + }); + probeName('y', () => { + return 2; + }); +} +export default probeName(3); +EOF +mkp head/x.ts <<'EOF' +export function probeName(n: number) { // @D + return (t: any, k: string) => {}; +} +export class Box { + @probeName(1) handle() { + return 3; + } +} +EOF +mkp head/y.rb <<'EOF' +def probeName(a) # @D + a +end +def other(probeName) + 1 +end +EOF +gp add -A; gp commit -qm base +gp checkout -qb sib +mkdir -p "$PB/sib"; cp -R "$PB/head/." "$PB/sib/" +mkp c1/m1.h <<'EOF' +#define probeName( a ) ( (a) + 1 ) // @D +EOF +mkp c1/m2.h <<'EOF' +#define probeName( a ) do { f( a ); } while( 0 ) // @D +EOF +mkp c1/d1.cpp <<'EOF' +decltype(auto) probeName( int a ) { return a; } // @D +EOF +mkp c1/d2.cpp <<'EOF' +__attribute__((noinline)) static int probeName( int a ) { return a; } // @D +EOF +mkp c1/d3.cpp <<'EOF' +#include +std::function probeName( int a ) { return [](){}; } // @D +EOF +mkp c1/d4.cpp <<'EOF' +template T probeName( T a ) { return a; } // @D +EOF +mkp c1/T.java <<'EOF' +public class T { + @Test(expected = Foo.class) public void probeName() { // @D + } +} +EOF +mkp c1/c.ts <<'EOF' +export class C { + @HostListener('click') probeName() { // @D + } +} +EOF +mkp c1/a.cs <<'EOF' +public class A +{ + [HttpGet("x")] public IActionResult probeName() { return null; } // @D +} +EOF +mkp c1/s.rs <<'EOF' +#[derive(Debug)] pub struct probeName { // @D + a: i32, +} +EOF +mkp c1/e.js <<'EOF' +module.exports = function probeName(req, res) { // @D + return req; +}; +EOF +mkp c1/h.js <<'EOF' +const handler = function probeName() { // @D + return 1; +}; +EOF +# the reviewer's branch-only repro (rv C1 repro 1): names HEAD never holds, defined only on this branch +printf '#define clampBudget( x ) ( (x) > 9 ? 9 : (x) )\n' >"$PB/c1/b.h" +printf 'int use( int a ) { return clampBudget( a ); }\n' >"$PB/c1/u.c" +printf 'public class T2 {\n @Test(timeout = 10) public void checkBudget() {\n }\n}\n' >"$PB/c1/T2.java" +gp add -A; gp commit -qm sib; gp checkout -q main +"$BIN" "$PB" --whereis=probeName --whereis-listing=all --limit=1000 --no-cache >"$TMP/pb.all" 2>/dev/null; rcPb=$? +if [ $rcPb -ne 0 ] || ! grep -q ' root (rc $rcPb)" +else + sed 's///' "$TMP/pb.all" | tr '<' '\n' | sed -n 's/^hit ref="\([^"]*\)".* p="\([^"]*\)" l="\([0-9]*\)" kind="\([a-z]*\)".*/\1 \2:\3 \4/p' >"$TMP/pb.kinds" + PB_WRONG=""; PB_SPLIT=""; PB_N=0 + for f in $( git -C "$PB" ls-tree -r --name-only sib -- head c1 | grep -v '^c1/\(b\.h\|u\.c\|T2\.java\)$' ); do + case "$f" in head/*) bp="sib/${f#head/}"; hp="$f";; *) bp="$f"; hp="";; esac + ln=0 + while IFS= read -r line || [ -n "$line" ]; do + ln=$((ln+1)) + case "$line" in *probeName*) ;; *) continue;; esac + want=ref; case "$line" in *@D*) want=def;; esac + case "$f:$line" in *.md:"# probeName") want=def;; esac # a markdown heading is a section symbol (no room for a marker) + bk="$( awk -v k="$bp:$ln" '$1=="sib" && $2==k {print $3}' "$TMP/pb.kinds" | head -1 )" + PB_N=$((PB_N+1)) + [ "$bk" = "$want" ] || PB_WRONG="$PB_WRONG [$bp:$ln $bk want $want]" + if [ -n "$hp" ]; then + hk="$( awk -v k="$hp:$ln" '$1=="HEAD" && $2==k {print $3}' "$TMP/pb.kinds" | head -1 )" + [ "$hk" = "$bk" ] || PB_SPLIT="$PB_SPLIT [$bp:$ln branch $bk HEAD $hk]" + fi + done < <( git -C "$PB" show "sib:$f" ) + done + { [ "$PB_N" -ge 50 ] && [ -z "$PB_WRONG" ]; } \ + && ok "whereis (parsed labels): $PB_N marked lines over C, C++, ObjC, Python, TS, JS, Java, Ruby, Go, Rust, Swift, C#, Bash and Markdown, plus the 12 C1 definition shapes — every branch row says what its marker says" \ + || no "whereis (parsed labels): branch rows mislabelled (of $PB_N):$PB_WRONG" + [ -z "$PB_SPLIT" ] \ + && ok 'whereis (parsed labels): every sib/ branch row says what HEAD'"'"'s index row says for the same bytes (the same way HEAD is labelled)' \ + || no "whereis (parsed labels): the branch parse and HEAD's index disagree on the same bytes:$PB_SPLIT" + grep -o ']*>' "$TMP/pb.all" | grep -q ' ref_labels="parsed" ' && ! grep -q ' (every parse-worthy blob was read)' \ + || no 'whereis (parsed labels): ref_labels= missing, or an element on an answer the guard never touched' +fi +# the branch-only definitions (rv C1 repro 1): the definition is the answer, and the call beside it is not +CB="$( "$BIN" "$PB" --whereis=clampBudget --whereis-listing=all --no-cache 2>/dev/null )" +KB="$( "$BIN" "$PB" --whereis=checkBudget --whereis-listing=all --no-cache 2>/dev/null )" +{ printf '%s' "$CB" | grep -q ']* on-head="0"' && printf '%s' "$CB" | grep -q ']*p="c1/b.h" l="1" kind="def"' \ + && printf '%s' "$CB" | grep -q ']*p="c1/u.c" l="1" kind="ref"' \ + && printf '%s' "$KB" | grep -q ']*p="c1/T2.java" l="2" kind="def"'; } \ + && ok 'whereis (branch-only): `#define clampBudget( x )` and `@Test(timeout = 10) public void checkBudget()` read kind="def" on the only ref holding them; the call beside the macro reads kind="ref"' \ + || { no 'whereis (branch-only): a branch-only definition lost its label, or its call site gained one'; printf '%s\n%s\n' "$CB" "$KB" | sed 's///'; } +# "One the parser cannot finish" (CHANGELOG, the UNPARSED clause): a branch .astro that opens a frontmatter fence and never +# closes it confirms nothing: its parse-worthy line reads kind="text", never def, and counts +# it with no next= (the guard left nothing, so there is nothing to lift). Its closed twin reads def. RED on 93aeab1d (no +# ; the unclosed line read def). +AS="$TMP/astro"; mkdir -p "$AS"; git -C "$AS" init -q -b main >/dev/null 2>&1; git -C "$AS" config commit.gpgsign false +printf 'x\n' >"$AS/a.txt"; git -C "$AS" add -A >/dev/null 2>&1; git -C "$AS" commit -qm base >/dev/null 2>&1; git -C "$AS" checkout -qb feat >/dev/null 2>&1 +printf -- '---\nfunction astroName(a) { return a; }\n---\n

{astroName(1)}
\n' >"$AS/ok.astro" +printf -- '---\nfunction astroOpen(a) { return a; }\n
{astroOpen(1)}
\n' >"$AS/p.astro" +git -C "$AS" add -A >/dev/null 2>&1; git -C "$AS" commit -qm feat >/dev/null 2>&1; git -C "$AS" checkout -q main >/dev/null 2>&1 +AO="$( "$BIN" "$AS" --whereis=astroOpen --whereis-listing=all --no-cache 2>/dev/null )"; rcAo=$? +AN="$( "$BIN" "$AS" --whereis=astroName --whereis-listing=all --no-cache 2>/dev/null )"; rcAn=$? +if [ $rcAo -ne 0 ] || [ $rcAn -ne 0 ] || ! printf '%s' "$AO" | grep -q ' root (rc $rcAo / $rcAn)" +else + { printf '%s' "$AO" | grep -q '' && printf '%s' "$AO" | grep -q ']*p="p.astro" l="2" kind="text"' \ + && ! printf '%s' "$AO" | grep -q 'kind="def"' \ + && printf '%s' "$AN" | grep -q ']*p="ok.astro" l="2" kind="def"' && ! printf '%s' "$AN" | grep -q ' (no next=); its closed twin reads def' \ + || { no 'whereis (parser could not finish): the unclosed .astro blob was labelled, or not disclosed, or the closed twin lost its def'; printf '%s\n%s\n' "$AO" "$AN" | sed 's///'; } +fi +# ref_labels= rides only an answer with another ref's row: a repo with no other ref answers as before +NR="$TMP/norefs"; mkdir -p "$NR"; git -C "$NR" init -q -b main >/dev/null 2>&1; git -C "$NR" config commit.gpgsign false +printf 'int lonelyName( int a ) { return a; }\nint z( void ) { return lonelyName( 1 ); }\n' >"$NR/x.c" +git -C "$NR" add -A >/dev/null 2>&1; git -C "$NR" commit -qm one >/dev/null 2>&1 +NRO="$( "$BIN" "$NR" --whereis=lonelyName --no-cache 2>/dev/null )"; rcNr=$? +[ $rcNr -eq 0 ] && printf '%s' "$NRO" | grep -q ' rides an answer without a branch row' +# CLI = MCP: the MCP whereis default is byte-identical to the CLI default on the parsed fixture +if command -v python3 >/dev/null 2>&1; then + PBD="$( "$BIN" "$PB" --whereis=probeName --no-cache 2>/dev/null )" + PBM="$( printf '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"whereis","arguments":{"path":"%s","symbol":"probeName"}}}\n' "$PB" \ + | "$BIN" --mcp 2>/dev/null | python3 -c 'import sys,json +for line in sys.stdin: + line=line.strip() + if not line: continue + d=json.loads(line); c=d.get("result",{}).get("content") + if c: print(c[0].get("text",""),end="")' )" + [ -n "$PBD" ] && [ "$PBD" = "$PBM" ] \ + && ok 'whereis (parsed labels): MCP whereis serves the CLI default byte for byte (one labeller on both surfaces)' \ + || no 'whereis (parsed labels): the MCP answer differs from the CLI default on the parsed fixture' +else + printf ' SKIP whereis (parsed labels) MCP twin (no python3)\n' +fi + +# ── 5d) the RUNAWAY GUARD on the parse batch (kWhereisParseMaxBlobs = 256) and what it leaves ───────────────────────── +# 300 distinct blobs on one branch, each a one-line definition of guardName (g.h is HEAD's own blob there: mirrored, never +# in the batch): the batch takes 256 (most rows first, then path: f000..f255), the other 44 confirm NOTHING — their parse-worthy rows read kind="text", never kind="def", and +# says so. --detail=1 lifts the guard: all 300 are definitions and the +# element is gone. The full legend carries the UNPARSED clause on exactly that answer and stays well-formed XML. +# u.c (HEAD's, 20 calls) gives the answer kind="ref" rows, so the default listing (defs) and listing=all are different +# pages: the next= replay arm below tells "the same question" from "the default question". +GG="$TMP/guard"; mkdir -p "$GG"; git -C "$GG" init -q -b main >/dev/null 2>&1; git -C "$GG" config commit.gpgsign false +printf 'int guardName( int a );\n' >"$GG/g.h" +i=0; : >"$GG/u.c"; while [ $i -lt 20 ]; do printf 'int use%d( void ) { return guardName( %d ); }\n' $i $i >>"$GG/u.c"; i=$((i+1)); done +git -C "$GG" add -A >/dev/null 2>&1; git -C "$GG" commit -qm base >/dev/null 2>&1 +git -C "$GG" checkout -qb many >/dev/null 2>&1 +i=0; while [ $i -lt 300 ]; do n=$( printf '%03d' $i ); printf 'int guardName( int a ) { return a + %d; }\n' $i >"$GG/f$n.c"; i=$((i+1)); done +git -C "$GG" add -A >/dev/null 2>&1; git -C "$GG" commit -qm many >/dev/null 2>&1; git -C "$GG" checkout -q main >/dev/null 2>&1 +GA="$( "$BIN" "$GG" --whereis=guardName --whereis-listing=all --limit=1000 --no-cache 2>/dev/null )"; rcGa=$? +GD="$( "$BIN" "$GG" --whereis=guardName --whereis-listing=all --limit=1000 --detail=1 --no-cache 2>/dev/null )"; rcGd=$? +GT="$( printf '%s' "$GA" | sed 's///' | tr '<' '\n' | grep '^hit ref="many" .* p="f' | grep -c 'kind="text"' )" +GF="$( printf '%s' "$GA" | sed 's///' | tr '<' '\n' | grep '^hit ref="many" .* p="f' | grep -c 'kind="def"' )" +GTL="$( printf '%s' "$GA" | sed 's///' | tr '<' '\n' | grep '^hit ref="many" ' | grep 'kind="text"' | sed -n 's/.* p="\([^"]*\)".*/\1/p' | sort | sed -n '1p;$p' | tr '\n' ' ' )" +if [ $rcGa -ne 0 ] || [ $rcGd -ne 0 ] || ! printf '%s' "$GA" | grep -q ' root (rc $rcGa / $rcGd)" +else + { [ "$GT" = "44" ] && [ "$GF" = "256" ] && [ "$GTL" = "f256.c f299.c " ] \ + && printf '%s' "$GA" | grep -q ''; } \ + && ok 'whereis (guard): 256 parsed (f000..f255, most rows first then path), 44 left: their rows read kind="text" (never def) and discloses them' \ + || { no "whereis (guard): want 256 def + 44 text rows (f256..f299) and the element; got def=$GF text=$GT [$GTL]"; printf '%s' "$GA" | grep -o ']*>'; } + { [ "$( printf '%s' "$GD" | sed 's///' | tr '<' '\n' | grep '^hit ref="many" .* p="f' | grep -c 'kind="def"' )" = "300" ] \ + && ! printf '%s' "$GD" | grep -q 'kind="text"' && ! printf '%s' "$GD" | grep -q '' \ + || no 'whereis (guard): --detail=1 did not parse every blob' + # X1 (rule 5, recoverable via next=): the next= REPLAYS the question. Its argv, run as written, answers the same + # listing with the guard lifted — byte for byte the explicit --whereis-listing=all --detail=1 page ($GD), not the + # default (defs) page — and draws no "IGNORED" verb-precedence note. The default page's next= names no listing. + GN="$( printf '%s' "$GA" | sed -n 's/.*]* next="\([^"]*\)".*/\1/p' )" + GR="$( "$BIN" "$GG" $GN --limit=1000 --no-cache 2>"$TMP/guard.replay.err" )"; rcGr=$? + { [ -n "$GN" ] && [ $rcGr -eq 0 ] && printf '%s' "$GR" | grep -q ' of a listing=all answer ($GN) replays it: the same listing=all page with the guard lifted" \ + || { no "whereis (guard): next='$GN' (rc $rcGr) does not replay the listing=all question with the guard lifted"; grep -o ']*>' <<<"$GR"; cat "$TMP/guard.replay.err"; } + GDN="$( "$BIN" "$GG" --whereis=guardName --no-cache 2>/dev/null | grep -o ']*>' )" + [ "$GDN" = '' ] \ + && ok 'whereis (guard): the default page'"'"'s next= names no listing (the default stays the default)' \ + || no "whereis (guard): the default page's is '$GDN'" + if command -v python3 >/dev/null 2>&1; then + # The MCP `kind` ref filter has no CLI whereis spelling: next= drops it (as does) instead of naming + # --stray-content=, a different verb that would take precedence over --whereis. + GM="$( mcpq "$GG" guardName ',"kind":"man","listing":"all"' )" + GMN="$( printf '%s' "$GM" | sed -n 's/.*]* next="\([^"]*\)".*/\1/p' )" + GMR="$( "$BIN" "$GG" $GMN --limit=1000 --no-cache 2>"$TMP/guard.mcpreplay.err" )"; rcGmr=$? + { printf '%s' "$GM" | grep -q ']* filter="man"' && [ "$GMN" = '--whereis=guardName --whereis-listing=all --detail=1' ] \ + && [ $rcGmr -eq 0 ] && printf '%s' "$GMR" | grep -q '/dev/null )" + GC="$( "$BIN" "$GG" --whereis=guardName --legend=compact --no-cache 2>/dev/null )" + { printf '%s' "$GL" | grep -q 'UNPARSED: the unparsed element' && printf '%s' "$GC" | grep -q 'unparsed blobs=N' && printf '%s' "$GC" | grep -q 'unparsed rows=N' \ + && printf '%s' "$GC" | grep -q 'ref_labels=parsed' && ! printf '%s' "$GD" | grep -q 'UNPARSED'; } \ + && ok 'whereis (guard): the full legend explains and the compact dictionary reads blobs=/rows=/ref_labels= on the answer that carries them (and not on the --detail=1 answer)' \ + || no 'whereis (guard): a legend lacks the / ref_labels= readings, or carries them where the element is absent' + if command -v xmllint >/dev/null 2>&1; then + printf '%s' "$GL" | xmllint --noout - 2>/dev/null \ + && ok 'whereis (guard): the full-legend page with the UNPARSED clause spliced in is well-formed XML (checklist 25)' \ + || no 'whereis (guard): the full-legend page with the UNPARSED clause is malformed XML' + else + printf ' SKIP whereis (guard): xmllint absent — the UNPARSED clause'"'"'s well-formedness is not checked\n' + fi +fi + +# ── 5e) every ref's tree from content-addressed tree objects (crossref.h listTreesOfRefs): ls-tree's own spelling ──── +# One branch holds files whose NAMES git must quote under core.quotepath=false (a double quote, a backslash, a tab) and +# ones it must not (a space, UTF-8, a leading dash). The p= of every row equals `git ls-tree -r` for that ref, path for +# path (XML-unescaped). RED on any walk that spells a path its own way. +OD="$TMP/oddnames"; mkdir -p "$OD/d"; git -C "$OD" init -q -b main >/dev/null 2>&1; git -C "$OD" config commit.gpgsign false +printf 'int oddName( int a );\n' >"$OD/d/plain.h"; git -C "$OD" add -A >/dev/null 2>&1; git -C "$OD" commit -qm base >/dev/null 2>&1 +git -C "$OD" checkout -qb odd >/dev/null 2>&1 +for nm in 'q"uote.h' 'back\slash.h' "$( printf 'ta\tb.h' )" 'sp ace.h' 'caf'"$( printf '\303\251' )"'.h' -- '-lead.h'; do + [ "$nm" = "--" ] && continue + printf 'int oddName( int a );\n' >"$OD/d/$nm" +done +git -C "$OD" add -A >/dev/null 2>&1; git -C "$OD" commit -qm odd >/dev/null 2>&1; git -C "$OD" checkout -q main >/dev/null 2>&1 +OO="$( "$BIN" "$OD" --whereis=oddName --whereis-listing=all --no-cache 2>/dev/null )"; rcOo=$? +OGOT="$( printf '%s' "$OO" | sed 's///' | tr '<' '\n' | grep '^hit ref="odd" ' | sed -n 's/.* p="\([^"]*\)" l=.*/\1/p' \ + | sed 's/"/"/g; s/'/'"'"'/g; s/<//g; s/&/\&/g' | sort )" +OWANT="$( git -c core.quotepath=false -C "$OD" ls-tree -r --name-only odd | sort )" +{ [ $rcOo -eq 0 ] && [ "$( printf '%s\n' "$OWANT" | grep -c . )" -ge 6 ] && [ "$OGOT" = "$OWANT" ]; } \ + && ok 'whereis (tree walk): every row'"'"'s p= is git ls-tree'"'"'s own spelling — quoted (", \, tab) and unquoted (space, UTF-8, leading dash) alike' \ + || { no "whereis (tree walk): p= differs from git ls-tree's spelling (rc $rcOo)"; printf 'got:\n%s\nwant:\n%s\n' "$OGOT" "$OWANT"; } + +# ── 5f) train 26c (#339 x the parsed branch labels): a branch's Rails schema column reads as HEAD's index reads it ──── +# HEAD's index makes a db/schema.rb column a definition (#339: captureRubyDefs, the schema recognized by content), and another +# ref's blob is parsed by the SAME extraction in its DefinitionsOnly posture (ingest_sidecap.h: the Ruby def walk runs before +# that posture's early return). So a column a branch adds reads kind="def" once its blob earns a parse (a parse-worthy hit line +# in it), as HEAD's own copy of a column does. RED if the Ruby def walk sat behind the DefinitionsOnly return (measured: the +# column row reads kind="ref"); the premise is HEAD's index labelling its own column a definition. +RS="$TMP/rbschema"; mkdir -p "$RS/db" "$RS/other"; git -C "$RS" init -q -b main >/dev/null 2>&1; git -C "$RS" config commit.gpgsign false +printf 'ActiveRecord::Schema[7.1].define(version: 2024_01_01_000000) do\n create_table "users", force: :cascade do |t|\n t.string "email"\n end\nend\n' >"$RS/db/schema.rb" +printf 'ActiveRecord::Schema[7.1].define(version: 2024_01_01_000000) do\n create_table "accounts", force: :cascade do |t|\n t.string "nickname"\n end\nend\n' >"$RS/other/schema.rb" +git -C "$RS" add -A >/dev/null 2>&1; git -C "$RS" commit -qm base >/dev/null 2>&1; git -C "$RS" checkout -qb feat >/dev/null 2>&1 +printf 'ActiveRecord::Schema[7.1].define(version: 2024_02_01_000000) do\n create_table "users", force: :cascade do |t|\n t.string "email"\n t.string "nickname"\n end\nend\ndef nickname( x )\n x\nend\n' >"$RS/db/schema.rb" +git -C "$RS" commit -qam feat >/dev/null 2>&1; git -C "$RS" checkout -q main >/dev/null 2>&1 +RSO="$( "$BIN" "$RS" --whereis=nickname --whereis-listing=all --no-cache 2>/dev/null )"; rcRs=$? +RSHEAD="$( printf '%s' "$RSO" | grep -oE ']*p="db/schema\.rb" l="4" kind="[a-z]+"' | head -1 )" +if [ $rcRs -ne 0 ] || [ "${RSHEAD##*kind=}" != '"def"' ]; then + no "whereis (#339 x parsed labels): premise — HEAD's index labels its own schema column a def (rc $rcRs, got: ${RSHEAD:-no row})" +elif [ "${RSCOL##*kind=}" = '"def"' ]; then + ok 'whereis (#339 x parsed labels): a column a branch adds to db/schema.rb reads kind="def", as HEAD'"'"'s index reads its own' +else + no "whereis (#339 x parsed labels): the branch's schema column is not a def (got: ${RSCOL:-no row})" +fi + +# ── 5g) train 26c (cli-perf-068 x the tree-object walk): the ls-tree fallback lists each DISTINCT tip once ───────────── +# A root below the top of the work tree cannot use the tree-object walk (listTreesOfRefs), so the trees come from +# `git ls-tree -r`, one per distinct tip (crossref.h listDistinctTips): HEAD + b2 + b3 sit on 2 tips (b3 = b2's tip; b1, at +# HEAD's tip, is not swept), so 2 listings, not one per ref, and the same 8 rows. RED on a per-ref fallback (3 listings). +LT="$TMP/lstips"; mkdir -p "$LT/sub"; git -C "$LT" init -q -b main >/dev/null 2>&1; git -C "$LT" config commit.gpgsign false +printf 'int tipProbe( int x ) { return x; }\nint tipCaller( void ) { return tipProbe( 1 ); }\n' >"$LT/sub/a.c"; printf 'top\n' >"$LT/top.txt" +git -C "$LT" add -A >/dev/null 2>&1; git -C "$LT" commit -qm one >/dev/null 2>&1; git -C "$LT" branch b1 >/dev/null 2>&1 +git -C "$LT" checkout -qb b2 >/dev/null 2>&1; printf 'int tipProbe( int x, int y ) { return x + y; }\n' >"$LT/sub/b.c" +git -C "$LT" add -A >/dev/null 2>&1; git -C "$LT" commit -qm two >/dev/null 2>&1; git -C "$LT" branch b3 >/dev/null 2>&1; git -C "$LT" checkout -q main >/dev/null 2>&1 +: >"$TMP/lstips.trace" +LTO="$( GIT_TRACE="$TMP/lstips.trace" "$BIN" "$LT/sub" --whereis=tipProbe --whereis-listing=all --no-cache 2>/dev/null )"; rcLt=$? +LTN="$( grep -c 'built-in: git ls-tree' "$TMP/lstips.trace" )" +LTH="$( printf '%s' "$LTO" | grep -oE ']* hits="[0-9]+"' | grep -oE 'hits="[0-9]+"' )" +LTR="$( printf '%s' "$LTO" | grep -oE ']* refs_scanned="[0-9]+"' | grep -oE 'refs_scanned="[0-9]+"' )" +{ [ $rcLt -eq 0 ] && [ "$LTR" = 'refs_scanned="2"' ] && [ "$LTH" = 'hits="8"' ] && [ "$LTN" = 2 ]; } \ + && ok "whereis (ls-tree fallback): a root below the work-tree top lists each distinct tip once (2 listings for HEAD + 2 refs on 2 tips), all 8 rows" \ + || no "whereis (ls-tree fallback): want rc 0, refs_scanned=\"2\", hits=\"8\" and 2 ls-tree listings — got rc $rcLt, $LTR, $LTH, $LTN listing(s)" + if command -v xmllint >/dev/null 2>&1; then if printf '%s' "$SW" | xmllint --noout - 2>/dev/null; then ok "capped stray-content still G4 clean"; else no "capped stray-content XML malformed"; fi if printf '%s' "$W1" | xmllint --noout - 2>/dev/null; then ok "paged whereis still G4 clean"; else no "paged whereis XML malformed"; fi diff --git a/test/crossrefdegradecheck.sh b/test/crossrefdegradecheck.sh index f22196105..30cd16845 100755 --- a/test/crossrefdegradecheck.sh +++ b/test/crossrefdegradecheck.sh @@ -22,6 +22,8 @@ # exactly one; and at exactly cap+1 items `size > shown` went false, so the element vanished ENTIRELY # and one row disappeared unmarked. Both the general case and that boundary are pinned here — nothing # in test/ pinned any more= count before, which is why the bug survived. +# 6) (cli-perf-068) A blob stream that never started (an unusable cache dir for its sha list) is refused by +# --whereis on both hosts, not printed as on-head="0" hits="0" with a near-miss retrying the same name. # 5) Two adjacent ways a real hit is reported as no hit at all: a symbol defined on an UNTERMINATED final # line (the scan only evaluated on '\n', so hits="0" — which the help text says means "this repo never # had the name"), and a branch whose name legally contains '|' (the for-each-ref parse split on it and @@ -37,6 +39,7 @@ set -u ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +. "$ROOT/test/lib/clean-env.sh" BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" [ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" TMP="$( mktemp -d )"; trap 'rm -rf "$TMP"' EXIT @@ -60,6 +63,64 @@ count_of(){ tr '<' '\n' | grep -v '^!--' | grep -c "^$1 "; } echo "crossrefdegradecheck: BIN=$BIN" +# ══ K75: --stray-content, --abi, --plan and --eval-stray REFUSE when the blob batch cannot run ═══════════════ +# +# Every cross-branch verdict is built from ONE `git cat-file --batch` whose sha list is written under the cache directory. With +# TMPDIR naming a directory that does not exist, that list cannot be written, the batch never starts, every blob reads as absent, +# and the answer was merged="1" refs="1" (a branch holding unmerged work), --abi exit 0 with no drift: a measured-looking zero made +# of nothing (the whereis hits="0" bug's siblings). The refusal is the whereis mechanism: exit 1, the cause named on stderr, no +# document on stdout. Arms: a control (the same repo answers when the directory exists, which proves the refusal is about the +# missing directory and not about the fixture), then the refusal on each verb and on the MCP twin. +K75="$TMP/k75repo"; mkdir -p "$K75" +k(){ git -C "$K75" "$@" >/dev/null 2>&1; } +k init -q -b main +k config commit.gpgsign false +printf 'int base( void )\n{\n return 1;\n}\n' > "$K75/a.c" +printf '#pragma once\nstruct Uniforms\n{\n float x;\n float y;\n};\nstatic_assert( sizeof( Uniforms ) == 8, "uniform GPU contract" );\n' > "$K75/shader.h" +k add -A; k commit -qm base +k checkout -qb side +printf 'int base( void )\n{\n return 1;\n}\n\nint sideOnlyThing( void )\n{\n return 42;\n}\n' > "$K75/a.c" +printf '#pragma once\nstruct Uniforms\n{\n float x;\n float y;\n float z;\n};\nstatic_assert( sizeof( Uniforms ) == 8, "uniform GPU contract" );\n' > "$K75/shader.h" +k commit -qam "side work" +k checkout -q main +printf 'int base( void )\n{\n return 1;\n}\n\nint mainWork( void )\n{\n return 7;\n}\n' > "$K75/a.c" +k commit -qam "live-line work" +K75GOOD="$TMP/k75tmp"; mkdir -p "$K75GOOD" +K75BAD="$TMP/k75-no-such-dir" # never created: TMPDIR names a directory that does not exist +printf 'side\tunmerged\n' > "$TMP/k75labels.tsv" +k75(){ local out="$1" tmpd="$2"; shift 2; env TMPDIR="$tmpd" "$BIN" "$K75" "$@" --no-cache --legend=full >"$out" 2>"$out.err"; } + +k75 "$TMP/k75_s_good.out" "$K75GOOD" --stray-content; rcS=$? +[ "$rcS" = 0 ] && grep -q 'v="unmerged"' "$TMP/k75_s_good.out" \ + && ok "K75 control: --stray-content answers the branch with work as unmerged when the cache dir works" \ + || no "K75 control: --stray-content (rc=$rcS) did not report v=\"unmerged\" - the refusal arms below would prove nothing" +k75 "$TMP/k75_a_good.out" "$K75GOOD" --stray-content --abi; rcA=$? +[ "$rcA" = 2 ] \ + && ok "K75 control: --abi reports the break (exit 2) when the cache dir works" \ + || no "K75 control: --abi exited $rcA, not 2" + +for v in "--stray-content" "--stray-content --abi" "--stray-content --plan" "--eval-stray=$TMP/k75labels.tsv"; do + # shellcheck disable=SC2086 + k75 "$TMP/k75_bad.out" "$K75BAD" $v; rcB=$? + if [ "$rcB" = 1 ] && [ ! -s "$TMP/k75_bad.out" ] && grep -q "could not read the branches' blobs" "$TMP/k75_bad.out.err" && grep -q 'TMPDIR' "$TMP/k75_bad.out.err"; then + ok "K75: $v with TMPDIR missing refuses (exit 1, no document, the cause and TMPDIR named on stderr)" + else + no "K75: $v with TMPDIR missing did not refuse (rc=$rcB, stdout $( wc -c <"$TMP/k75_bad.out" | tr -d ' ' ) bytes): $( grep -v 'math degraded\|compact legend' "$TMP/k75_bad.out.err" | head -c 200 )" + fi +done +printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize"}' \ + '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"stray_content","arguments":{"path":"'"$K75"'"}}}' \ + | env TMPDIR="$K75BAD" "$BIN" --mcp 2>/dev/null | tail -1 >"$TMP/k75_mcp.json" +grep -q '"code":-32603' "$TMP/k75_mcp.json" && grep -q "could not read the branches" "$TMP/k75_mcp.json" \ + && ok "K75: the MCP stray_content twin answers -32603 naming the cause, not a verdict or 'not a git repository'" \ + || no "K75: the MCP twin did not refuse with -32603: $( head -c 240 "$TMP/k75_mcp.json" )" +printf '%s\n' '{"jsonrpc":"2.0","id":1,"method":"initialize"}' \ + '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"stray_content","arguments":{"path":"'"$K75"'"}}}' \ + | env TMPDIR="$K75GOOD" "$BIN" --mcp 2>/dev/null | tail -1 >"$TMP/k75_mcp_good.json" +grep -q 'unmerged' "$TMP/k75_mcp_good.json" && ! grep -q '"error"' "$TMP/k75_mcp_good.json" \ + && ok "K75 control: the MCP twin answers when the cache dir works" || no "K75 control: the MCP twin did not answer on a good TMPDIR" + + # ══ fixture A: a SHALLOW clone — the CI default, and the case that reported every branch as merged ═══════ # # `side` carries two lines of genuinely unmerged work. The shallow clone keeps one commit per ref, so the @@ -273,6 +334,69 @@ REFS_BEFORE="$( git -C "$C" for-each-ref --format='%(refname) %(objectname)' 2>/ [ "$( git -C "$C" for-each-ref --format='%(refname) %(objectname)' 2>/dev/null | cksum )" = "$REFS_BEFORE" ] \ && ok "read-only: no ref was created, moved or deleted" || no "read-only: a crossref verb wrote a ref" +# ══ 6) a blob stream that never ran is not a measured zero (cli-perf-068; the cli-floor §4 finding) ═════════ +# --whereis reads every blob through ONE `git cat-file --batch` whose sha list is a file under the cache dir. With that +# dir unusable (TMPDIR naming a directory that does not exist) the batch never started, and the answer still printed +# on-head="0" hits="0" plus a near-miss note whose retry= was the very name asked — on a repository whose HEAD defines +# it. Now both hosts REFUSE (CLI: exit 1 naming the cause; MCP: an error) instead of printing a count nothing measured. +# 6c is the near-miss negative: a GENUINE zero over a working cache dir must still answer hits="0", exit 0. +H="$TMP/blobless"; mkdir -p "$H" "$TMP/goodtmp" +printf 'int whereisProbeName( int x ) { return x; }\nint whereisProbeCaller( void ) { return whereisProbeName( 1 ); }\n' > "$H/a.c" +git -C "$H" init -q -b main && git -C "$H" add a.c && git -C "$H" commit -qm one +TMPDIR="$TMP/goodtmp/" "$BIN" "$H" --whereis=whereisProbeName >"$TMP/b6a.xml" 2>"$TMP/b6a.err"; rc6a=$? +H6A="$( attr hits < "$TMP/b6a.xml" )" +if [ "$rc6a" = 0 ] && [ -n "$H6A" ] && [ "$H6A" -ge 2 ] 2>/dev/null; then ok "6a control: a working cache dir finds the name ($H6A hits, exit 0)" +else no "6a control: expected exit 0 and >=2 hits, got rc=$rc6a hits=${H6A:-none}"; fi +TMPDIR="$TMP/no-such-dir/" "$BIN" "$H" --whereis=whereisProbeName >"$TMP/b6b.out" 2>"$TMP/b6b.err"; rc6b=$? +if [ "$rc6b" != 0 ] && ! grep -q '"$TMP/b6c.xml" 2>/dev/null; rc6c=$? +if [ "$rc6c" = 0 ] && [ "$( attr hits < "$TMP/b6c.xml" )" = "0" ]; then ok "6c negative: a genuine zero over a working cache dir still answers hits=\"0\", exit 0" +else no "6c negative: a genuine zero was refused or miscounted (rc=$rc6c, hits=$( attr hits < "$TMP/b6c.xml" ))"; fi +if command -v python3 >/dev/null 2>&1; then + M6="$( TMPDIR="$TMP/no-such-dir/" python3 - "$BIN" "$H" <<'PY' +import json, subprocess, sys +msgs = [ { "jsonrpc": "2.0", "id": 1, "method": "initialize" }, + { "jsonrpc": "2.0", "id": 2, "method": "tools/call", "params": { "name": "whereis", "arguments": { "path": sys.argv[ 2 ], "symbol": "whereisProbeName" } } } ] +p = subprocess.run( [ sys.argv[ 1 ], "--mcp" ], input = "".join( json.dumps( m ) + "\n" for m in msgs ), capture_output = True, text = True, timeout = 300 ) +lines = [ l for l in p.stdout.splitlines() if l.strip() ] +d = json.loads( lines[ -1 ] ) if lines else {} +res = d.get( "result", {} ) +text = ( res.get( "content" ) or [ {} ] )[ 0 ].get( "text", "" ) +if "error" in d: + print( "ERROR " + d[ "error" ].get( "message", "" ) ) +elif res.get( "isError" ): + print( "ERROR " + text ) +else: + print( "ANSWER " + text[ :200 ] ) +PY +)" + case "$M6" in + "ERROR "*"could not read the trees"*) ok "6d MCP twin: the same unreadable stream is an error naming the cause, not a hit count" ;; + *) no "6d MCP twin: expected an error naming the unread blobs, got: $( printf '%s' "$M6" | head -c 200 )" ;; + esac +fi + +# 6e (train 26c: the K75 x cli-perf-068 one-renderer item): --whereis and the cross-branch sweep refuse in ONE sentence that +# differs only in its noun (crossref::blobsUnreadCause), so the --whereis refusal names TMPDIR and its MCP twin names the cache +# directory, as the sweep's refusal (the K75 loop above, its last stderr) always did. Red before the fold: the --whereis CLI +# sentence never asked about TMPDIR, and its MCP twin never named the directory the sha list is written under. +K75WORDS="(the one \`git cat-file --batch\` did not start or stopped early; its sha list is written under " +K75ASK=" - is TMPDIR a writable directory?)" +if grep -qF "could not read the trees' blobs $K75WORDS" "$TMP/b6b.err" && grep -qF "$K75ASK" "$TMP/b6b.err" \ + && grep -qF "could not read the branches' blobs $K75WORDS" "$TMP/k75_bad.out.err" && grep -qF "$K75ASK" "$TMP/k75_bad.out.err"; then + ok "6e one renderer: the --whereis and cross-branch refusals share every word but the noun, TMPDIR named in both" +else + no "6e one renderer: the two refusals differ beyond the noun — whereis: $( grep 'could not read' "$TMP/b6b.err" | head -c 260 ) | sweep: $( grep 'could not read' "$TMP/k75_bad.out.err" | head -c 260 )" +fi +if [ -n "${M6:-}" ]; then + case "$M6" in + "ERROR "*"could not read the trees' blobs $K75WORDS"*"$K75ASK"*) ok "6e MCP: the whereis error carries the same sentence (the directory and TMPDIR named)" ;; + *) no "6e MCP: the whereis error is not the shared sentence: $( printf '%s' "$M6" | head -c 260 )" ;; + esac +fi + # G4: every emitted document must survive xmllint if command -v xmllint >/dev/null 2>&1; then g4=0 diff --git a/test/docmdcachecheck.sh b/test/docmdcachecheck.sh index 7b516f9ce..cc40e00cb 100755 --- a/test/docmdcachecheck.sh +++ b/test/docmdcachecheck.sh @@ -10,6 +10,8 @@ # 4. changing the doc's bytes re-invokes (the key is the content, not the path) # 5. an EMPTY extraction is never cached (machine fact, not a byte fact): a no-output fake is # re-invoked on the next run rather than wedging "" into the cache +# 6-9. (cli-perf-068) no shell is started for a markitdown that cannot run, and that answer is never memoized: +# absent-then-installed, a directory ahead on PATH, a non-executable file, an empty PATH entry (the cwd) # Does NOT edit test/regression.sh (the orchestrator wires it). # # RIPWIRE_BIN=build/ripwire bash test/docmdcachecheck.sh @@ -74,5 +76,52 @@ PATH="$TMP/bin:$PATH" "$BIN" "$TMP/corpus" >/dev/null 2>&1 PATH="$TMP/bin:$PATH" "$BIN" "$TMP/corpus" >/dev/null 2>&1 if [ "$( count )" = "5" ]; then ok "empty extraction not cached (silent bridge re-invoked each run)"; else no "empty-extraction arm: expected 5 total invocations, got $( count )"; fi +# 6-9 (cli-perf-068): the post-pass asks the shell's own question — could `markitdown` start at all? — before it +# forks one, so a machine without markitdown no longer pays a /bin/sh per bridge doc per run. Absence is not memoized: +# each arm below would go red if the answer were cached, mis-walked PATH, or ruled the shell out when it could run. +# The PATH is pinned to the fake's dirs plus /usr/bin:/bin so a markitdown installed on the host cannot answer. +SYS_PATH="/usr/bin:/bin" +mkdir -p "$TMP/b6" "$TMP/b7dir/markitdown" "$TMP/b7" "$TMP/b8" "$TMP/b9" +cat > "$TMP/fake" <<'FAKE' +#!/bin/bash +echo "invoked $1" >> "$FAKE_LOG" +echo "# Extracted Deck" +echo "hello from fake markitdown" +FAKE +chmod +x "$TMP/fake" +newdoc(){ printf 'arm %s doc bytes %s $$=%s' "$1" "$( date +%s%N 2>/dev/null || date +%s )" "$$" > "$TMP/corpus/deck.docx"; } + +# 6. absent, then installed between two runs: the first run invokes nothing and carries no doc Section; the second +# sees the new program (a memoized "absent" would keep it at zero) +newdoc 6; before="$( count )" +PATH="$TMP/b6:$SYS_PATH" "$BIN" "$TMP/corpus" > "$TMP/absent.xml" 2>/dev/null +if [ "$( count )" = "$before" ] && ! grep -q 'p="deck.docx"' "$TMP/absent.xml" && grep -q 'a.cpp' "$TMP/absent.xml"; then + ok "absent bridge: no invocation, no doc Section, map still produced" +else no "absent bridge: expected $before invocations and no Section, got $( count ) (map: $( wc -c < "$TMP/absent.xml" ) B)"; fi +cp "$TMP/fake" "$TMP/b6/markitdown" +PATH="$TMP/b6:$SYS_PATH" "$BIN" "$TMP/corpus" > "$TMP/installed.xml" 2>/dev/null +if [ "$( count )" = "$(( before + 1 ))" ] && grep -q 'p="deck.docx"' "$TMP/installed.xml"; then + ok "bridge installed after an absent run is invoked on the next run (absence is not memoized)" +else no "installed-after-absent: expected $(( before + 1 )) invocations and a Section, got $( count )"; fi + +# 7. a DIRECTORY named markitdown earlier on PATH is not a program (sh skips it): the real one after it still runs +newdoc 7; before="$( count )"; cp "$TMP/fake" "$TMP/b7/markitdown" +PATH="$TMP/b7dir:$TMP/b7:$SYS_PATH" "$BIN" "$TMP/corpus" >/dev/null 2>&1 +if [ "$( count )" = "$(( before + 1 ))" ]; then ok "a directory named markitdown ahead on PATH does not hide the program behind it" +else no "dir-ahead arm: expected $(( before + 1 )) invocations, got $( count )"; fi + +# 8. a NON-executable file named markitdown is no program either (sh: rc 126 -> ""): same map as the absent run +cp "$TMP/fake" "$TMP/b8/markitdown"; chmod -x "$TMP/b8/markitdown" +newdoc 8; before="$( count )" +PATH="$TMP/b8:$SYS_PATH" "$BIN" "$TMP/corpus" > "$TMP/noexec.xml" 2>/dev/null +if [ "$( count )" = "$before" ] && ! grep -q 'p="deck.docx"' "$TMP/noexec.xml"; then ok "a non-executable markitdown is treated as absent, as sh treats it" +else no "non-executable arm: expected $before invocations and no Section, got $( count )"; fi + +# 9. an EMPTY PATH entry is the current directory to sh: a markitdown there must still run +newdoc 9; before="$( count )"; cp "$TMP/fake" "$TMP/b9/markitdown" +( cd "$TMP/b9" && PATH=":$SYS_PATH" "$BIN" "$TMP/corpus" >/dev/null 2>&1 ) +if [ "$( count )" = "$(( before + 1 ))" ]; then ok "an empty PATH entry (the cwd) reaches a markitdown there, as sh does" +else no "empty-PATH-entry arm: expected $(( before + 1 )) invocations, got $( count )"; fi + [ "$fail" = 0 ] && echo "ALL PASS" || echo "FAILURES ABOVE" exit $fail diff --git a/test/estchargecheck.sh b/test/estchargecheck.sh index 05117459d..b78dd55f2 100755 --- a/test/estchargecheck.sh +++ b/test/estchargecheck.sh @@ -23,6 +23,14 @@ # gate — that band is the recurrence guard, and it is the reason this file asserts a band and never a # bit-exact token count (house rule: float/estimate assertions get a tolerance, never bit-equality). # +# BY-DESIGN BREAK (rule 4; owner ruling 2026-10-09, option B; reports/lego-charge.md fix round 5): #11 A7's old bar +# "--token-budget=2000 --detail=20 --with-graph on src stays inside N x 2.36 x 1.15" no longer holds for that ONE shape. +# With BOTH trailing sections under an explicit ceiling the answer KEEPS its ranked rows (a gold owner row, textual-12 +# Reactive._set, was cut when the two sections were reserved out of ) and goes over the budget, labelled +# over_ceiling="1" with a TRUE reason and a next= on every cut: answers first, a budget never silently removes an +# answer. The old assertion is kept as A7 twin wherever it must still hold (each flag alone at 2000, and both flags at +# a budget the rows fit); A7B asserts the new contract for both-flags-over-budget. +# # Usage: bash test/estchargecheck.sh [BIN] | RIPWIRE_BIN=asan/ripwire bash test/estchargecheck.sh # Exits non-zero on any failure; prints PASS/FAIL per check, ALL PASS on success. @@ -127,7 +135,23 @@ for entry in "${SHAPES[@]}"; do done # ── #2: MONOTONE charging — adding a payload to the same map can never LOWER the reported estimate. ──── -E_MAP="$( est_of "$TMP/map.out" )" +# D7 (2026-10-08, by-design break, PROCESS rule 4): the bare map now carries the inventory tier (, the files its +# ranked rows leave out), and a payload verb's ride-along map does not — so "the same map" is the bare map WITHOUT that +# tier. --no-inventory is that map (the same rows, no tier; #2-rows proves both premises). The old comparison is kept +# as #2-tier: the bare map with its tier must charge at least what the rows alone do. +"$BIN" src --top-k=10 --no-inventory --no-cache >"$TMP/map_rows.out" 2>/dev/null +E_MAP="$( est_of "$TMP/map_rows.out" )" +E_TIER="$( est_of "$TMP/map.out" )" +rows_of(){ sed 's/ (est $E_MAP)" +else + no "#2-rows premise broken: the rows-only twin is not the bare map minus its tier (est ${E_MAP:-?}) — #2/#3 compare against the wrong map" +fi +[ -n "$E_TIER" ] && [ -n "$E_MAP" ] && [ "$E_TIER" -ge "$E_MAP" ] 2>/dev/null \ + && ok "#2-tier: the bare map with its inventory reports est_tokens=$E_TIER >= the rows alone ($E_MAP)" \ + || no "#2-tier: the bare map with its inventory reports ${E_TIER:-?} < the rows alone (${E_MAP:-?}) — the tier is uncharged" mono=1 for label in metrics sigs src bodies outline "bodies+sigs"; do E="$( est_of "$TMP/$label.out" )" @@ -141,7 +165,8 @@ done # streamed 12 KB / 67 KB. Budget is set from the bare map's own estimate (so the map alone always # fits) and each payload arm must fail it. ──────────────────────────────────────────────────────────── TB=$(( E_MAP + 20 )) -"$BIN" src --top-k=10 --token-budget=$TB --no-cache >"$TMP/tb_map.out" 2>"$TMP/tb_map.err" +# D7: the control is the rows-only twin above (the bare map's tier is charged and need not fit this budget) +"$BIN" src --top-k=10 --no-inventory --token-budget=$TB --no-cache >"$TMP/tb_map.out" 2>"$TMP/tb_map.err" rc_map=$? [ "$rc_map" -eq 0 ] \ && ok "#3 control: the bare map fits --token-budget=$TB (exit 0)" \ @@ -580,17 +605,48 @@ done # A7 — and --token-budget must SHAPE the bodies, not just the signatures. D10: --for trims to fit, exit 0. # The pre-fix binary streamed 68 035 B here against a 4 248 B ceiling; the bar is the stated ceiling plus the # documented single-entry overshoot tolerance (serialize.h kCeilingFirstEntryTolerance = 1.15). +# BY-DESIGN BREAK (owner ruling 2026-10-09, option B — see this file's header): the old arm ran ONLY the both-flags shape +# at 2000, which now keeps its rows and goes over on purpose. A7 TWIN keeps the old assertion, unchanged in substance +# (exit 0, delivered <= the allowance), on every shape where it must still hold: each trailing flag alone at 2000, and +# both flags at 5000 where the ranked rows fit (measured at fix round 5: 4727 / 4769 / 12337 B vs 5428 / 5428 / 13570 B). TBF=2000 ALLOW="$( awk "BEGIN{printf \"%d\", $TBF*2.36*1.15}" )" +for tw in "2000:--detail=20" "2000:--with-graph" "5000:--detail=20 --with-graph"; do + tw_n="${tw%%:*}"; tw_args="${tw#*:}" + tw_allow="$( awk "BEGIN{printf \"%d\", $tw_n*2.36*1.15}" )" + # shellcheck disable=SC2086 + "$BIN" src --for="$FOR_TASK" --token-budget=$tw_n $tw_args --no-cache >"$TMP/f_tw.out" 2>/dev/null + rc_tw=$? + tw_b="$( bytes_of "$TMP/f_tw.out" )" + { [ "$rc_tw" -eq 0 ] && [ -n "$tw_b" ] && [ "$tw_b" -le "$tw_allow" ]; } 2>/dev/null \ + && ok "#11 A7 twin --for --token-budget=$tw_n $tw_args: $tw_b B within the $tw_allow B allowance (exit 0, SHAPED)" \ + || no "#11 A7 twin --for --token-budget=$tw_n $tw_args: ${tw_b:-?} B vs the $tw_allow B allowance, exit $rc_tw — the budget does not bound the appended sections" +done +# A7B — THE NEW CONTRACT for both flags OVER the budget (owner ruling, option B). Same query, same 2000 tokens. It must: +# (premise) deliver past the allowance at exit 0 — else the shape now fits and this arm must be re-anchored, not passed; +# (rows) keep every row the same budget shows WITHOUT the trailing flags, and those rows are the uncapped +# answer's top rows in its order (no higher-ranked row of the uncapped answer is missing) — the trailing +# sections never remove an answer row; +# (label) carry over_ceiling="1" on the root; +# (reason) name the reason TRUTHFULLY: the last-rung note is the kept-rows one, not "the header floor ... exceeds", +# and what it names is there — the header alone fits the allowance, and rows, the graph block and a +# first --detail body are all present; +# (next) a cut first body carries a next= that recovers exactly the lines it cut (checked on A7R's corpus below, +# whose first body is longer than the floor; here a first body served whole carries no truncated=). +KEPT_NOTE='[over_ceiling= is 1 on the root: the ranked rows are kept, not cut for the trailing sections' +FLOOR_NOTE='[over_ceiling= is 1 on the root: the header floor' +rows_of(){ grep -aoE ']*>' "$1" | sed -nE 's/.* n="([^"]*)".*/\1/p'; } "$BIN" src --for="$FOR_TASK" --token-budget=$TBF --detail=20 --with-graph --no-cache >"$TMP/f_tb.out" 2>"$TMP/f_tb.err" rc_ftb=$? +"$BIN" src --for="$FOR_TASK" --token-budget=$TBF --no-cache >"$TMP/f_tbp.out" 2>/dev/null FTB="$( bytes_of "$TMP/f_tb.out" )" # THE UNPAID TWIN (knob-honesty-068; orchestrator rulings 2026-10-07, PROCESS rules 4/5). A capped under a # hard ceiling carries next= — the call that recovers its cut — and pays for it from its rows ONLY where that makes the # answer fit (ruling C3). Where it cannot (the rank 1..4 floor has nothing left to shed, or the answer lands past its ceiling # paid or not) no row is dropped for it: the recovery handle ALWAYS ships, unpaid, and the document may then land past the # allowance, and only if it says so. This is not a wider allowance: every condition of the old bound still holds for -# everything except the disclosed handle. sigs_unpaid_twin FILE ALLOW → exit 0 iff ALL of: the root carries +# everything except the disclosed handle. Train 26c: the single A7 arm it guarded is superseded by the lane's A7 twin loop + +# A7B below; the twin itself still serves the sweep and the floor arm. sigs_unpaid_twin FILE ALLOW → exit 0 iff ALL of: the root carries # over_ceiling="1"; the legend carries the unpaid clause defining it; is capped and carries next="--for=…"; # est_tokens is positive and prices the delivered bytes inside the 2.00-4.20 B/tok band (honest: it moved with the handle); # and the document WITHOUT the handle and its label (the next= attribute, over_ceiling="1", the clause) is within ALLOW — @@ -611,19 +667,67 @@ est = re.search( r' est_tokens="(\d+)"', root.group( 0 ) ) if root else None b = len( d.encode( 'utf-8' ) ) if not est or int( est.group( 1 ) ) <= 0 or not ( 200 <= b * 100 // int( est.group( 1 ) ) <= 420 ): why.append( 'est_tokens absent or out of band' ) rest = b - ( len( nxt.group( 0 ) ) if nxt else 0 ) - len( ' over_ceiling="1"' ) - len( clause ) -lastRung = '[over_ceiling= is 1 on the root: the header floor' in d +lastRung = '[over_ceiling= is 1 on the root: ' in d # either last-rung reason: the header floor, or (lego-charge) the kept rows if rest > allow and not lastRung: why.append( f'{rest} B without the handle > {allow} B (and no last-rung disclosure)' ) print( '; '.join( why ) if why else f'{b} B, {rest} B without the handle <= {allow} B, est_tokens={est.group( 1 )}' ) sys.exit( 1 if why else 0 ) PYF } -if { [ "$rc_ftb" -eq 0 ] && [ "$FTB" -le "$ALLOW" ]; } 2>/dev/null; then - ok "#11 A7 --for --token-budget=$TBF --detail=20 --with-graph: $FTB B within the $ALLOW B allowance (exit 0, SHAPED)" -elif [ "$rc_ftb" -eq 0 ] && a7why="$( sigs_unpaid_twin "$TMP/f_tb.out" "$ALLOW" )"; then - ok "#11 A7 --for --token-budget=$TBF --detail=20 --with-graph: the unpaid twin — $a7why, labelled over_ceiling=\"1\" with next= (exit 0, SHAPED)" +a7b_root="$( grep -aoE '^]*>' "$TMP/f_tb.out" | head -1 )" +a7b_hdr="$( python3 -c 'import sys; d=open(sys.argv[1],"rb").read(); print(d.find(b""$TMP/a7b.rows"; rows_of "$TMP/f_tbp.out" >"$TMP/a7b.plain"; rows_of "$TMP/f_for.out" >"$TMP/a7b.unc" +a7b_n="$( wc -l < "$TMP/a7b.rows" | tr -d ' ' )"; a7b_pn="$( wc -l < "$TMP/a7b.plain" | tr -d ' ' )" +a7b_why="" +{ [ "$rc_ftb" -eq 0 ] && [ -n "$a7b_root" ] && [ -n "$FTB" ] && [ "$FTB" -gt "$ALLOW" ]; } 2>/dev/null \ + || a7b_why="$a7b_why premise:${FTB:-?}B<=${ALLOW}B-or-exit-$rc_ftb(re-anchor)" +{ [ "$a7b_pn" -ge 1 ] && [ "$a7b_n" -ge "$a7b_pn" ] && [ -z "$( grep -vxF -f "$TMP/a7b.rows" "$TMP/a7b.plain" )" ] \ + && [ "$( head -n "$a7b_n" "$TMP/a7b.unc" )" = "$( cat "$TMP/a7b.rows" )" ]; } 2>/dev/null \ + || a7b_why="$a7b_why rows:$a7b_n-vs-plain-$a7b_pn-or-not-the-uncapped-prefix" +[ "${a7b_root#* over_ceiling=\"1\"}" != "$a7b_root" ] || a7b_why="$a7b_why no-over_ceiling" +{ grep -aqF "$KEPT_NOTE" "$TMP/f_tb.out" && ! grep -aqF "$FLOOR_NOTE" "$TMP/f_tb.out" && [ "${a7b_hdr:--1}" -gt 0 ] && [ "$a7b_hdr" -le "$ALLOW" ] \ + && grep -aq ']*>/dev/null \ + || a7b_why="$a7b_why reason:not-the-kept-rows-note-or-untrue(header ${a7b_hdr:-?}B)" +a7b_b1="$( grep -aoE ']*>' "$TMP/f_tb.out" | head -1 )" +{ [ -n "$a7b_b1" ] && { [ "${a7b_b1#* truncated=\"1\"}" = "$a7b_b1" ] || [ "${a7b_b1#* next=\"--expand=}" != "$a7b_b1" ]; }; } \ + || a7b_why="$a7b_why first-body-cut-without-next" +[ -z "$a7b_why" ] \ + && ok "#11 A7B (by-design, owner option B) --token-budget=$TBF --detail=20 --with-graph: $FTB B past the $ALLOW B allowance, over_ceiling=\"1\", keeps all $a7b_pn rows of the flagless budget answer ($a7b_n rows, the uncapped answer's top), reason names the kept rows (header ${a7b_hdr} B fits alone)" \ + || no "#11 A7B (by-design, owner option B) --token-budget=$TBF --detail=20 --with-graph —$a7b_why" +# A7B-X (fix round 6, checklist 25): the kept-rows note rides INSIDE the header XML comment, so the answers that carry it +# must be well-formed XML — its first spelling held two flag names with their leading double hyphens and every such +# answer failed xmllint ("Double hyphen within comment"), on the compact default and on --legend=full alike. +if command -v xmllint >/dev/null 2>&1; then + "$BIN" src --for="$FOR_TASK" --token-budget=$TBF --detail=20 --with-graph --legend=full --no-cache >"$TMP/f_tbl.out" 2>/dev/null + a7x_bad="" + for f in f_tb f_tbl; do + grep -aqF "$KEPT_NOTE" "$TMP/$f.out" || a7x_bad="$a7x_bad $f:no-kept-rows-note(premise)" + xmllint --noout "$TMP/$f.out" 2>/dev/null || a7x_bad="$a7x_bad $f:xmllint-rejected" + done + [ -z "$a7x_bad" ] \ + && ok "#11 A7B-X the kept-rows answers (compact + --legend=full) carry the note and are well-formed XML" \ + || no "#11 A7B-X the kept-rows note breaks the answer's XML or no longer rides —$a7x_bad" else - no "#11 A7 --for --token-budget=$TBF --detail=20 --with-graph: $FTB B vs the $ALLOW B allowance, exit $rc_ftb — the budget does not bound the appended sections${a7why:+ (unpaid twin: $a7why)}" + printf ' SKIP #11 A7B-X xmllint not installed\n' fi +# A7B-N (fix round 6, checklist 1 — the negative of A7B's reason): the kept-rows note is chosen only when the header as +# built fits the allowance by itself. Same both-flags shape, but a 700-char task at 300 tokens: the verbatim task echo +# alone puts the header past the 814 B allowance, so the TRUE reason is the header floor and the kept-rows note would be +# false. Red on the mutant that drops the header-fits term from keptTrailingOverrun (review mutant MX). +A7N_TASK="$( python3 -c 'w="serialize the map rows into the output buffer while keeping the row order stable and flushing the writer "; print((w*10)[:700].strip())' )" +"$BIN" src --for="$A7N_TASK" --token-budget=300 --detail=3 --with-graph --no-cache >"$TMP/f_tbn.out" 2>/dev/null +rc_a7n=$? +a7n_allow="$( awk "BEGIN{printf \"%d\", 300*2.36*1.15}" )" +a7n_root="$( grep -aoE '^]*>' "$TMP/f_tbn.out" | head -1 )" +a7n_hdr="$( python3 -c 'import sys; d=open(sys.argv[1],"rb").read(); print(d.find(b"/dev/null \ + || a7n_why="$a7n_why premise:header-${a7n_hdr:-?}B-not-past-${a7n_allow}B-or-exit-$rc_a7n(re-anchor)" +[ "${a7n_root#* over_ceiling=\"1\"}" != "$a7n_root" ] || a7n_why="$a7n_why no-over_ceiling" +grep -aqF "$FLOOR_NOTE" "$TMP/f_tbn.out" || a7n_why="$a7n_why no-header-floor-note" +! grep -aqF "$KEPT_NOTE" "$TMP/f_tbn.out" || a7n_why="$a7n_why kept-rows-note-false-here" +[ -z "$a7n_why" ] \ + && ok "#11 A7B-N 700-char task at --token-budget=300 --detail=3 --with-graph: header ${a7n_hdr} B alone past the ${a7n_allow} B allowance, the last rung names the header floor, not the kept rows" \ + || no "#11 A7B-N header-alone-over shape names the wrong reason —$a7n_why" # and without an explicit --token-budget the bodies keep their own budget: the bundle must NOT have shrunk { [ "$( bytes_of "$TMP/f_for_detail.out" )" -gt "$ALLOW" ]; } 2>/dev/null \ && ok "#11 A7 no --token-budget: --detail keeps its --pack-budget-bytes budget (unbudgeted bundle unshrunk)" \ @@ -685,11 +789,20 @@ for spec in "default:760:1500:10:serialize the map:" "detail_graph:2880:3080:10: s_b="$( bytes_of "$A7S/o.xml" )" s_a="$( awk "BEGIN{printf \"%d\", $N*2.36*1.15}" )" s_root="$( grep -aoE '^]*>' "$A7S/o.xml" | head -1 )" - # past the allowance: the ladder's last rung says so, or (knob-honesty-068) the unpaid twin above holds in full - if [ "$s_rc" -eq 0 ] && [ "$s_b" -gt "$s_a" ] && ! grep -aqF '[over_ceiling= is 1 on the root: the header floor' "$A7S/o.xml" \ + # fix round 5 (lego-charge): the last rung now names one of two reasons, each only where it is true — the header floor, + # or (both trailing sections under the ceiling) the kept rows. Either is the disclosure; the kept-rows one must also be + # TRUE here: only on the both-flags shape, and with the header alone inside the allowance. + s_hdr="$( python3 -c 'import sys; d=open(sys.argv[1],"rb").read(); print(d.find(b" unpaid twin above + # holds in full + elif [ "$s_rc" -eq 0 ] && [ "$s_b" -gt "$s_a" ] && ! grep -aqF '[over_ceiling= is 1 on the root: ' "$A7S/o.xml" \ && sigs_unpaid_twin "$A7S/o.xml" "$s_a" >/dev/null; then a7s_floor=$(( a7s_floor + 1 )) - elif [ "$s_rc" -ne 0 ] || { [ "$s_b" -gt "$s_a" ] && ! grep -aqF '[over_ceiling= is 1 on the root: the header floor' "$A7S/o.xml"; }; then + elif [ "$s_rc" -ne 0 ] || { [ "$s_b" -gt "$s_a" ] && ! grep -aqF '[over_ceiling= is 1 on the root: ' "$A7S/o.xml"; }; then a7s_badn=$(( a7s_badn + 1 )) [ "$a7s_badn" -le 6 ] && a7s_bad="$a7s_bad $s_label@$N=${s_b}/${s_a}B(exit $s_rc)" elif [ "$s_b" -le "$s_a" ] && [ "${s_root#* over_ceiling=\"1\"}" != "$s_root" ]; then @@ -705,18 +818,27 @@ done [ "$a7s_inside_labelled" -gt 0 ] \ && ok "#11 A7 sweep control: $a7s_inside_labelled budget(s) carry a root over_ceiling=\"1\" INSIDE the allowance — the sweep crosses the late-label band" \ || no "#11 A7 sweep control: no budget carried over_ceiling=\"1\" inside the allowance — the sweep no longer reaches the est_tokens > N band, re-anchor its ranges" +# TRAIN 26c RE-PIN (2026-10-10) of the three regime arms below — the floor twin, C3 and its near miss. lego-charge charges +# / to the budget at their served size and #362's rides the bundle, so on this corpus the +# merged binary's 4-row floor answer costs ~270 B more than 26b's: at "serialize the map" the cut never leaves 5 rows while +# both the unpaid and the paid answer land over (the paid 4-row answer fits from 920, the cut leaves 5 rows from 920 — no C3 +# window at any budget 700..1400, swept step 10 on the merged binary and on lego-charge's pre-continuation binary 1cfbfdcc). +# The task "map row serializer" on the same corpus has all three regimes: the floor at 890..910 (4 rows, the handle unpaid, +# no last-rung note; the pre-continuation binary fits with 4 rows), C3 at 920..939 (the cut leaves 5 rows — 1cfbfdcc serves +# 5, over, last rung fired — and the paid 4-row answer is over too), the payable near miss from 940 (the paid 4-row answer +# fits, no label). Pinned at 900 / 925 / 970. The sweep above keeps "serialize the map". Assertions unchanged. # A7 UNPAID TWIN, EXERCISED EVERY RUN (knob-honesty-068; rulings 2026-10-07). The A7 arm and the sweep accept the twin only # as an alternative, so a fixture that stopped reaching it would leave it untested. On this git-less corpus at # --token-budget=780 the default bundle's sits at its rank 1..4 floor: the answer the cut alone leaves FITS (the # pre-continuation binary, 255dc199, serves it at est_tokens=774 with no label), its next= cannot be paid in rows, and the # document lands past the allowance only by the handle — the twin must hold, with no last-rung disclosure to lean on. -( cd "$A7S" && "$BIN" corpus --for="serialize the map" --token-budget=780 --no-cache ) >"$A7S/floor.xml" 2>/dev/null; fl_rc=$? -fl_a="$( awk 'BEGIN{printf "%d", 780*2.36*1.15}' )" +( cd "$A7S" && "$BIN" corpus --for="map row serializer" --token-budget=900 --no-cache ) >"$A7S/floor.xml" 2>/dev/null; fl_rc=$? +fl_a="$( awk 'BEGIN{printf "%d", 900*2.36*1.15}' )" if [ "$fl_rc" -eq 0 ] && ! grep -aqF '[over_ceiling= is 1 on the root: the header floor' "$A7S/floor.xml" \ && fl_why="$( sigs_unpaid_twin "$A7S/floor.xml" "$fl_a" )"; then - ok "#11 A7 unpaid twin @780: $fl_why — next= ships at the floor and the overshoot is labelled over_ceiling=\"1\" with its clause" + ok "#11 A7 unpaid twin @900: $fl_why — next= ships at the floor and the overshoot is labelled over_ceiling=\"1\" with its clause" else - no "#11 A7 unpaid twin @780 (exit $fl_rc): ${fl_why:-the ladder last rung fired, so the twin is not the only disclosure} — the floor did not ship its next= with a labelled, honestly priced overshoot" + no "#11 A7 unpaid twin @900 (exit $fl_rc): ${fl_why:-the ladder last rung fired, so the twin is not the only disclosure} — the floor did not ship its next= with a labelled, honestly priced overshoot" fi # C3 (orchestrator ruling 2026-10-07): NEVER DROP ROWS TO PAY FOR next= WHEN THE ANSWER IS OVER ITS CEILING ANYWAY. At 790 the # answer the cut alone leaves is past its ceiling already (255dc199 serves 5 rows there at est_tokens=952, last rung fired); @@ -724,30 +846,214 @@ fi # own rows — at least 255dc199's 5 (re-pin from a pre-continuation binary if row bytes change; smaller rows only raise it) — # and MORE than the paid near miss at 850 serves (4 rows: there paying is what makes the answer fit), with next= riding # unpaid and over_ceiling="1" + the unpaid clause on the root. RED on dd6e4c8e (4 rows). -( cd "$A7S" && "$BIN" corpus --for="serialize the map" --token-budget=790 --no-cache ) >"$A7S/c3.xml" 2>/dev/null; c3_rc=$? -( cd "$A7S" && "$BIN" corpus --for="serialize the map" --token-budget=850 --no-cache ) >"$A7S/paid.xml" 2>/dev/null; pd_rc=$? +( cd "$A7S" && "$BIN" corpus --for="map row serializer" --token-budget=925 --no-cache ) >"$A7S/c3.xml" 2>/dev/null; c3_rc=$? +( cd "$A7S" && "$BIN" corpus --for="map row serializer" --token-budget=970 --no-cache ) >"$A7S/paid.xml" 2>/dev/null; pd_rc=$? sigs_shown(){ grep -aoE ']*>' "$1" | tail -1 | grep -oE ' shown="[0-9]+"' | tr -dc '0-9'; } c3_sh="$( sigs_shown "$A7S/c3.xml" )"; pd_sh="$( sigs_shown "$A7S/paid.xml" )" c3_root="$( grep -aoE '^]*>' "$A7S/c3.xml" | head -1 )" if [ "$c3_rc" -eq 0 ] && [ -n "$c3_sh" ] && [ -n "$pd_sh" ] && [ "$c3_sh" -ge 5 ] && [ "$c3_sh" -gt "$pd_sh" ] \ && grep -aqE ']* capped="1"[^>]* next="--for=' "$A7S/c3.xml" && [ "${c3_root#* over_ceiling=\"1\"}" != "$c3_root" ] \ && grep -aqF 'next= rides unpaid: no row is dropped to pay for it' "$A7S/c3.xml"; then - ok "#11 A7 C3 @790: over its ceiling either way, the answer keeps the cut's own $c3_sh rows (>= 255dc199's 5, > the paid 850 answer's $pd_sh) — next= unpaid, over_ceiling=\"1\" + the unpaid clause" + ok "#11 A7 C3 @925: over its ceiling either way, the answer keeps the cut's own $c3_sh rows (>= the pre-continuation binary's 5, > the paid 970 answer's $pd_sh) — next= unpaid, over_ceiling=\"1\" + the unpaid clause" else - no "#11 A7 C3 @790 (exit $c3_rc): shown=${c3_sh:-?} (want >= 5 and > the paid 850 answer's ${pd_sh:-?}), next= + over_ceiling=\"1\" + the unpaid clause — rows were dropped to pay for a handle that cannot make the answer fit" + no "#11 A7 C3 @925 (exit $c3_rc): shown=${c3_sh:-?} (want >= 5 and > the paid 970 answer's ${pd_sh:-?}), next= + over_ceiling=\"1\" + the unpaid clause — rows were dropped to pay for a handle that cannot make the answer fit" fi # Its near miss, 850 — the PAYABLE case, unchanged: paying is what makes the answer fit (255dc199: 5 rows, est_tokens=911, # over), so the handle is paid (the block sheds its bytes above the floor) — next= rides inside the allowance, and neither # over_ceiling="1" nor the unpaid clause appears (a label that rode on every capped answer would pass the arms above alone). -pd_b="$( bytes_of "$A7S/paid.xml" )"; pd_a="$( awk 'BEGIN{printf "%d", 850*2.36*1.15}' )" +pd_b="$( bytes_of "$A7S/paid.xml" )"; pd_a="$( awk 'BEGIN{printf "%d", 970*2.36*1.15}' )" pd_root="$( grep -aoE '^]*>' "$A7S/paid.xml" | head -1 )" if [ "$pd_rc" -eq 0 ] && [ "$pd_b" -le "$pd_a" ] && grep -aqE ']* capped="1"[^>]* next="--for=' "$A7S/paid.xml" \ && [ "${pd_root#* over_ceiling=\"1\"}" = "$pd_root" ] && ! grep -aqF 'next= rides unpaid' "$A7S/paid.xml"; then - ok "#11 A7 unpaid twin near miss @850: the capped pays for its next= — $pd_b B <= $pd_a B, no over_ceiling=, no unpaid clause" + ok "#11 A7 unpaid twin near miss @970: the capped pays for its next= — $pd_b B <= $pd_a B, no over_ceiling=, no unpaid clause" else - no "#11 A7 unpaid twin near miss @850 (exit $pd_rc): $pd_b B vs $pd_a B — expected a paid next= inside the allowance with no label" + no "#11 A7 unpaid twin near miss @970 (exit $pd_rc): $pd_b B vs $pd_a B — expected a paid next= inside the allowance with no label" fi +# ── #11 A7R (lego-charge fix round 4): THE TRAILING-SECTION RESERVE. Under --token-budget, --with-graph's block and the +# first --detail body were charged only AFTER had claimed the whole sig-side ceiling, so both rode past the +# budget, and the ladder's last rung then blamed the HEADER floor ("no payload left to trim") while was the +# trimmable payload. MEASURED on the parent binary over this arm's corpus: --detail=20 --with-graph at 2000 tokens = +# 6054 B vs a 5428 B allowance; --detail=3 served its first body as a one-line over_ceiling stub at every budget. +# runForLens now reserves the graph bytes and a MEASURED first-body floor out of the sig claim +# (forSigClaimWithTrailingReserve / forDetailFirstBodyFloor). The corpus is git-less, run from a relative path, every +# body is longer than a quarter of the smallest budget's bundle (so the floor's head-cut + next= path runs), and its +# 40 signature rows overflow the default sig share (so is capped="1" in every regime below — the negatives +# compare a CUT section, not a whole one). +A7R="$TMP/a7reserve" +mkdir -p "$A7R/corpus" +python3 - "$A7R/corpus" <<'PYG' +import os, sys +out = sys.argv[1] +for i in range( 6 ): + lines = [] + for j in range( 12 ): + nxt = f"serializeRow{i}_{j + 1}( map, row )" if j + 1 < 12 else "0" + body = [ f" int acc = map + row + {i * 7 + j};" ] + [ f" acc += flushMapRow( map, row, {k} ) * {i + 1};" for k in range( 30 + j ) ] + lines += [ f"// serializeRow{i}_{j}: serialize one map row into the output buffer the map writer flushes, keeping the row order stable", + f"int serializeRow{i}_{j}( int map, int row, int column, int stride, int width, int height, int depth, int offset, int flags, int mode )", + "{" ] + body + [ f" return acc + {nxt};", "}", "" ] + with open( os.path.join( out, f"mod{i}.cpp" ), "w" ) as fh: + fh.write( "\n".join( lines ) ) +PYG +A7R_Q="serialize the map row" +a7r_out(){ ( cd "$A7R" && "$BIN" corpus --for="$A7R_Q" "$@" --no-cache ) 2>/dev/null; } +a7r_sigs(){ python3 -c 'import re,sys; m=re.search(r"",open(sys.argv[1]).read(),re.S); print(m.group(0) if m else "")' "$1"; } +# A7R-1 (positive): every budget, three shapes — exit 0, inside the allowance, NO over_ceiling on the root (est_tokens fits +# budget_tokens), no ladder last rung, present, and with --detail a first that is not the one-line over_ceiling +# stub; a first body the floor cut carries the next= that serves the rest (rule 5: every cut recoverable). +a7r_bad=""; a7r_runs=0 +for spec in "graph:--with-graph" "detail:--detail=3"; do + r_label="${spec%%:*}"; r_args="${spec#*:}" + for N in 1500 2000 2500 3000; do + # shellcheck disable=SC2086 + a7r_out --token-budget=$N $r_args >"$A7R/o.xml" + r_rc=$? + a7r_runs=$(( a7r_runs + 1 )) + r_b="$( bytes_of "$A7R/o.xml" )" + r_a="$( awk "BEGIN{printf \"%d\", $N*2.36*1.15}" )" + r_root="$( grep -aoE '^]*>' "$A7R/o.xml" | head -1 )" + r_why="" + [ "$r_rc" -eq 0 ] || r_why="exit $r_rc" + [ -n "$r_root" ] && grep -aq '/dev/null || r_why="$r_why ${r_b:-?}B>${r_a}B" + [ "${r_root#* over_ceiling=\"1\"}" = "$r_root" ] || r_why="$r_why over_ceiling" + ! grep -aqF '[over_ceiling= is 1 on the root: ' "$A7R/o.xml" || r_why="$r_why last-rung" # either reason (fix round 5) + if [ "${r_args#*--detail}" != "$r_args" ]; then + r_b1="$( grep -aoE ']*>' "$A7R/o.xml" | head -1 )" + [ -n "$r_b1" ] || r_why="$r_why no-first-body" + [ "${r_b1#* over_ceiling=}" = "$r_b1" ] || r_why="$r_why first-body-stub" + { [ "${r_b1#* truncated=\"1\"}" = "$r_b1" ] || [ "${r_b1#* next=\"--expand=}" != "$r_b1" ]; } || r_why="$r_why cut-without-next" + fi + [ -z "$r_why" ] || a7r_bad="$a7r_bad $r_label@$N:[${r_why# }]" + done +done +[ "$a7r_runs" -eq 8 ] && [ -z "$a7r_bad" ] \ + && ok "#11 A7R reserve: 8 budgeted runs (--with-graph / --detail=3 at 1500..3000) — inside N x 2.36 x 1.15, no over_ceiling, no last rung, first body served (cut ones carry next=)" \ + || no "#11 A7R reserve: spent the bytes of the graph block / first --detail body —$a7r_bad" +# A7R-K (the PRE-REGISTERED KILL, reports/lego-charge.md fix round 4): with BOTH --detail and --with-graph the reserve is +# off — reserving both cut a gold-relevant owner row out of on the measured question set. So that shape's ranked +# rows keep their claim: never FEWER rows than the same budget's --with-graph run (which reserves the graph alone), +# a delivery past the allowance is disclosed (root over_ceiling="1"), and the first body still gets its floor — not the +# one-line over_ceiling stub, and a head-cut carries next=. +a7k_bad="" +for N in 2000 2500 3000; do + a7r_out --token-budget=$N --detail=20 --with-graph >"$A7R/k.xml"; k_rc=$? + a7r_out --token-budget=$N --with-graph >"$A7R/kg.xml" + k_rows="$( grep -aoE ']*>' "$A7R/k.xml" | wc -l | tr -d ' ' )"; g_rows="$( grep -aoE ']*>' "$A7R/kg.xml" | wc -l | tr -d ' ' )" + k_b="$( bytes_of "$A7R/k.xml" )"; k_a="$( awk "BEGIN{printf \"%d\", $N*2.36*1.15}" )" + k_root="$( grep -aoE '^]*>' "$A7R/k.xml" | head -1 )" + k_b1="$( grep -aoE ']*>' "$A7R/k.xml" | head -1 )" + k_why="" + [ "$k_rc" -eq 0 ] && [ -n "$k_root" ] || k_why="exit $k_rc / no root" + [ "${k_rows:-0}" -ge 1 ] && [ "${k_rows:-0}" -ge "${g_rows:-0}" ] 2>/dev/null || k_why="$k_why rows ${k_rows:-?}/dev/null || [ "${k_root#* over_ceiling=\"1\"}" != "$k_root" ] || k_why="$k_why ${k_b:-?}B>${k_a}B-unlabelled" + [ -n "$k_b1" ] && [ "${k_b1#* over_ceiling=}" = "$k_b1" ] || k_why="$k_why first-body-stub" + { [ "${k_b1#* truncated=\"1\"}" = "$k_b1" ] || [ "${k_b1#* next=\"--expand=}" != "$k_b1" ]; } || k_why="$k_why cut-without-next" + [ -z "$k_why" ] || a7k_bad="$a7k_bad @$N:[${k_why# }]" +done +[ -z "$a7k_bad" ] \ + && ok "#11 A7R kill: --detail=20 --with-graph at 2000..3000 keeps at least the --with-graph run's rows, labels any overshoot over_ceiling, serves the first body at its floor" \ + || no "#11 A7R kill: the both-sections shape —$a7k_bad" +# A7R-K2 (fix round 5, owner ruling option B — the A7B contract on a corpus whose first body is LONGER than the floor): +# at 2000 and 2500 tokens --detail=20 --with-graph delivers past the allowance with over_ceiling="1", the last rung names +# the kept rows (never the header floor: the header alone fits), and the head-cut first body's next= RECOVERS EXACTLY the +# lines it cut — the cut CDATA plus the next= answer's CDATA is the function's source, line for line (rule 5). +a7k2_bad="" +for N in 2000 2500; do + a7r_out --token-budget=$N --detail=20 --with-graph >"$A7R/k2.xml"; k2_rc=$? + k2_b="$( bytes_of "$A7R/k2.xml" )"; k2_a="$( awk "BEGIN{printf \"%d\", $N*2.36*1.15}" )" + k2_root="$( grep -aoE '^]*>' "$A7R/k2.xml" | head -1 )" + k2_hdr="$( python3 -c 'import sys; d=open(sys.argv[1],"rb").read(); print(d.find(b"/dev/null \ + || k2_why="$k2_why premise/label:${k2_b:-?}B-vs-${k2_a}B" + { grep -aqF "$KEPT_NOTE" "$A7R/k2.xml" && ! grep -aqF "$FLOOR_NOTE" "$A7R/k2.xml" && [ "${k2_hdr:--1}" -gt 0 ] && [ "$k2_hdr" -le "$k2_a" ]; } 2>/dev/null \ + || k2_why="$k2_why reason" + # fix round 6 (checklist 25): the kept-rows note rides in the header XML comment here too — the answer must parse + if command -v xmllint >/dev/null 2>&1; then + xmllint --noout "$A7R/k2.xml" 2>/dev/null || k2_why="$k2_why xmllint-rejected" + fi + k2_next="$( grep -aoE ']*>' "$A7R/k2.xml" | head -1 | sed -nE 's/.* truncated="1".* next="--expand=([^"]*)".*/\1/p' )" + if [ -z "$k2_next" ]; then + k2_why="$k2_why first-body-not-head-cut-with-next" + else + ( cd "$A7R" && "$BIN" corpus --expand="$k2_next" --no-cache ) >"$A7R/k2n.xml" 2>/dev/null + python3 - "$A7R/k2.xml" "$A7R/k2n.xml" "$A7R/corpus" <<'PYR' || k2_why="$k2_why next=-does-not-recover-the-cut" +import re, sys, os +cut = open( sys.argv[1] ).read(); nxt = open( sys.argv[2] ).read() +m1 = re.search( r']* p="([^"]+)" n="[^"]+" lines="1-(\d+)/(\d+)" truncated="1"[^>]*>', cut, re.S ) +l1 = re.search( r']* l="(\d+)" p="', cut ) +m2 = re.search( r']* lines="(\d+)-(\d+)/(\d+)"[^>]*>', nxt, re.S ) +if not ( m1 and l1 and m2 ): sys.exit( 1 ) +k, t = int( m1.group( 2 ) ), int( m1.group( 3 ) ) +if ( int( m2.group( 1 ) ), int( m2.group( 2 ) ), int( m2.group( 3 ) ) ) != ( k + 1, t, t ): sys.exit( 1 ) +src = open( os.path.join( sys.argv[3], m1.group( 1 ) ) ).read().split( "\n" ) +start = int( l1.group( 1 ) ) - 1 +want = "\n".join( src[ start : start + t ] ) +got = m1.group( 4 ).rstrip( "\n" ) + "\n" + m2.group( 4 ) +sys.exit( 0 if got.rstrip( "\n" ) == want.rstrip( "\n" ) else 1 ) +PYR + fi + [ -z "$k2_why" ] || a7k2_bad="$a7k2_bad @$N:[${k2_why# }]" +done +[ -z "$a7k2_bad" ] \ + && ok "#11 A7R kill+next (by-design, owner option B): --detail=20 --with-graph at 2000/2500 is over_ceiling with the kept-rows reason, and the cut first body's next= returns exactly the lines it cut" \ + || no "#11 A7R kill+next (by-design, owner option B): —$a7k2_bad" +# A7R-2 (the floor's cut is the one the floor priced): at 1500 tokens the first body is longer than the floor, so it ships +# head-cut — truncated="1" with lines= and next= — never dropped and never the bare one-line stub. +a7r_out --token-budget=1500 --detail=3 >"$A7R/f.xml" +r_b1="$( grep -aoE ']*>' "$A7R/f.xml" | head -1 )" +r_hi="$( printf '%s' "$r_b1" | sed -nE 's/.* lines="1-([0-9]+)\/[0-9]+".*/\1/p' )" +{ [ "${r_b1#* truncated=\"1\"}" != "$r_b1" ] && [ "${r_b1#* over_ceiling=}" = "$r_b1" ] && [ "${r_hi:-0}" -ge 2 ] 2>/dev/null \ + && [ "${r_b1#* next=\"--expand=}" != "$r_b1" ]; } \ + && ok "#11 A7R floor: the first --detail body at 1500 tokens is head-cut at the floor with lines= and next= ($r_b1)" \ + || no "#11 A7R floor: the first --detail body at 1500 tokens is not the floor's head-cut of >= 2 lines (got '${r_b1:-none}')" +# A7R-N1 (near-miss negative, NO explicit ceiling): the reserve must not exist — is byte-identical with and +# without --with-graph (the default regime's graph rides on top, as before). +a7r_out >"$A7R/n0.xml"; a7r_out --with-graph >"$A7R/n1.xml" +n0="$( a7r_sigs "$A7R/n0.xml" )"; n1="$( a7r_sigs "$A7R/n1.xml" )" +{ [ -n "$n0" ] && [ "$n0" = "$n1" ] && [ "${n0#*capped=\"1\"}" != "$n0" ]; } \ + && ok "#11 A7R no ceiling: (capped=1) byte-identical with and without --with-graph — nothing reserved without --token-budget" \ + || no "#11 A7R no ceiling: differs with --with-graph and no --token-budget (or the arm's is not cut) — the reserve leaked into the default regime" +# A7R-N2 (near-miss negative, a WIDE explicit ceiling in the auto bundle): the sig side is already frozen at the default +# share (forSigSideCeiling) and the bundle holds that share AND the graph, so the reserve does not bind — +# byte-identical with and without --with-graph at 8000 tokens (forbudgetmonotoncheck's invariant, seen from this side). +a7r_out --token-budget=8000 >"$A7R/w0.xml"; a7r_out --token-budget=8000 --with-graph >"$A7R/w1.xml" +w0="$( a7r_sigs "$A7R/w0.xml" )"; w1="$( a7r_sigs "$A7R/w1.xml" )" +{ [ -n "$w0" ] && [ "$w0" = "$w1" ] && [ "${w0#*capped=\"1\"}" != "$w0" ]; } \ + && ok "#11 A7R wide ceiling: (capped=1) byte-identical with and without --with-graph at --token-budget=8000 — the reserve binds only when the bundle cannot hold both" \ + || no "#11 A7R wide ceiling: differs with --with-graph at --token-budget=8000 (or is not cut) — the reserve shrank a sig share that had room" + +# ── #11 A7D (fix round 4, review rv-wrap-recipe-k29 F1; rule 5 — never a silent cut): under --token-budget the compact +# section is no longer DROPPED with no element when the ceiling is spent. The arm corpus is on the conceptual route. +# A7D-1: at 2000 tokens the section is the counted marker — shown="0" total=N capped="1" and next= the same question with +# no budget — its legend clause rides the header, the answer stays inside its budget (the disclosure's bytes are +# reserved), and the next= RECOVERS the rows: that call's serves at least one row of the same total. +a7r_out --token-budget=2000 >"$A7R/h.xml" +h_tag="$( grep -aoE ']*>' "$A7R/h.xml" | head -1 )" +h_root="$( grep -aoE '^]*>' "$A7R/h.xml" | head -1 )" +h_tot="$( printf '%s' "$h_tag" | sed -nE 's/.* total="([0-9]+)".*/\1/p' )" +a7r_out >"$A7R/hd.xml" +hd_tag="$( grep -aoE ']*>' "$A7R/hd.xml" | head -1 )" +hd_shown="$( printf '%s' "$hd_tag" | sed -nE 's/" ] && [ "${h_tot:-0}" -ge 1 ] 2>/dev/null \ + && grep -aqF 'hops shown=0 capped=1: this budget left out the call-hop rows' "$A7R/h.xml" && [ "${h_root#* over_ceiling=}" = "$h_root" ] \ + && [ "${hd_shown:-0}" -ge 1 ] 2>/dev/null && [ "$hd_tot" = "$h_tot" ]; } \ + && ok "#11 A7D hops: the spent-ceiling is the counted marker ($h_tag), defined, inside budget; its next= serves $hd_shown of $hd_tot rows" \ + || no "#11 A7D hops: spent-ceiling not the counted, continued marker (got '${h_tag:-none}', next= answer '${hd_tag:-none}', root '${h_root:0:80}…')" +# A7D-2: a PARTIAL hops cut under the budget carries the same next= (5000 tokens: some rows fit, not all). +a7r_out --token-budget=5000 >"$A7R/hp.xml" +hp_tag="$( grep -aoE ']*>' "$A7R/hp.xml" | head -1 )" +{ [ "${hp_tag#*shown=\"0\"}" = "$hp_tag" ] && [ "${hp_tag#* capped=\"1\"}" != "$hp_tag" ] && [ "${hp_tag#* next=\"--for='$A7R_Q'\"}" != "$hp_tag" ]; } \ + && ok "#11 A7D partial hops cut at 5000 tokens carries next= ($hp_tag)" \ + || no "#11 A7D partial hops cut at 5000 tokens: no next= on a cut with rows (got '${hp_tag:-none}')" +# A7D-N (near-miss negative): WITHOUT --token-budget the default regime is unchanged — its cut (capped="1") gets no +# next= from this change (a named deferral: adding it there moves every default --for answer). +{ [ "${hd_tag#* capped=\"1\"}" != "$hd_tag" ] && [ "${hd_tag#* next=}" = "$hd_tag" ]; } \ + && ok "#11 A7D no budget: the default regime's cut is byte-unchanged (capped=1, no next= added): $hd_tag" \ + || no "#11 A7D no budget: the default changed or is not cut (got '${hd_tag:-none}') — the disclosure leaked into the default regime" + # A9/A10 — the header's own spliced attributes are inside the number. IDENTITY, not a band: for a bundle with # no --detail bodies, est_tokens is markup-only, so it must equal round(delivered bytes / 2.50) EXACTLY # (kBytesPerTokenDefault). Pre-fix the est_tokens attribute (~19 B) and weak="1" (9 B) sat outside the sum, diff --git a/test/expandbodyfirstcheck.sh b/test/expandbodyfirstcheck.sh index 8c39d11fc..c382ed4a4 100755 --- a/test/expandbodyfirstcheck.sh +++ b/test/expandbodyfirstcheck.sh @@ -36,13 +36,17 @@ TMP="$( mktemp -d )"; trap 'rm -rf "$TMP"' EXIT # L1 (2026-09-19): the CLI default legend is compact and spells / inside its comment; these arms measure the # byte offsets and counts of the REAL elements, so the three runs ask for the full legend. -"$BIN" "$FIX" --expand=dupTarget --pack-budget-bytes=10 --no-cache --legend=full >"$TMP/dup.xml" 2>"$TMP/dup.err" +# expand-lean-k64 (2026-10-08): an --expand whose definitions are all served no longer carries a map by itself (see +# expandtopk0check (H)); the map still rides — body-first reorder, note=, stderr note — wherever a composed payload owns +# it. --outline beside --expand is that shape, so this gate's probe composes the two. The ride-along mechanics under test +# are unchanged; only the way to reach them moved. +"$BIN" "$FIX" --expand=dupTarget --outline=uniqueTarget --pack-budget-bytes=10 --no-cache --legend=full >"$TMP/dup.xml" 2>"$TMP/dup.err" -# ── sanity: the fixture really landed in bundle mode (map+bodies both ride) — otherwise every arm below -# proves nothing (identical to expandtopk0check.sh (B)'s own sanity arm, same fixture, same forcing flag). -grep -q 'mode="bundle"' "$TMP/dup.xml" \ - && ok "(A) sanity: the ambiguous probe landed in bundle mode (map+bodies both ride)" \ - || no "(A) sanity failed: --pack-budget-bytes=10 did not force bundle mode — arm proves nothing" +# ── sanity: the map really rides beside the bodies (a composed --outline owns it) — otherwise every arm below +# proves nothing. +grep -q ' starts before the ranked map's , or no map rides at all. ─────────── bodiesOff="$( grep -bo '"$TMP/ambig.xml" 2>/dev/null ambTotal="$( wc -c <"$TMP/ambig.xml" | tr -d ' ' )" amb_num="$( reason_num "$TMP/ambig.xml" bundle )" -if ! grep -q '"$TMP/ambig5.xml" 2>/dev/null +if grep -q ']*>' "$TMP/ambig5.xml" )" fi [ "$fail" = 0 ] && echo "ALL PASS" || echo "FAILURES ABOVE" diff --git a/test/expandtopk0check.sh b/test/expandtopk0check.sh index 6cb00e2b1..70216185e 100755 --- a/test/expandtopk0check.sh +++ b/test/expandtopk0check.sh @@ -58,27 +58,33 @@ else ok "(A) no ride-along stderr note (nothing is riding along to warn about)" fi -# ── (B) ambiguous name (2 defs), no --top-k: the ORDINARY default applies — map rides, note fires ──────── -# --pack-budget-bytes=10 forces M6's whole-file candidate over budget (dupA.c+dupB.c together are well -# over 10 B), so chooseExpandServe picks mode="bundle" deterministically regardless of this tiny fixture's -# absolute byte counts — the note only ever fires in bundle mode ("never in whole-file mode" is the -# pre-existing, correct rule; this arm needs bundle mode to observe it at all). +# ── (B) ambiguous name (2 defs), the ORDINARY default where the lean default does not apply ──────────────── +# expand-lean-k64 (2026-10-08): a multi-definition name whose definitions are ALL served now drops the map too — see (H) +# below. This TWIN pins the path that stays: wherever the lean default is excluded the caller's ordinary default applies, the +# ranked map rides along and the pre-existing "ranked top-N map rides along" stderr note still fires, with NO topk_default=. +# --max-tokens=3000 is such a shape (it sizes the map, so the exact-name/lean defaults both stand aside); the explicit +# --top-k=5 twin right below pins the other half (the caller asked for the map). "$BIN" "$FIX" --expand=dupTarget --top-k=5 --no-cache >"$TMP/dup5.xml" 2>/dev/null grep -q '"$TMP/dup.xml" 2>"$TMP/dup.err" -grep -q 'mode="bundle"' "$TMP/dup.xml" \ - && ok "(B) sanity: the ambiguous probe landed in bundle mode (the note's only firing ground)" \ - || no "(B) sanity failed: --pack-budget-bytes=10 did not force bundle mode — arm proves nothing" -if grep -q 'topk_default="0"' "$TMP/dup.xml"; then - no "(B) ambiguous --expand (2 matches) wrongly got the exact-name default" +if grep -qE 'topk_default=|map_next=' "$TMP/dup5.xml"; then + no "(B) explicit --top-k=5 on a multi-def name carries lean-default decoration — the caller chose, not the tool" else - ok "(B) ambiguous --expand carries NO topk_default= — the multi-match case is untouched" + ok "(B) explicit --top-k=5 on a multi-def name keeps the classic shape (no topk_default=, no map_next=)" +fi +"$BIN" "$FIX" --expand=dupTarget --max-tokens=3000 --pack-budget-bytes=10 --no-cache >"$TMP/dup.xml" 2>"$TMP/dup.err" +grep -q '"$TMP/tk5.xml" 2>/dev/null @@ -180,9 +186,240 @@ else no "(G-b) no reason=\"bundle NNNB ...\" clause found on the real-repo default root — unexpected mode, see (G) above: $( grep -oE ']*>' "$TMP/real_default.xml" )" fi +# ── (H) expand-lean-k64: a multi-definition name, EVERY definition served — no ride-along map ───────────────────────── +# `subtokens` has three definitions (one C++ header, two Python fixtures) and the default budget serves all of them in +# bundle mode (the header is far over the pack budget as a whole file). The root discloses the default (topk_default="0"), +# carries the recoverable pointer to the map it did not ship (map_next=), the bodies keep p= l= for each def, and no / +# rows ride. (The tiny dupTarget fixture serves whole-file instead — (J2) pins that.) +"$BIN" "$ROOT" --expand=subtokens --no-cache --legend=full >"$TMP/mdef.xml" 2>"$TMP/mdef.err" +grep -q 'topk_default="0"' "$TMP/mdef.xml" && grep -q 'map_next="ripwire ' "$TMP/mdef.xml" \ + && ok "(H) multi-def, all served: root carries topk_default=\"0\" and map_next=" \ + || no "(H) multi-def default lacks topk_default=/map_next=: $( grep -oE ']*>' "$TMP/mdef.xml" )" +if grep -qE '"$TMP/mapnext.xml" 2>/dev/null + grep -q '"$TMP/mdef_tk.xml" 2>/dev/null +bodiesOf(){ perl -0777 -ne 'print $1 if /()/s' "$1"; } +[ -n "$( bodiesOf "$TMP/mdef.xml" )" ] && [ "$( bodiesOf "$TMP/mdef.xml" )" = "$( bodiesOf "$TMP/mdef_tk.xml" )" ] \ + && ok "(H2) the lean answer's are byte-identical to the explicit --top-k=3 twin's" \ + || no "(H2) the lean answer's differ from the explicit twin's (every body must still be served)" +nb=$( grep -oE ', shown==total)" \ + || no "(H2) the lean answer serves $nb located , expected >= 3 with shown==total" + +# ── (I) bodies CUT: a symbol-scoped aid (the unserved defs), never the generic map ───────────────────────────────── +"$BIN" "$FIX" --expand=dupTarget --pack-budget-bytes=10 --no-cache >"$TMP/cut.xml" 2>/dev/null +if grep -qE ']*>' "$TMP/cut.xml" )" +unNext=$( grep -oE 'unserved_next="[^"]*"' "$TMP/cut.xml" | head -1 | sed 's/^unserved_next="//;s/"$//' ) +case "$unNext" in + --expand=*dupB.c:*:dupTarget) + if grep -q 'dupB.c:2:dupTarget' <<<"$unNext"; then + "$BIN" "$FIX" "$unNext" --no-cache >"$TMP/cut2.xml" 2>/dev/null + grep -q 'return 2' "$TMP/cut2.xml" \ + && ok "(I) unserved_next serves exactly the cut definition (dupB.c's body)" \ + || no "(I) unserved_next did not serve dupB.c's body" + grep -q 'return 1' "$TMP/cut2.xml" \ + && no "(I) unserved_next re-served the definition that was already served" \ + || ok "(I) unserved_next names only the unserved def (dupA.c's body is not re-served)" + else + no "(I) unserved_next names the wrong line: $unNext" + fi ;; + *) no "(I) unserved_next missing or names the wrong def: '$unNext'" ;; +esac +grep -q ']*p="dupA.c"' "$TMP/cut.xml" \ + && ok "(I) the served def (dupA.c) is still in the cut answer" || no "(I) the first def is missing from the cut answer" + +# ── (J) the whole-file comparison prices the PAYLOAD, not a map ───────────────────────────────────────────────────── +# (J1) real repo: emitTo (2 overloads, one file) used to serve the whole 24 KB file because the bundle was priced with the +# ~19 KB map. Now bundle mode, the priced bundle is the served document, and the document is far under the file. +"$BIN" "$ROOT" --expand=emitTo --no-cache --legend=full >"$TMP/emit.xml" 2>/dev/null +if grep -q 'mode="bundle"' "$TMP/emit.xml"; then + pb=$( grep -oE 'reason="bundle [0-9]+B' "$TMP/emit.xml" | grep -oE '[0-9]+' ) + sz=$( wc -c < "$TMP/emit.xml" | tr -d ' ' ) + [ -n "$pb" ] && [ "$pb" = "$sz" ] && [ "$sz" -lt 12000 ] \ + && ok "(J1) emitTo: bundle mode, priced bundle ${pb}B == served ${sz}B (< 12000B; was the whole 24788B file)" \ + || no "(J1) emitTo bundle priced '${pb}' vs served ${sz}B" +else + no "(J1) emitTo is not in bundle mode: $( grep -oE ']*>' "$TMP/emit.xml" )" +fi +# (J2) near-miss: where the file really is cheaper (tiny fixture) whole-file STILL wins, with the lean disclosure intact +"$BIN" "$FIX" --expand=dupTarget --no-cache --legend=full >"$TMP/small.xml" 2>/dev/null +grep -q 'mode="whole-file"' "$TMP/small.xml" && grep -q 'topk_default="0"' "$TMP/small.xml" && ! grep -q ']*>' "$TMP/small.xml" )" +if grep -q 'unserved_' "$TMP/small.xml"; then + no "(J2) whole-file serving names unserved defs though the file carries every one" +else + ok "(J2) whole-file serving carries no unserved_* (the file has every definition)" +fi + +# ── (K) --outline follows the same rule ────────────────────────────────────────────────────────────────────────────── +"$BIN" "$FIX" --outline=dupTarget --no-cache >"$TMP/ol.xml" 2>/dev/null +if grep -qE '"$TMP/ol5.xml" 2>/dev/null +grep -q '"$TMP/olcut.xml" 2>/dev/null +grep -q 'unserved_total=' "$TMP/olcut.xml" && grep -q 'unserved_next="--outline=' "$TMP/olcut.xml" && ! grep -q ']*>' "$TMP/olcut.xml" )" + +# ── (L) near-miss negatives: the lean default must NOT apply where the caller or a composed verb owns the map ───────── +"$BIN" "$FIX" --expand=dupTarget --top-k=0 --no-cache >"$TMP/mtk0.xml" 2>/dev/null +if grep -qE 'topk_default=|map_next=|mode="' "$TMP/mtk0.xml"; then + no "(L) explicit --top-k=0 on a multi-def name picked up lean decoration — must stay the undecorated lean form" +else + ok "(L) explicit --top-k=0 on a multi-def name stays undecorated" +fi +"$BIN" "$FIX" --expand=dupTarget --pack-top-n=1 --no-cache >"$TMP/mpack.xml" 2>/dev/null +if grep -qE 'topk_default=|map_next=' "$TMP/mpack.xml"; then + no "(L) --expand composed with --pack-top-n wrongly got the lean default (the composed verb owns the map)" +else + ok "(L) --expand composed with --pack-top-n keeps its own shape (no lean decoration)" +fi +"$BIN" "$FIX" --expand=uniqueTarget,dupTarget --pack-budget-bytes=1000000 --no-cache >"$TMP/mtok.xml" 2>/dev/null +if grep -q ']*>' "$TMP/mtok.xml" )" +else + ok "(L) a multi-token --expand with every def served is lean too" +fi +"$BIN" "$FIX" --expand=uniqueTarget --no-cache >"$TMP/u1.xml" 2>/dev/null +if grep -q 'map_next=' "$TMP/u1.xml"; then + no "(L) the single-definition exact-name answer changed shape (map_next= appeared)" +else + ok "(L) the single-definition exact-name answer is unchanged (topk_default only, no map_next=)" +fi + +# ── (M) fix round 1: map_next= replays EVERY root and the crawl-scope flags, shell-quoted (nextFlag) ───────────────── +# A multi-root call's map is the map of all its roots; the pointer used to fall back to "ripwire ." (the parent tree), and +# it dropped --ignore-tests/--exclude, so pasting it printed a map of a different corpus. Exact pointers, then pasted. +xmlUnesc(){ sed -e 's/'/'"'"'/g' -e 's/"/"/g' -e 's/<//g' -e 's/&/\&/g'; } +attrOf(){ grep -oE "$1=\"[^\"]*\"" "$2" | head -1 | sed "s/^$1=\"//;s/\"\$//"; } +MR="$TMP/mr"; mkdir -p "$MR/r1" "$MR/r2" +printf 'int multiWork(void) { return 1; }\n' >"$MR/r1/w.c" +printf 'int multiWork(void) { return 2; }\n' >"$MR/r2/w.c" +( cd "$MR" && "$BIN" r1 r2 --expand=multiWork --no-cache ) >"$TMP/mroot.xml" 2>/dev/null +mrNext="$( attrOf map_next "$TMP/mroot.xml" )" +[ "$mrNext" = "ripwire r1 r2" ] \ + && ok "(M) multi-root: map_next names BOTH roots (\"$mrNext\")" \ + || no "(M) multi-root: map_next='$mrNext', want 'ripwire r1 r2': $( grep -oE ']*>' "$TMP/mroot.xml" | head -1 )" +grep -q 'return 1' "$TMP/mroot.xml" && grep -q 'return 2' "$TMP/mroot.xml" \ + && ok "(M) multi-root: both definitions are served (premise of the lean answer)" \ + || no "(M) multi-root: the two definitions are not both served" +( cd "$MR" && "$BIN" r1 r2 --no-cache --top-k=3 ) >"$TMP/mroot_map.xml" 2>/dev/null +( cd "$MR" && eval "\"\$BIN\" ${mrNext#ripwire } --no-cache --top-k=3" ) >"$TMP/mroot_paste.xml" 2>/dev/null +grep -q '"$SC/fix/tests/test_dup.c" +( cd "$SC" && "$BIN" fix --expand=dupTarget --ignore-tests '--exclude=nomatch*' --no-cache ) >"$TMP/scope.xml" 2>/dev/null +scNext="$( attrOf map_next "$TMP/scope.xml" )" +[ "$scNext" = "ripwire fix --exclude='nomatch*' --ignore-tests" ] \ + && ok "(M) crawl scope: map_next replays --exclude (quoted) and --ignore-tests" \ + || no "(M) crawl scope: map_next='$scNext', want the root plus --exclude='nomatch*' --ignore-tests" +scShell="$( printf '%s' "$scNext" | xmlUnesc )" +( cd "$SC" && eval "\"\$BIN\" ${scShell#ripwire } --no-cache --top-k=50" ) >"$TMP/scope_paste.xml" 2>/dev/null +( cd "$SC" && "$BIN" fix --ignore-tests '--exclude=nomatch*' --no-cache --top-k=50 ) >"$TMP/scope_map.xml" 2>/dev/null +( cd "$SC" && "$BIN" fix --no-cache --top-k=50 ) >"$TMP/scope_wide.xml" 2>/dev/null +cmp -s "$TMP/scope_map.xml" "$TMP/scope_wide.xml" \ + && no "(M) crawl scope premise: --ignore-tests does not change this tree's map — the paste arm would prove nothing" \ + || ok "(M) crawl scope premise: --ignore-tests changes this tree's map" +grep -q '"$TMP/plainptr.xml" 2>/dev/null +[ "$( attrOf map_next "$TMP/plainptr.xml" )" = "ripwire test/expandtopk0fix" ] \ + && ok "(M) negative: with no crawl-scope flag map_next is just the root" \ + || no "(M) negative: plain map_next='$( attrOf map_next "$TMP/plainptr.xml" )', want 'ripwire test/expandtopk0fix'" + +# ── (N) fix round 1: unserved_next= is shell-quoted, and past 16 cut definitions it says how many it lists ────────── +# Each file under "sub dir/" holds a 62-line (~2.8 KB) `def work`; a 3500 B pack budget serves one. With 4 files the 3 cut ones are +# all listed (no unserved_listed=); with 20 files the 19 cut ones exceed the 16-selector guard: unserved_listed="16", the +# paste (default budget: 16 bodies fit) serves exactly those 16, and the same call with a larger budget serves all 20 (the +# legend's "the rest"). Served = rows in bundle mode or blocks in whole-file mode, whichever the call picks. +rootOf(){ grep -oE ']*>' "$1" | head -1; } +mkwork(){ local d="$1" n="$2" i j; mkdir -p "$d/sub dir" + for i in $( seq -w 1 "$n" ); do + { echo "def work():"; for j in $( seq 1 61 ); do echo " value_$j = 'padding to about forty-five bytes $j'"; done; } >"$d/sub dir/m$i.py" + done; } +selectorsOf(){ printf '%s' "$1" | xmlUnesc | sed -e "s/^--expand=//" -e "s/^'//" -e "s/'\$//" | tr ',' '\n' | sort; } +servedOf(){ grep -oE '"$TMP/sp4.xml" 2>/dev/null +un4="$( attrOf unserved_next "$TMP/sp4.xml" )" +grep -q 'unserved_total="3"' "$TMP/sp4.xml" && case "$un4" in "--expand='sub dir/"*) true ;; *) false ;; esac \ + && ok "(N) space path: unserved_total=\"3\" and unserved_next is single-quoted for the shell" \ + || no "(N) space path: $( grep -oE ']*>' "$TMP/sp4.xml" | head -1 )" +rootOf "$TMP/sp4.xml" | grep -q 'unserved_listed=' \ + && no "(N) negative: unserved_listed= appeared though all 3 cut definitions are listed" \ + || ok "(N) negative: no unserved_listed= when every cut definition is listed" +( cd "$SP4" && eval "\"\$BIN\" . --no-cache $( printf '%s' "$un4" | xmlUnesc )" ) >"$TMP/sp4_paste.xml" 2>/dev/null +[ "$( selectorsOf "$un4" | wc -l | tr -d ' ' )" = 3 ] && [ "$( servedAnyOf "$TMP/sp4_paste.xml" )" = "$( selectorsOf "$un4" )" ] \ + && ok "(N) space path: the pasted unserved_next serves exactly the 3 cut definitions" \ + || no "(N) space path: pasted unserved_next served [$( servedAnyOf "$TMP/sp4_paste.xml" | tr '\n' ' ' )], listed [$( selectorsOf "$un4" | tr '\n' ' ' )]" +SP20="$TMP/sp20"; mkwork "$SP20" 20 +( cd "$SP20" && "$BIN" . --expand=work --pack-budget-bytes=3500 --no-cache ) >"$TMP/sp20.xml" 2>/dev/null +tot20="$( attrOf unserved_total "$TMP/sp20.xml" )"; un20="$( attrOf unserved_next "$TMP/sp20.xml" )" +served20="$( servedOf "$TMP/sp20.xml" | wc -l | tr -d ' ' )" +[ -n "$tot20" ] && [ "$tot20" -gt 16 ] && [ "$(( tot20 + served20 ))" = 20 ] && rootOf "$TMP/sp20.xml" | grep -q 'unserved_listed="16"' \ + && ok "(N) >16 cut: unserved_total=\"$tot20\" (+$served20 served = 20) and unserved_listed=\"16\"" \ + || no "(N) >16 cut: total='$tot20' served=$served20: $( grep -oE ']*>' "$TMP/sp20.xml" | head -1 | cut -c1-300 )" +( cd "$SP20" && eval "\"\$BIN\" . --no-cache $( printf '%s' "$un20" | xmlUnesc )" ) >"$TMP/sp20_paste.xml" 2>/dev/null +[ "$( selectorsOf "$un20" | wc -l | tr -d ' ' )" = 16 ] && [ "$( servedAnyOf "$TMP/sp20_paste.xml" )" = "$( selectorsOf "$un20" )" ] \ + && rootOf "$TMP/sp20_paste.xml" | grep -q 'schema=' && ! rootOf "$TMP/sp20_paste.xml" | grep -q 'unserved_' \ + && ok "(N) >16 cut: the pasted unserved_next serves exactly the 16 it lists, and nothing is cut in that answer" \ + || no "(N) >16 cut: pasted unserved_next served $( servedAnyOf "$TMP/sp20_paste.xml" | wc -l | tr -d ' ' ): $( rootOf "$TMP/sp20_paste.xml" | cut -c1-200 )" +( cd "$SP20" && "$BIN" . --expand=work --pack-budget-bytes=1000000 --no-cache ) >"$TMP/sp20_all.xml" 2>/dev/null # (deprecated knob, still the one that cut) +[ "$( servedAnyOf "$TMP/sp20_all.xml" | wc -l | tr -d ' ' )" = 20 ] && rootOf "$TMP/sp20_all.xml" | grep -q 'schema=' && ! rootOf "$TMP/sp20_all.xml" | grep -q 'unserved_' \ + && [ -z "$( comm -23 <( servedOf "$TMP/sp20_paste.xml" ) <( servedAnyOf "$TMP/sp20_all.xml" ) )" ] \ + && ok "(N) >16 cut: the rest are recoverable — a larger --pack-budget-bytes serves all 20, no unserved_*" \ + || no "(N) >16 cut: the larger-budget call did not serve all 20: $( grep -oE ']*>' "$TMP/sp20_all.xml" | head -1 | cut -c1-200 )" + +# ── (O) fix round 1 (review N1): several --outline tokens keep the ordinary map (the lean rule is for ONE token) ───── +"$BIN" "$FIX" --outline=uniqueTarget,dupTarget --no-cache >"$TMP/ol2.xml" 2>/dev/null +grep -q ']*>' "$TMP/ol2.xml" | head -1 )" + # ── (F) well-formedness + determinism ───────────────────────────────────────────────────────────────── if command -v xmllint >/dev/null 2>&1; then - for f in uniq dup5 dup tk5 tk0 big real_default real_tk0; do + for f in uniq dup5 dup tk5 tk0 big real_default real_tk0 mdef cut ol olcut emit mroot scope sp4 sp20 sp20_paste ol2; do if xmllint --noout "$TMP/$f.xml" 2>/dev/null; then ok "(F) $f.xml well-formed"; else no "(F) $f.xml fails xmllint"; fi done else diff --git a/test/fixedbufsweep.sh b/test/fixedbufsweep.sh index 3e760a6bd..e4f04c655 100755 --- a/test/fixedbufsweep.sh +++ b/test/fixedbufsweep.sh @@ -254,7 +254,7 @@ NUMERIC_ONLY = { ( "src/partition.h", "nb" ): 1, ( "src/quality.h", "b" ): 1, ( "src/quality.h", "cidHex" ): 1, - ( "src/quality.h", "hex" ): 7, + ( "src/quality.h", "hex" ): 6, # cli-perf-068: blobShardHex and ackMapKey now share hex16 (one buffer for two) ( "src/quality.h", "name" ): 1, ( "src/recall.h", "scoreText" ): 1, ( "src/serialize.h", "...)" ): 1, @@ -617,7 +617,10 @@ if not bad: # lines NAME rw::formatTo without calling it: the format-arity family table and two comments in # src/defectshape.h, the defect-shape legend sentence (verbs_quality.h) and the --help paragraph (cli.h). # train 26b (cr-qd-kinds merged onto the train's 358/248/248/109): re-derived on the merged tree. -EXPECTED = { "mentions": 363, "calls": 248, "sites": 248, "rows": 109, "widthforms": 0 } +# 2026-10-08 (lane/cli-perf-068): -1 call/-1 mention/-1 site, rows unchanged (the lane's own delta on its base: +# 351/241/241/105 -> 350/240/240/105). +# train 26c (cli-perf merged onto the train's 363/248/248/109): re-derived on the merged tree. +EXPECTED = { "mentions": 362, "calls": 247, "sites": 247, "rows": 109, "widthforms": 0 } # 2026-09-30 (#325 ruby_bases_unscoped=): +2 calls/+2 mentions/+2 sites, rows unchanged — graphGaugeAttrXml/Json (graphlegend.h) each format the absent-at-zero Ruby gauge into the SAME local buf[160]: one size_t, no string argument (21 + 20 digits worst case). # 2026-09-04 (capture-audit L6, H9): +1 call/+1 mention, sites/rows UNCHANGED — re-read, not # re-counted. packConnect gained ONE snprintf into a new `char connectCeiling[32]` for the diff --git a/test/fordisclosurecheck.sh b/test/fordisclosurecheck.sh index 537b01889..5893c5821 100755 --- a/test/fordisclosurecheck.sh +++ b/test/fordisclosurecheck.sh @@ -98,9 +98,35 @@ CT="$TMP/c400" grep -q 'bundle="compact" bodies="0" reason="budget"' "$CT" \ && ok "#1b compact route, ceiling exhausted: bodies=\"0\" reason=\"budget\" disclosed" \ || no "#1b compact route, ceiling exhausted: no reason=\"budget\" disclosure (the silent shape, compact copy)" -grep -q ' element leaked past a spent ceiling" \ - || ok "#1b compact route, ceiling exhausted: no element (the attribute IS the disclosure there)" +# BY-DESIGN CHANGE (lego-charge fix round 4, review rv-wrap-recipe-k29 F1, rule 5 "never a silent cut"; reported under +# PROCESS rule 4): this arm used to require NO element at all, which made the cut silent — the hop rows vanished +# with nothing naming them. What it protected still holds and is asserted: no hop ROW () and no hops section legend +# leaks past a spent ceiling. The one allowed is the counted marker: empty, shown="0", capped="1", next= the same +# question without the budget. +ct_hops="$( grep -oE ']*>' "$CT" | head -1 )" +if [ -z "$ct_hops" ]; then + no "#1b compact route, ceiling exhausted: no element — the cut hop rows are silent (rule 5)" +elif printf '%s' "$ct_hops" | grep -qE "^\$" && ! grep -q ' is the counted marker only, no hop row leaked ($ct_hops)" +else + no "#1b compact route, ceiling exhausted: a other than the empty counted marker leaked past a spent ceiling ($ct_hops)" +fi +# #1c (fix round 4): the cut file tail under the same spent ceiling carries the call that lists the files it left out — +# the --for file page — and that call answers (rc 0, a page holding at least the tail's total). Negative: the +# default regime's tail (no --token-budget) is unchanged — no next= (a named deferral). +ct_tail="$( grep -oE ']*>' "$CT" | head -1 )" +ct_ttot="$( printf '%s' "$ct_tail" | sed -nE 's/.* total="([0-9]+)".*/\1/p' )" +"$BIN" src --for="$CONCTASK" --limit=40 --no-cache >"$TMP/cpage" 2>/dev/null; cp_rc=$? +cp_tot="$( grep -oE '^]*>' "$TMP/cpage" | sed -nE 's/.* total="([0-9]+)".*/\1/p' )" +{ printf '%s' "$ct_tail" | grep -qE "capped=\"1\" next=\"--for='$CONCTASK' --limit=40\">\$" && [ "$cp_rc" -eq 0 ] \ + && [ -n "$cp_tot" ] && [ -n "$ct_ttot" ] && [ "$cp_tot" -ge "$ct_ttot" ] 2>/dev/null; } \ + && ok "#1c compact route, ceiling exhausted: the cut names the file page (next=), which pages $cp_tot files >= the tail's $ct_ttot" \ + || no "#1c compact route, ceiling exhausted: the cut has no working next= (tail '$ct_tail', page rc=$cp_rc total='${cp_tot:-none}')" +"$BIN" src --for="$CONCTASK" --no-cache >"$TMP/cdef" 2>/dev/null +cd_tail="$( grep -oE ']*>' "$TMP/cdef" | head -1 )" +{ [ -n "$cd_tail" ] && [ "${cd_tail#* next=}" = "$cd_tail" ]; } \ + && ok "#1c default regime: unchanged, no next= added ($cd_tail)" \ + || no "#1c default regime: changed or missing ('$cd_tail') — the explicit-budget next= leaked into the default" grep -q 'bundle=compact:' "$CT" \ && no "#1b compact route, ceiling exhausted: the legend leaked past a spent ceiling (no reserve there)" \ || ok "#1b compact route, ceiling exhausted: legend dropped (only the attribute's reserved bytes are spent)" diff --git a/test/fornotesbudgetcheck.sh b/test/fornotesbudgetcheck.sh index d0697b819..f4f5c8e31 100755 --- a/test/fornotesbudgetcheck.sh +++ b/test/fornotesbudgetcheck.sh @@ -231,7 +231,12 @@ jsonRows(){ "$BIN" "$CORPUS" --for="$TASK" --token-budget="$1" --json 2>/dev/nul # ── arm 1: est_tokens must fit the ceiling the user asked for, in BOTH dialects ──────────────────── # (tight budget 1100, re-anchored 2026-09-11 — see the CEILING MARGIN block above for the arithmetic) -for tb in 1100 1640 3000; do +# RE-ANCHORED 2026-10-10 (train 26c): the tight rung 1100 -> 1200. lego-charge charges / at their served size +# and #362's rides the bundle, so at 1100 this corpus's XML answer is the 4-row floor with its next= unpaid +# (est_tokens 1213, over_ceiling="1" + the unpaid clause: disclosed, not silent — the knob-honesty contract, not the leak +# this arm pins). Swept on the merged binary: 1150 / 1200 serve 4 rows at est_tokens=1121 (JSON 1098), 1250..1350 five rows +# at 1246; 1200 sits mid-plateau with 79 tokens of headroom, one step clear of each edge. 1640 and 3000 unchanged. +for tb in 1200 1640 3000; do xe="$( xmlEst "$tb" )"; je="$( jsonEst "$tb" )" if [ -z "$xe" ] || [ -z "$je" ]; then no "budget=$tb: could not read est_tokens from one of the dialects (xml='$xe' json='$je')"; continue; fi if [ "$xe" -le "$tb" ]; then ok "budget=$tb: XML est_tokens=$xe fits the ceiling" @@ -242,7 +247,7 @@ done # ── arm 2: the two dialects select COMPARABLE row counts (they need not be equal) ────────────────── # Before the fix the XML lens bought 2-2.4x the rows with the same budget, because notes were free. -for tb in 1100 1640 3000; do +for tb in 1200 1640 3000; do xr="$( xmlRows "$tb" )"; jr="$( jsonRows "$tb" )" if [ -z "$jr" ] || [ "$jr" -eq 0 ]; then no "budget=$tb: JSON selected no rows — the comparison has no denominator"; continue; fi if [ "$xr" -le $(( jr * 13 / 10 + 1 )) ] && [ "$xr" -ge $(( jr * 7 / 10 )) ]; then @@ -365,7 +370,10 @@ fi # ── arm 4: L3 inertness — a tree with no notes is unaffected by any of this ──────────────────────── rm -f "$CORPUS/.ripwire_notes" -bare="$( "$BIN" "$CORPUS" --for="$TASK" --token-budget=800 2>/dev/null )" +# RE-ANCHORED 2026-10-10 (train 26c): 800 -> 900 for the no-notes arms below (same cause as arm 1's rung: at 800 the merged +# binary serves the 4-row floor with its unpaid next=, est_tokens 988 default / 951 full, over and labelled; from 850 both +# postures fit — 808 / 817 — and 900 holds 865 in the full legend with margin). The notes corpus's 800 runs above are unchanged. +bare="$( "$BIN" "$CORPUS" --for="$TASK" --token-budget=900 2>/dev/null )" case "$bare" in *" element";; *) ok "a tree with no notes emits none (L3 inertness)";; esac # L1 (2026-09-19): the 800-token fit was calibrated in the full legend, the default when it was written, and is asked # for by name. The DEFAULT (compact) answer on this tree carries more rows and lands over 800 at this one budget; that is @@ -374,20 +382,20 @@ case "$bare" in *"/dev/null )" +bareFull="$( "$BIN" "$CORPUS" --for="$TASK" --token-budget=900 --legend=full 2>/dev/null )" bareEst="$( printf '%s' "$bareFull" | grep -o 'est_tokens="[0-9]*"' | head -1 | tr -dc '0-9' )" -if [ -n "$bareEst" ] && [ "$bareEst" -le 800 ]; then ok "no-notes tree also fits the ceiling in the full legend (est_tokens=$bareEst)" -else no "no-notes tree reports est_tokens='$bareEst' against a budget of 800 (full legend)"; fi +if [ -n "$bareEst" ] && [ "$bareEst" -le 900 ]; then ok "no-notes tree also fits the ceiling in the full legend (est_tokens=$bareEst)" +else no "no-notes tree reports est_tokens='$bareEst' against a budget of 900 (full legend)"; fi defEst="$( printf '%s' "$bare" | grep -o 'est_tokens="[0-9]*"' | head -1 | tr -dc '0-9' )" if [ -z "$defEst" ]; then no "no-notes tree at the default posture carries no est_tokens=" -elif [ "$defEst" -le 800 ]; then ok "no-notes tree at the default posture fits (est_tokens=$defEst)" +elif [ "$defEst" -le 900 ]; then ok "no-notes tree at the default posture fits (est_tokens=$defEst)" else - case "$bare" in *'over_ceiling="1"'*) ok "no-notes tree at the default posture is over 800 (est_tokens=$defEst) and SAYS so (over_ceiling=\"1\")" ;; - *) no "no-notes tree at the default posture is over 800 (est_tokens=$defEst) in SILENCE" ;; esac + case "$bare" in *'over_ceiling="1"'*) ok "no-notes tree at the default posture is over 900 (est_tokens=$defEst) and SAYS so (over_ceiling=\"1\")" ;; + *) no "no-notes tree at the default posture is over 900 (est_tokens=$defEst) in SILENCE" ;; esac fi # ── arm 5: still deterministic and well-formed after the accounting change ───────────────────────── -if [ "$( "$BIN" "$CORPUS" --for="$TASK" --token-budget=800 2>/dev/null )" = "$bare" ]; then ok "output is byte-identical run-to-run" +if [ "$( "$BIN" "$CORPUS" --for="$TASK" --token-budget=900 2>/dev/null )" = "$bare" ]; then ok "output is byte-identical run-to-run" else no "output is not deterministic"; fi if command -v xmllint >/dev/null 2>&1; then if printf '%s' "$OUT800" | xmllint --noout - 2>/dev/null; then ok "note-bearing XML is well-formed (G4)"; else no "note-bearing XML fails xmllint"; fi diff --git a/test/forrankordercheck.sh b/test/forrankordercheck.sh index f260451e1..9f0faf355 100755 --- a/test/forrankordercheck.sh +++ b/test/forrankordercheck.sh @@ -138,7 +138,13 @@ REPO_Q=( "rank graph teleport" "compact legend rewrite" "edit receipt post-check # replace, so the byte-shaped section fits five more signature rows, and the legend that grew 154 B (the sc= # rule, the route= code's reading and the merged-callee reading) is paid for several times over in rows. The other # nine stay at their for-widen bases: -1.11%, +0.36%, -0.45%, +1.54%, +0.02%, +1.66%, +0.18%, -0.56%, +2.30%. -REPO_BASE=( 10134 10029 10080 10102 9880 9397 10009 10070 9966 10379 ) +REPO_BASE=( 10134 10029 10642 10102 9880 9397 10009 10070 9966 10379 ) +# q3 RE-PINNED 2026-10-10 (train 26c): 10080 -> 10642 B (+5.58% on the merged stack), an OUTPUT change of two signed lanes +# that each stay inside the 4% alone: lego-charge serves two more signature rows (25 -> 27 of 40) because / +# are charged to the budget at the size they are SERVED (a collapsing section its stub), and owner-hop adds the +# question's owner hop row with its callees. Measured on this tree: the 26c parent 9,716 B / 25 rows, the seven phase-A lanes +# 10,198 B / 25 rows, the lego-charge lane alone 10,356 B / 27 rows, the merged binary 10,642 B / 27 rows. The other nine stay +# at their bases: +0.25%, -2.64%, -7.65%, +2.00%, +1.86%, -2.25%, -1.50%, -2.88%, -6.06%. # q10 RE-PINNED 2026-09-19 (lane/r1-for-sections-stub, L2 round-1 lever B1): 9861 -> 10379 B (+5.25%), an # OUTPUT change, the q5 precedent's pattern. This query's ranked head reaches an interface with # implementors — the ONLY one of the ten reference queries that does — so it is the one query in this @@ -588,6 +594,32 @@ else no "(10) MCP budget_tokens=3800: shown/capped/handle/over/est = ${*:-} — want a paid handle inside the budget" fi +# ── (10c) THE at= STAMP IS READ ONCE PER RUN, WHICHEVER PAYMENT MODE SERVES (CodeRabbit on #383, verbs_for.h:3843) ───────────── +# A --token-budget run renders in up to three payment modes (unpaid, paid, unpaid again) before a byte reaches stdout; each +# render re-read the git stamp (rev-parse + status --porcelain), so a run over its ceiling either way spawned three +# `status --porcelain` children. The ranking is computed once per run and so is the stamp now. Fixture: fxcut as a one-commit +# git repository (the stamp only reads a git root); budget 500 against an answer of ~1100 est tokens is over either way +# (over_ceiling="1" — the premise); a PATH git shim logs every git call and runs the real git. RED on the pre-fix binary +# (3 status children); GREEN: exactly 1, and at= still rides the answer. +FXG="$TMP/fxcutgit"; cp -R "$TMP/fxcut" "$FXG" +git -C "$FXG" init -q && git -C "$FXG" add -A && git -C "$FXG" -c user.name=t -c user.email=t@t commit -qm init +GSHIM="$TMP/gitshim"; mkdir -p "$GSHIM"; REALGIT="$( command -v git )" +cat > "$GSHIM/git" <> "$GSHIM/calls.log" +exec "$REALGIT" "\$@" +EOF +chmod +x "$GSHIM/git"; : > "$GSHIM/calls.log" +out="$( cd "$TMP" && PATH="$GSHIM:$PATH" "$BIN" fxcutgit --for="gadget assembler" --no-cache --token-budget=500 2>/dev/null )" +st="$( grep -c 'status --porcelain' "$GSHIM/calls.log" | tr -d ' ' )" +if ! printf '%s' "$out" | grep -q 'over_ceiling="1"'; then + no "(10c) premise: --token-budget=500 on fxcutgit is not over its ceiling either way (no over_ceiling=\"1\"): $( printf '%s' "$out" | grep -o ']*>' | head -c 300 )" +elif [ "$st" = 1 ] && printf '%s' "$out" | grep -q ' at="'; then + ok "(10c) a three-render --token-budget run reads the git stamp once (1 status --porcelain child), and at= rides the answer" +else + no "(10c) status --porcelain children: $st (want 1; one per render = 3 before the fix); at= present: $( printf '%s' "$out" | grep -c ' at="' )" +fi + # ── (11) C3 COMPLETED: PAY FOR next= ONLY WHEN PAYING IS WHAT MAKES THE ANSWER FIT (orchestrator ruling 2026-10-08) ───────────── # The answer with every row the cut leaves PLUS the unpaid handle is the first candidate; it is served whenever it fits, and only # when it lands past its ceiling are rows dropped to pay (and served only if THAT fits — (10)'s @3800). 45a2eeba paid first and @@ -595,6 +627,10 @@ fi # an alternative that also fit. Same fixture and relative path as (10). Pins are 255dc199's own rows at each budget (the cut's # rows; re-pin from a pre-continuation binary if row bytes or the header change), budgets chosen with room on both sides: # CLI XML @2160: 255dc199 17 rows (est 2046); 45a2eeba 16 (est 1998) — RED there; head 17 + next= (est 2078 <= 2160). +# train 26c re-pin (2026-10-10, lego-charge): the lane charges / to the budget at their served size, so +# the cut alone leaves 16 rows at 2160 on this fixture (the lane's own pre-continuation binary 1cfbfdcc: 16 rows, est 1970; +# the 26c parent and the seven phase-A lanes: 17, est 1985); the merged binary serves those 16 + the unpaid next= (est 2002 +# <= 2160, no over_ceiling=). Re-pinned from the pre-continuation binary, as this comment asks; the assertions are unchanged. # --json @2000: 255dc199 16 rows (est 1657); 45a2eeba 15 — RED there; head 16 + "sigs_next" (est 1691). # MCP @4340: 255dc199 40 rows (est 4257); 45a2eeba 39 — RED there; head 40 + next_budget_tokens= (est 4308 <= 4340). # Each: rows >= the cut's own, the handle rides, est_tokens <= the budget, no over_ceiling. A binary that always pays when it @@ -608,7 +644,7 @@ c11check(){ # LABEL BUDGET WANT ROWS CAPPED HANDLE OVER EST no "(11) $label @$bt: shown/capped/handle/over/est = ${*:-} — want >= $want rows, the handle, est <= $bt, no over_ceiling= (paid where it already fit?)" fi } -c11check "CLI --for" 2160 17 $( c11row "$( cd "$TMP" && "$BIN" fxcut --for="gadget assembler" --token-budget=2160 --no-cache 2>/dev/null | cut_arm )" ) +c11check "CLI --for" 2160 16 $( c11row "$( cd "$TMP" && "$BIN" fxcut --for="gadget assembler" --token-budget=2160 --no-cache 2>/dev/null | cut_arm )" ) c11check "MCP for" 4340 40 $( c11row "$( mcprel 4340 )" ) set -- $( cd "$TMP" && "$BIN" fxcut --for="gadget assembler" --token-budget=2000 --json --no-cache 2>/dev/null \ | python3 -c 'import json,sys; d=json.load(sys.stdin); print(len(d["sigs"]), "sigs_next" in d, bool(d.get("over_ceiling")), d["est_tokens"])' 2>/dev/null ) diff --git a/test/forrootlegendcheck.sh b/test/forrootlegendcheck.sh index 88ecb160e..b4b6fe245 100755 --- a/test/forrootlegendcheck.sh +++ b/test/forrootlegendcheck.sh @@ -76,24 +76,30 @@ done # rows the header then cannot pay for: on this very fixture the a4a58141 default (before those definitions) was over its # budget at 870, 930 and 1000 tokens and fit 850 by three tokens. That overshoot is labelled over_ceiling="1" and counted in # the CHANGELOG; the root= clause's own cost is what this arm is about, and the full dialect measures it. -OUT2="$( "$BIN" "$TMP/tiny" --for="widget routine dispatcher" --token-budget=850 --no-cache --legend=full 2>/dev/null )" +# RE-ANCHORED 2026-10-10 (train 26c): 850 -> 1000. lego-charge charges / at their served size and #362's +# rides the bundle, so at 850 this fixture is the 4-row floor with its next= unpaid, labelled over_ceiling="1" +# at est_tokens=967 in the full legend (red on two macOS shards of run 38041394072 and on a 10-core host alike). Swept +# 900..1200 step 25 on the merged binary (full legend / default): 925 serves 5 rows at 892 / 880, 950..975 six rows at 939 / +# 942, 1000 six rows at 940 / 989 — 60 tokens of headroom in the full legend, the dialect this arm measures (the 2026-09-12 +# re-anchor's 36-token posture or better); 1025+ seven rows at 987. The assertions are unchanged. +OUT2="$( "$BIN" "$TMP/tiny" --for="widget routine dispatcher" --token-budget=1000 --no-cache --legend=full 2>/dev/null )" EST2="$( printf '%s' "$OUT2" | grep -o 'est_tokens="[0-9]*"' | head -1 | tr -dc '0-9' )" -if [ -n "$EST2" ] && [ "$EST2" -le 850 ]; then - ok "arm2: --token-budget=850 fits the ceiling in the full legend (est_tokens=$EST2)" +if [ -n "$EST2" ] && [ "$EST2" -le 1000 ]; then + ok "arm2: --token-budget=1000 fits the ceiling in the full legend (est_tokens=$EST2)" else - no "arm2: --token-budget=850 est_tokens=${EST2:-unreadable} exceeds the ceiling — the legend clause is too expensive" + no "arm2: --token-budget=1000 est_tokens=${EST2:-unreadable} exceeds the ceiling — the legend clause is too expensive" fi -OUT2D="$( "$BIN" "$TMP/tiny" --for="widget routine dispatcher" --token-budget=850 --no-cache 2>/dev/null )" +OUT2D="$( "$BIN" "$TMP/tiny" --for="widget routine dispatcher" --token-budget=1000 --no-cache 2>/dev/null )" EST2D="$( printf '%s' "$OUT2D" | grep -o 'est_tokens="[0-9]*"' | head -1 | tr -dc '0-9' )" -if [ -n "$EST2D" ] && { [ "$EST2D" -le 850 ] || printf '%s' "$OUT2D" | head -c 600 | grep -q ' over_ceiling="1"'; }; then - ok "arm2: the default answer at --token-budget=850 fits or says it does not (est_tokens=$EST2D)" +if [ -n "$EST2D" ] && { [ "$EST2D" -le 1000 ] || printf '%s' "$OUT2D" | head -c 600 | grep -q ' over_ceiling="1"'; }; then + ok "arm2: the default answer at --token-budget=1000 fits or says it does not (est_tokens=$EST2D)" else - no "arm2: the default answer at --token-budget=850 is over (est_tokens=${EST2D:-unreadable}) with no over_ceiling label" + no "arm2: the default answer at --token-budget=1000 is over (est_tokens=${EST2D:-unreadable}) with no over_ceiling label" fi if printf '%s' "$OUT2" | grep -qF "$CLAUSE_SNIPPET"; then - ok "arm2: the clause SURVIVES at --token-budget=850 (not dropped by the ceiling ladder)" + ok "arm2: the clause SURVIVES at --token-budget=1000 (not dropped by the ceiling ladder)" else - no "arm2: the clause is missing at --token-budget=850 — it was silently dropped instead of fitting" + no "arm2: the clause is missing at --token-budget=1000 — it was silently dropped instead of fitting" fi if printf '%s' "$OUT2" | xmllint --noout - 2>/dev/null; then ok "arm2: tight-budget output is well-formed (G4)"; else no "arm2: tight-budget output fails xmllint"; fi diff --git a/test/forsectioncollapsecheck.sh b/test/forsectioncollapsecheck.sh index 53de643b0..27eb844cd 100644 --- a/test/forsectioncollapsecheck.sh +++ b/test/forsectioncollapsecheck.sh @@ -25,7 +25,8 @@ # Fixtures: test/legofix (lego collapses: three small interfaces still clear the threshold) and # test/sectionpricefix (both lego and compose collapse: a purpose-built interface + HAS-A owners sized # comfortably above threshold) prove the ABOVE-threshold side; test/hasafix (compose stays whole: two -# short rows, priced BELOW threshold) proves the round-2 delta itself, arm (10). +# short rows, priced BELOW threshold) proves the round-2 delta itself, arm (10). test/legochargefix (a lego +# block bigger than the budget) proves the stub is what the budget is charged for, arm (13). # # Usage: bash test/forsectioncollapsecheck.sh [path-to-ripwire-binary] # RIPWIRE_BIN=build/ripwire bash test/forsectioncollapsecheck.sh @@ -393,5 +394,59 @@ else ok "(12) MCP legend disclosure arm skipped (python3 absent)" fi +# ── (13) lego-charge (K18 root cause): the budget charges a collapsing section at its STUB, not its full render. +# test/legochargefix is textual-shaped: its query names 14 interfaces with 4 implementors each, so the full render +# (~10 KB) is bigger than the default budget, while the default answer serves it as a ~150 B stub. Charging the +# full render left 1 B for : 4 rows in a ~3 KB answer under a 7.5 KB ceiling (textual's round-1 rows: 4 of 40). +# Rules (rw::forSectionsSigChargePlan, serialize.h): no explicit ceiling → the default answer's charge, also under +# --sections= (so the stub's next= restores both sections with the SAME , byte-identically); an explicit ceiling +# (--token-budget, MCP budget_tokens) → a section --sections= opts into is charged in full (hard bound). +LCQ="how do render stage classes arrange and reflow widgets" +lc_shown(){ grep -o '' "$1"; } +"$BIN" test/legochargefix --no-cache --for="$LCQ" >"$TMP/lc_def.xml" 2>/dev/null; rc_def=$? +"$BIN" test/legochargefix --no-cache --for="$LCQ" --sections=lego,compose >"$TMP/lc_rest.xml" 2>/dev/null; rc_rest=$? +"$BIN" test/legochargefix --no-cache --for="$LCQ" --token-budget=3000 >"$TMP/lc_tb.xml" 2>/dev/null; rc_tb=$? +"$BIN" test/legochargefix --no-cache --for="$LCQ" --token-budget=3000 --sections=lego,compose >"$TMP/lc_tbrest.xml" 2>/dev/null; rc_tbrest=$? +LC_DEF="$( lc_shown "$TMP/lc_def.xml" )"; LC_REST="$( lc_shown "$TMP/lc_rest.xml" )" +LC_TB="$( lc_shown "$TMP/lc_tb.xml" )"; LC_TBREST="$( lc_shown "$TMP/lc_tbrest.xml" )"; LC_TBEST="$( lc_est "$TMP/lc_tbrest.xml" )" +if [ "$rc_def$rc_rest$rc_tb$rc_tbrest" != "0000" ] || ! [[ "$LC_DEF" =~ ^[0-9]+$ && "$LC_REST" =~ ^[0-9]+$ && "$LC_TB" =~ ^[0-9]+$ && "$LC_TBREST" =~ ^[0-9]+$ && "$LC_TBEST" =~ ^[0-9]+$ ]]; then + no "(13) legochargefix: a run failed or printed no /est_tokens= (rc=$rc_def/$rc_rest/$rc_tb/$rc_tbrest shown=$LC_DEF/$LC_REST/$LC_TB/$LC_TBREST est=$LC_TBEST)" +else + # (13a) RED on the base: the stub is served, so the budget pays the stub — the answer carries a budget's worth of rows + grep -Eq '' "$TMP/lc_def.xml" && [ "$LC_DEF" -ge 24 ] \ + && ok "(13a) default: is served as a stub and shows $LC_DEF rows (the stub's price was charged, not the full render's)" \ + || no "(13a) default: (want >= 24 with a stub) — the budget paid for a lego render it never served" + # (13b) the stub's next= restores both sections byte-identically: same block, the full on top + [ "$( lc_sigs "$TMP/lc_def.xml" )" = "$( lc_sigs "$TMP/lc_rest.xml" )" ] && grep -q ' as the stubbed answer ($LC_REST rows) plus the full " \ + || no "(13b) --sections=lego,compose: differs from the stubbed answer's ($LC_DEF) or no full — next= does not restore the cut it disclosed" + LC_STUBTOTAL="$( grep -o ' ($LC_IFACES rows, display cap 12)" \ + || no "(13c) the stub's total=\"$LC_STUBTOTAL\" vs $LC_IFACES restored rows (display cap 12)" + # (13d) NEAR-MISS: under an explicit ceiling, a section --sections= opts into is charged in full — the hard bound holds + # (its is then narrowed to the few rendered rows' files — here none of stages.h — so the bound is read off ) + [ "$LC_TBEST" -le 3000 ] && [ "$LC_TBREST" -lt "$LC_TB" ] \ + && ok "(13d) --token-budget=3000 --sections=lego,compose: est_tokens=$LC_TBEST <= 3000, $LC_TBREST < $LC_TB (the full was charged)" \ + || no "(13d) --token-budget=3000 --sections=lego,compose: est_tokens=$LC_TBEST, $LC_TBREST vs $LC_TB stubbed — the opted-in section was not charged in full under an explicit ceiling" +fi +if command -v python3 >/dev/null 2>&1; then + lc_mcp(){ printf '{"jsonrpc":"2.0","id":1,"method":"tools/call","params":{"name":"for","arguments":{"path":"test/legochargefix","task":"%s"%s}}}\n' "$LCQ" "$1" \ + | "$BIN" --mcp 2>/dev/null | python3 -c " +import json,sys,re +t = json.loads(sys.stdin.readline())['result']['content'][0]['text'] +m = re.search(r' $LCM_DEF rows by default; under budget_tokens + sections $LCM_TBREST < $LCM_TB" \ + || no "(13e) MCP for: default=$LCM_DEF budget=$LCM_TB budget+sections=$LCM_TBREST (want >= 24, and the last below the middle)" +else + printf ' SKIP %s\n' "(13e) python3 absent: the MCP twin of the lego charge is not checked" +fi + [ "$fail" -eq 0 ] && echo "ALL PASS" || echo "SOME FAILED" exit "$fail" diff --git a/test/forwidencheck.sh b/test/forwidencheck.sh index 4bee48adf..eb6e1d5b9 100755 --- a/test/forwidencheck.sh +++ b/test/forwidencheck.sh @@ -276,7 +276,7 @@ run --for="$THIN" --limit=0 >/dev/null; rc=$? if [ "$rc" != 0 ] && grep -q -- '--limit' "$TMP/err"; then ok "(6) --limit=0 is refused (rc=$rc)"; else no "(6) --limit=0 not refused: rc=$rc $( head -c 160 "$TMP/err" )"; fi run --for="$THIN" --limit=abc >/dev/null; rc=$? if [ "$rc" != 0 ] && grep -q -- '--limit' "$TMP/err"; then ok "(6) --limit=abc is refused (rc=$rc)"; else no "(6) --limit=abc not refused: rc=$rc $( head -c 160 "$TMP/err" )"; fi -for flag in --json --format=candidates --detail=1 --signatures-only --token-budget=2000; do +for flag in --json --format=candidates --detail=1 --signatures-only; do run --for="$THIN" --limit=5 $flag >"$TMP/shape.out"; rc=$? if [ "$rc" != 0 ] && [ ! -s "$TMP/shape.out" ]; then ok "(6) the page refuses $flag rather than ignoring it (rc=$rc)" @@ -284,6 +284,17 @@ for flag in --json --format=candidates --detail=1 --signatures-only --token-budg no "(6) the page accepted $flag: rc=$rc, $( wc -c <"$TMP/shape.out" | tr -d ' ' ) bytes on stdout, stderr: $( head -c 160 "$TMP/err" )" fi done +# PAGING-POC (issue #294): --token-budget beside a window no longer refuses — the combination now has a +# DEFINED meaning, the budgeted-bundle CANDIDATE PAGE (a different document, verbs_for.h's shared +# emitForCandidatePage). The old refusal guarded against budget-silently-ignored beside this file page; +# the new arm pins the STRONGER contract: budget+window answers with the -rooted candidate page +# carrying the pageview quintet, so THIS file page stays budgetless by construction, never by refusal. +run --for="$THIN" --limit=5 --token-budget=2000 >"$TMP/shape.out"; rc=$? +if [ "$rc" = 0 ] && grep -q '^/dev/null; } +ref_run(){ spec_env "$BIN" "$SPEC" --top-k=400 --no-cache 2>/dev/null; } +# same_as_ref LABEL SYMBOL want(present|absent): one cached run against one --no-cache run, then the symbol's state +same_as_ref(){ + spec_run >"$TMP/spec.out"; ref_run >"$TMP/ref.out" + if ! cmp -s "$TMP/spec.out" "$TMP/ref.out"; then no "14$1: the cached (speculated) run differs from --no-cache"; return; fi + if grep -q "$2" "$TMP/spec.out"; then found=present; else found=absent; fi + if [ "$found" = "$3" ]; then ok "14$1: cached run == --no-cache, and $2 is $3" + else no "14$1: $2 should be $3, the cached run has it $found"; fi +} +spec_run >/dev/null +if [ -n "$( find "$SC" -name 'ripwire-ignoreprobe-*.bin' 2>/dev/null | head -1 )" ]; then ok "14a: a cached run on a git root keeps the ignore answer for the next call's speculation" +else no "14a: no ignore-probe sidecar under the cache dir — the probe is not speculated"; fi +same_as_ref b rwGateGeneratedSymbol absent # warm: the speculation matches git +printf 'gen_out/\ntracked_anyway.cpp\n' >"$SPEC/.gitignore" # c: an EDITED .gitignore un-ignores generated.cpp +same_as_ref c rwGateGeneratedSymbol present +printf 'int rwGateNestedSymbol( int a ) { return a + 7; }\n' >"$SPEC/keep/extra.cpp"; spec_run >/dev/null +printf 'extra.cpp\n' >"$SPEC/keep/.gitignore" # d: a NEW nested .gitignore +same_as_ref d rwGateNestedSymbol absent +printf 'int rwGateInfoExcludeSymbol( int a ) { return a + 8; }\n' >"$SPEC/info_ex.cpp"; spec_run >/dev/null +mkdir -p "$SPEC/.git/info"; printf 'info_ex.cpp\n' >>"$SPEC/.git/info/exclude" # e: .git/info/exclude +same_as_ref e rwGateInfoExcludeSymbol absent +printf 'int rwGateGlobalSymbol( int a ) { return a + 9; }\n' >"$SPEC/glob_ex.cpp"; spec_run >/dev/null +printf 'glob_ex.cpp\n' >"$TMP/xdg/git/ignore" # f1: the DEFAULT global excludes file +same_as_ref f1 rwGateGlobalSymbol absent +printf 'int rwGateExcludesFileSymbol( int a ) { return a + 10; }\n' >"$SPEC/cfg_ex.cpp"; : >"$TMP/excludes" +git -C "$SPEC" config core.excludesFile "$TMP/excludes"; spec_run >/dev/null +printf 'cfg_ex.cpp\n' >"$TMP/excludes" # f2: core.excludesFile's file, edited +same_as_ref f2 rwGateExcludesFileSymbol absent +printf 'scratch_*.cpp\n' >>"$SPEC/.gitignore"; spec_run >/dev/null # g: no rule FILE changes, only the tree: +printf 'int rwGateScratchSymbol( int a ) { return a + 11; }\n' >"$SPEC/scratch_new.cpp" # a new file an old rule ignores +same_as_ref g rwGateScratchSymbol absent +printf 'gen_out/\n' >"$SPEC/.gitignore" # i: a LOST speculation whose walk the memory +spec_env env RIPWIRE_TEST_MEMGUARD=crawl:6 TMPDIR="$SC/" "$BIN" "$SPEC" --top-k=400 >"$TMP/spec.i" 2>"$TMP/spec.i.err"; rcSpecI=$? # guard stops: +spec_env env RIPWIRE_TEST_MEMGUARD=crawl:6 "$BIN" "$SPEC" --top-k=400 --no-cache >"$TMP/ref.i" 2>/dev/null; rcRefI=$? # the redo is re-armed +if ! grep -q 'stopped the crawl' "$TMP/spec.i.err"; then no "14i: the crawl:6 seam did not stop the walk — the arm tests nothing" +elif [ "$rcSpecI" = "$rcRefI" ] && cmp -s "$TMP/spec.i" "$TMP/ref.i"; then ok "14i: a lost speculation under a crawl memory stop answers as --no-cache does (rc $rcSpecI)" +else no "14i: a lost speculation under a crawl memory stop differs from --no-cache (rc $rcSpecI vs $rcRefI)"; fi +spec_run >"$TMP/spec.a"; spec_run >"$TMP/spec.b" +if cmp -s "$TMP/spec.a" "$TMP/spec.b"; then ok "14h: two cached runs in a row are byte-identical"; else no "14h: two cached runs in a row differ"; fi + # ── 13. the flag is in --help (the deckcheck allowlist row for --no-ignore retires with it). "$BIN" --help=all 2>&1 | grep -q -- '--no-ignore' && ok "--no-ignore is documented in --help" \ || no "--no-ignore is missing from --help" diff --git a/test/greptiercheck.sh b/test/greptiercheck.sh index e864b5cc0..5bdb2edbe 100644 --- a/test/greptiercheck.sh +++ b/test/greptiercheck.sh @@ -390,6 +390,25 @@ fi [ "$( jkey total "$V_OK" )" = "$d_hits" ] \ && ok "(9d-b) an explicit in=\"code\" still answers (the refusal is on unknown values, not on presence)" \ || no "(9d-b) in=\"code\" was refused or changed the answer — the value check is too broad" +# ── (9e) a NON-STRING in= refuses through the shared shape gate and echoes the value SENT (CodeRabbit 5468003465's +# sibling: `in` used to be read inside the grep arm, after that gate, so in:5 decoded to "" and the verb answered the +# default tier as if nothing had been sent). Near-miss twins: (9d) a string outside the set still refuses by name. +for arm in '5|5' '["any"]|["any"]' '{"v":1}|{"v":1}' 'true|true'; do + val="${arm%%|*}"; echo_="${arm#*|}" + V_NS="$( mcpcall '{"jsonrpc":"2.0","id":2,"method":"tools/call","params":{"name":"grep","arguments":{"path":"'"$SB"'","pattern":"TIERTOKEN_frob","in":'"$val"'}}}' )" + # the message is read out of the JSON (a quote inside the echoed value is \" on the wire), as crossrefcheck (L10b) does + V_MSG="$( printf '%s' "$V_NS" | python3 -c 'import sys,json +for line in sys.stdin: + line=line.strip() + if not line: continue + d=json.loads(line) + if "error" in d: print("ERROR", d["error"].get("code"), d["error"].get("message",""), end="") + elif d.get("result",{}).get("content"): print("ANSWER", end="")' 2>/dev/null )" + case "$V_MSG" in + "ERROR -32602 invalid value for field: in"*"got '$echo_'"*) ok "(9e) the live MCP grep verb refuses a non-string in=$val through the shape gate and echoes '$echo_'" ;; + *) no "(9e) the live MCP grep verb did not refuse in=$val echoing '$echo_' — got: ${V_MSG:-nothing}" ;; + esac +done # ═══════════════════════════════════════════════════════════════════════════ echo "=== (11) M17 — a class label decided under a budget says so ===" diff --git a/test/hazardpatterncheck.sh b/test/hazardpatterncheck.sh index 7714a19ba..a714765e3 100755 --- a/test/hazardpatterncheck.sh +++ b/test/hazardpatterncheck.sh @@ -487,6 +487,7 @@ REGA cat > "$TMP/reg_b1.tsv" <<'REGB1' infra/emit.h emitTo silent 1 not an answer path: std::print's failed write is made silent to keep fputs's contract, which the header documents ingest_astquery.h astQueryGrouped silent 1 FINDING: a file whose query walk throws drops out of --match/--lint hits with no disclosure in any build; file held by the regexguard and crash-fixes lanes +ingest_astquery.h definitionLinesWorker alert-only 1 records in every build: the handler is the sink form DISCLOSE( ( BlobDefsSlot{ slot } ), WorkerThrew, … ), whose disclose() clears that blob's lines and marks it Failed, so the blob confirms no definition (its rows read kind="text"); the parenthesized sink is what the RECORDS pattern does not see ingest_astquery.h spanTiersOfFiles alert-only 1 FINDING (disclosure lane): a tier worker that throws leaves its files unclassified for --grep-in, silent in Release ingest_docpass.h runDocPostPass alert-only 1 FINDING (disclosure lane): a doc post-pass throw skips that document, silent in Release mcpindex.h maybePrefetchHeadSnapshot silent 1 not an answer path: optional background prefetch of the HEAD snapshot; a request recomputes whatever it did not fill (§2b rule 3) @@ -497,6 +498,8 @@ infra/dynamic_map.hpp at 1 vendored container API; no ripwire call site reaches regexguard.h throwIfMatchFaultInjected 1 the RIPWIRE_FAULT_REGEX_MATCH fault switch, off unless the variable is set; its three call sites are the first statement inside GuardedRegex's search, search and forEachMatch try blocks, which catch std::regex_error by type and return Exhausted REGB2 cat > "$TMP/reg_c.tsv" <<'REGC' +crossref.h listTreeRows .at 2 readTreeLevels returns its table only when every level was served whole by catFileBatch and every child tree id was queued unless already read, so the root tree and each subtree id it walks are keys +crossref.h listTreesOfRefs .at 1 rootTreesOf returns its map only when catFileBatch served every distinct tip (an answer for another object fails the read) and each served tip is assigned its tree, so every ref's tip is a key infra/dynamic_map.hpp at .at 1 the non-const at forwards to the const at of the same container; no ripwire caller ingest_astquery.h nearestNodeKindHint .at 1 the key was just read out of this same map by nearestNameByEditDistance, so it is present REGC @@ -522,6 +525,7 @@ ingest_astquery.h collectGatedLocalNames ts_parser_new 1 deleted before each of ingest_astquery.h collectGatedLocalNames ts_parser_parse_string 1 deleted before the final return ingest_astquery.h compileGrammarQueries ts_query_new 2 moved into GrammarQueries (or deleted when the combined query does not add up); astQueryGrouped deletes every one by hand after the pass ingest_astquery.h computeGrammarDisclosure ts_query_new 1 a compile probe, deleted inside the same if +ingest_astquery.h definitionLinesWorker ts_query_cursor_new 1 deleted on both exits of the noexcept worker: the no-parser return (when non-null) and after the claim loop, whose one throwing call sits inside try/catch ingest_astquery.h spanTiersOfFiles ts_parser_parse_string 1 the tree is deleted right after the walk ingest_crawl.h compileQueryStandalone ts_query_new 1 returned into the process-lifetime CompiledQueryCache, whose destructor deletes every query ingest_parsepool.h runParseWorker ts_query_cursor_new 1 deleted at the end of the worker; the returns in between belong to lambdas diff --git a/test/importcapcheck.sh b/test/importcapcheck.sh index b443f199a..ff726ff07 100755 --- a/test/importcapcheck.sh +++ b/test/importcapcheck.sh @@ -1,5 +1,5 @@ #!/usr/bin/env bash -# importcapcheck.sh — the gate for the shared import-capture vocabulary (issue #358), C-family slice. +# importcapcheck.sh — the gate for the shared import-capture vocabulary (issue #358), C-family and Go slices. # # WHAT THIS PINS. `#include` / `#import` for c, cpp, objc (and the .h/.hpp/.cu/.cuh/.metal extensions # that ride those grammars) no longer come from a per-language extractor in src/ingest_relations.h. They @@ -40,6 +40,20 @@ # identically, which is the evidence the records did not change # (RIPWIRE_BASE_BIN, the house name; SKIPs when unset, as on CI) # +# +# THE GO SLICE (arms 14-19, below the C-family arms). Go's `import_declaration` — single and grouped, aliased, +# dot, blank, cgo `import "C"` — is captured as `@import.path` in queries/go/tags.scm and read by the ONE +# DepDialect::Go normaliser in src/ingest_importcap.h; the walk in captureIncludes no longer runs for Go. +# 14 the Go capture is COMPILED IN — `(import_declaration) @import.path` is in the binary's embedded tags.scm +# 15 every Go form, exact rows — single/aliased/dot/blank/group/cgo/tab/semicolon/raw-string/UTF-8 cut/error +# recovery, pinned as the byte-for-byte rows the extractor emitted +# 16 no widening — an import_declaration the grammar never makes a child of the file root +# (inside a function body) stays out, exactly as the walk left it +# 17 resolution through imports — a go.mod `replace` still maps an import onto a sibling root's package +# (single, aliased and grouped), and a bogus replace target removes the edge +# 18 the use-site half — `--uses=` reports role="import" for a Go import +# 19 Go cache + determinism — warm == cold == --no-cache on --deps over the Go fixture +# # Usage: test/importcapcheck.sh # RIPWIRE_BIN=asan/ripwire test/importcapcheck.sh # Exits non-zero on any failure. Does NOT edit test/regression.sh or test/golden.xml. @@ -443,6 +457,511 @@ else no "non-deterministic output" fi +# ═══════════════════════════════════════════════════════════════════════════════════════════════════ +# JS/TS SLICE (DepDialect::Web) — TypeScript, TSX, JavaScript and the .astro frontmatter. +# +# WHAT THIS PINS. `import … from 'x'`, the re-exports (`export * from`, `export { a as b } from`, +# `export type { T } from`), and the CommonJS / dynamic `require('x')` / `import('x')` no longer come from +# captureIncludes' walk (directiveTargetOf's import_statement-source / export_statement / call_expression +# branches). They come from the same `@import.path` name, three patterns per queries/{typescript,tsx, +# javascript}/tags.scm, normalised by the DepDialect::Web arm of src/ingest_importcap.h. The EMITTED edges +# are exactly the ones the walk emitted, for every spelling — and for every spelling the walk did NOT emit: +# +# J1 the Web patterns are COMPILED IN, and the walk's TS/JS branches are GONE from src/ +# J2 exact edge list, in source order, for .ts .js .tsx .mjs .cjs .jsx and the .astro frontmatter — +# ES default/named/namespace/side-effect/type imports, three re-export spellings, CommonJS in every +# container shape the walk entered (declaration, assignment, export, callback, await, if/try/for…) +# J3 THE NEGATIVES — the query matches ANY `x('s')` call at ANY depth; every `neg_*` specifier below is +# a shape the walk never turned into an edge (member callee, `.foo` chained on the call, ternary, +# array, two arguments, a comment before the string, template literal, computed, `require.resolve`, +# a `new` argument, TS `import x = require()`, a `declare module` body, a class method) and none may +# become one. The leading text gate (callee is `require`/`import`) and the reach rule are what hold. +# J4 the LAZY bit — a require()/import() written inside a function body is lazy="1" on --impact's import +# tier; a top-level one is lazy="0"; one file required both ways is lazy="0" (the pair rule) +# J5 the use-site half — --uses reports the import-role ref of a named import +# J6 the DEPTH BOUND on blocks — kept at 255 and 256 containers, cut AND announced at 257, as the walk did +# J7 the DELIBERATE disclosure differences (the two of the C-family arm 12c, same reason, plus a non-import under a too-deep nest) +# J8 cache round-trip, a pre-change binary's cache accepted (no kParserVer bump), determinism +# +# The expected lists are the SPECIFICATION and were read off the pre-change binary, never reasoned out: this +# slice is a refactor, so main's set is the contract. The `neg_*` / `lazy_*` / `cjs_*` names say what each +# edge (or non-edge) is, so a diff reads without opening the fixture. + +web_fixture_ts() { local W="$1" + cat > "$W/main.ts" <<'TS' +import def from "./es_default"; +import { named1, named2 as alias2 } from "./es_named"; +import * as ns from "./es_namespace"; +import "./es_sideeffect"; +import type { T } from "./es_type_named"; +import type DefT from "./es_type_default"; +import { type U, V } from "./es_inline_type"; +import dflt, { n1 } from "./es_default_and_named"; +import json from "./es_import_attributes" with { type: "json" }; +export * from "./re_star"; +export { default } from "./re_default"; +export { default as D } from './re_default_as_single_quote'; +export * as hns from "./re_star_as"; +export { x as y } from "./re_named_as"; +export type { Z } from "./re_type"; +import eq = require("./neg_import_equals"); +import type eq2 = require("./neg_import_equals_type"); +export = ns; +const r1 = require("./cjs_const"); +const { q1, q2 } = require("./cjs_destructured"); +var r2 = require('./cjs_var_single'); +require("./cjs_bare"); +module.exports = require("./cjs_module_exports"); +const dyn = await import("./dyn_await"); +const opt = require?.("./cjs_optional_call"); +const memb = require("./neg_member").foo; +const tern = cond ? require("./neg_ternary") : null; +const arr = [require("./neg_array")]; +const two = require("./neg_two_args", "./neg_second"); +const cm = require(/*c*/ "./neg_comment"); +const tpl = require(`./neg_template`); +const nonlit = require(neg_variable); +const other = foo.require("./neg_foo_require"); +const cat = require("./neg_concat" + sfx); +const res = require.resolve("./neg_resolve"); +const nw = new Foo(require("./neg_new_arg")); +const nest = foo(bar(require("./fn_arg_nested"))); +const th = import("./neg_dynamic_then").then(( m ) => m); +const bang = !require("./neg_unary"); +function f() { return require("./lazy_function"); } +const g2 = () => import("./lazy_arrow"); +class K { m() { return require("./neg_class_method"); } } +const o = { get G() { return require("./lazy_getter"); } }; +if (x) { require("./if_then"); } else { require("./if_else"); } +try { require("./try_body"); } catch (e) { require("./catch_body"); } +for (;;) { require("./for_body"); } +switch (x) { case 1: require("./switch_case"); } +export default require("./export_default_require"); +export const ex = require("./export_const_require"); +export function ef() { return import("./export_fn_dynamic"); } +foo(function () { require("./callback_fn"); }); +foo(() => require("./callback_arrow"), require("./callback_second")); +const pe = (require("./paren_require")); +const aw = await require("./await_require"); +label: require("./labeled"); +TS +} +web_fixture_js() { local W="$1" + cat > "$W/main.js" <<'JS' +import def from "./es_default"; +import { named1, named2 as alias2 } from "./es_named"; +import * as ns from "./es_namespace"; +import "./es_sideeffect"; +export * from "./re_star"; +export * as hns from "./re_star_as"; +export { x as y } from "./re_named_as"; +const r1 = require("./cjs_const"); +const { q1, q2 } = require("./cjs_destructured"); +require("./cjs_bare"); +module.exports = require("./cjs_module_exports"); +const dyn = await import("./dyn_await"); +const memb = require("./neg_member").foo; +const arr = [require("./neg_array")]; +const two = require("./neg_two_args", "./neg_second"); +const cm = require(/*c*/ "./neg_comment"); +const nonlit = require(neg_variable); +function f() { return require("./lazy_function"); } +const g2 = () => import("./lazy_arrow"); +class K { m() { return require("./neg_class_method"); } } +exports.run = function () { return require("./lazy_exports_fn"); }; +JS +} +web_fixture_small() { local W="$1" # .tsx .mjs .cjs .jsx and the .astro frontmatter + cat > "$W/neg.ts" <<'NEG' +declare module "amb" { import q from "./neg_ambient"; } +namespace NS { import w = require("./neg_ns_equals"); const z2 = require("./neg_ns_const"); } +NEG + cat > "$W/comp.tsx" <<'TSX' +import React from "react"; +import { Button } from "./tsx_button"; +export { Card } from "./tsx_card"; +export const View = () => ; +const side = require("./tsx_cjs"); +TSX + printf 'import m from "./mjs_import";\nexport * from "./mjs_reexport";\nconst c = require("./mjs_cjs");\n' > "$W/mod.mjs" + printf 'const a = require("./cjs_file");\nmodule.exports = { get b() { return require("./cjs_lazy"); } };\n' > "$W/mod.cjs" + printf 'import J from "./jsx_import";\nconst V = () => ;\n' > "$W/view.jsx" + printf '%s\n' '---' 'import A from "./astro_import";' 'const m = require("./astro_require");' 'export * from "./astro_reexport";' '---' '
{A}
' > "$W/page.astro" +} +web_fixture_stubs() { local W="$1" + # every specifier above names a real file, so each edge RESOLVES and the lazy bit and --uses are observable + local spec + for spec in $(cat "$W"/main.ts "$W"/neg.ts "$W"/main.js "$W"/comp.tsx "$W"/mod.mjs "$W"/mod.cjs "$W"/view.jsx "$W"/page.astro \ + | grep -o '"\./[A-Za-z_0-9]*"\|'"'"'\./[A-Za-z_0-9]*'"'" | tr -d "\"'" | sort -u); do + printf 'export const %s = 1;\n' "${spec#./}" > "$W/${spec#./}.js" + done +} +web_fixture() { # $1 = an empty directory + mkdir -p "$1" + web_fixture_ts "$1"; web_fixture_js "$1"; web_fixture_small "$1"; web_fixture_stubs "$1" +} + +# The expected Include targets per file, in source order, as one space-separated line each — read off the +# PRE-CHANGE binary on exactly this fixture. 40 / 15 / 4 / 3 / 2 / 1 / 3 edges. +W_MAIN_TS='./es_default ./es_named ./es_namespace ./es_sideeffect ./es_type_named ./es_type_default ./es_inline_type ./es_default_and_named ./es_import_attributes ./re_star ./re_default ./re_default_as_single_quote ./re_star_as ./re_named_as ./re_type ./cjs_const ./cjs_destructured ./cjs_var_single ./cjs_bare ./cjs_module_exports ./dyn_await ./cjs_optional_call ./fn_arg_nested ./lazy_function ./lazy_arrow ./lazy_getter ./if_then ./if_else ./try_body ./catch_body ./for_body ./export_default_require ./export_const_require ./export_fn_dynamic ./callback_fn ./callback_arrow ./callback_second ./paren_require ./await_require ./labeled' +W_MAIN_JS='./es_default ./es_named ./es_namespace ./es_sideeffect ./re_star ./re_star_as ./re_named_as ./cjs_const ./cjs_destructured ./cjs_bare ./cjs_module_exports ./dyn_await ./lazy_function ./lazy_arrow ./lazy_exports_fn' +W_COMP_TSX='react ./tsx_button ./tsx_card ./tsx_cjs' +W_MOD_MJS='./mjs_import ./mjs_reexport ./mjs_cjs' +W_MOD_CJS='./cjs_file ./cjs_lazy' +W_VIEW_JSX='./jsx_import' +W_PAGE_ASTRO='./astro_import ./astro_require ./astro_reexport' + +WEB="$TMP/web"; web_fixture "$WEB" +"$BIN" "$WEB" --deps --no-cache > "$TMP/web.deps.xml" 2> "$TMP/web.deps.err" +web_incs(){ sed 's/= 6 occurrences of the JS/TS @import.path patterns in the binary, got '${WEB_PATTERNS:-none}'" +fi +for gone in 'kindIs( t, "export_statement" )' 'target = jsModuleLoadTarget( n, src );'; do + if grep -rqF "$gone" "$ROOT/src"; then + no "J1: the walk's TS/JS branch \`$gone\` is still in src/ — it should have been removed with this language" + else + ok "J1: the walk's TS/JS branch is gone from src/ ($gone)" + fi +done + +# ── J2 + J3: exact edge list per file, which also pins every negative ────────────────────────────────── +WEB_BAD=0 +for pair in "main.ts:$W_MAIN_TS" "main.js:$W_MAIN_JS" "comp.tsx:$W_COMP_TSX" "mod.mjs:$W_MOD_MJS" \ + "mod.cjs:$W_MOD_CJS" "view.jsx:$W_VIEW_JSX" "page.astro:$W_PAGE_ASTRO"; do + f=${pair%%:*}; want=${pair#*:} + got=$(web_incs "$f") + if [ -z "$got" ] || [ "$got" != "$want" ]; then + WEB_BAD=1 + no "J2: $f Include targets differ from the pre-change set" + printf ' expected: %s\n got: %s\n' "$want" "${got:-}" + fi +done +[ "$WEB_BAD" -eq 0 ] && ok "J2: exact Include set and order for .ts .js .tsx .mjs .cjs .jsx and the .astro frontmatter (68 edges)" +NEG=$(grep -o ' "$LZ/entry.js" +for n in eager lazy_only both; do printf 'function %s_fn() { return 1; }\nmodule.exports = %s_fn;\n' "$n" "$n" > "$LZ/$n.js"; done +LZ_BAD=0 +for pair in eager_fn:0 lazy_only_fn:1 both_fn:0; do + s=${pair%%:*}; want=${pair#*:} + got=$("$BIN" "$LZ" --impact="$s" --no-cache 2>/dev/null | grep -o '/dev/null | grep -o ' "$WB/$1/deep.js" + local i=0 + { while [ "$i" -lt "$1" ]; do printf '{'; i=$((i + 1)); done + printf ' require("./deep"); ' + i=0; while [ "$i" -lt "$1" ]; do printf '}'; i=$((i + 1)); done; printf '\n'; } > "$WB/$1/a.js" +} +WB_BAD=0 +for n in 254 255 256; do + web_bound "$n" + e=$("$BIN" "$WB/$n" --deps --no-cache 2>/dev/null | grep -o '/dev/null | grep -c 'why="extract-partial"' | tr -d '[:space:]' || true) + if [ "$n" -le 255 ]; then want_e=1; want_f=0; else want_e=0; want_f=1; fi + if [ "$e" != "$want_e" ] || [ "$f" != "$want_f" ]; then + WB_BAD=1 + no "J6: $n nested blocks: edge=$e extract-partial=$f (want $want_e/$want_f)" + fi +done +[ "$WB_BAD" -eq 0 ] && ok "J6: depth bound — kept under 255 and 256 containers, cut AND announced under 257 (edges identical to the walk's)" + +# ── J7: the DELIBERATE disclosure differences, in both directions (the C-family arm 12c, for JS) ──── +# (i) QUIETER: `require('./deep')` is ITSELF a container, so the walk refused to descend into it from depth +# 256 and announced, although the edge was kept and nothing was dropped. n=255 above is that shape: edge +# kept, and this binary says nothing because nothing was lost (the walk said extract-partial). +# (ii) LOUDER: 300 blocks inside a class METHOD. The walk never entered the class (class_declaration is not +# an import container) so it said nothing; the ancestor loop reaches the bound first and announces. The +# safe direction — a floor that says so — and the same trade the C-family slice accepted. +WL="$TMP/wloud"; mkdir -p "$WL"; printf 'export const deep = 1;\n' > "$WL/deep.js" +{ printf 'class K { m() { '; i=0; while [ "$i" -lt 300 ]; do printf '{'; i=$((i + 1)); done + printf ' require("./deep"); '; i=0; while [ "$i" -lt 300 ]; do printf '}'; i=$((i + 1)); done; printf ' } }\n'; } > "$WL/a.js" +LE=$("$BIN" "$WL" --deps --no-cache 2>/dev/null | grep -o '/dev/null | grep -c 'why="extract-partial"' | tr -d '[:space:]' || true) +QF=$("$BIN" "$WB/255" --skipped --no-cache 2>/dev/null | grep -c 'why="extract-partial"' | tr -d '[:space:]' || true) +if [ "$LE" = "0" ] && [ "$LF" = "1" ] && [ "$QF" = "0" ]; then + ok "J7: disclosure differences hold — silent when the edge was kept (255 blocks), loud when the bound cut an unreachable import (class method)" +else + no "J7: class-method-under-300: edges=$LE partial=$LF (want 0/1); 255-blocks partial=$QF (want 0)" +fi + +# (iii) QUIETER, and no import in the file at all: 300 nested blocks around `foo( "x" )` (a call the Web pattern +# captures and the text gate drops), and 300 nested #ifdef around `#pragma once` (captured by the C/C++ +# preproc_call pattern, dropped by the #import gate). The walk announced on any too-deep CONTAINER, and the +# first version of this arm announced because the bound test ran before the text gate; both said "an import +# was cut" when nothing import-shaped existed. The text gate now runs first, so neither does. +WQ="$TMP/wquiet"; mkdir -p "$WQ/js" "$WQ/c" +{ i=0; while [ "$i" -lt 300 ]; do printf '{'; i=$((i + 1)); done + printf ' foo("x"); '; i=0; while [ "$i" -lt 300 ]; do printf '}'; i=$((i + 1)); done; printf '\n'; } > "$WQ/js/a.js" +{ i=0; while [ "$i" -lt 300 ]; do printf '#ifdef G%s\n' "$i"; i=$((i + 1)); done + printf '#pragma once\n'; i=0; while [ "$i" -lt 300 ]; do printf '#endif\n'; i=$((i + 1)); done; } > "$WQ/c/a.c" +QJ=$("$BIN" "$WQ/js" --skipped --no-cache 2>/dev/null | grep -c 'why="extract-partial"' | tr -d '[:space:]' || true) +QC=$("$BIN" "$WQ/c" --skipped --no-cache 2>/dev/null | grep -c 'why="extract-partial"' | tr -d '[:space:]' || true) +if [ "$QJ" = "0" ] && [ "$QC" = "0" ]; then + ok "J7: no announcement when nothing import-shaped was cut (foo( \"x\" ) under 300 blocks, #pragma once under 300 #ifdef)" +else + no "J7: non-import under a too-deep nest announced an import cut: js=$QJ c=$QC (want 0/0)" +fi + +# ── J8: cache round-trip and a pre-change cache ───────────────────────────────────────────────────── +WC="$TMP/web.cache.bin" +"$BIN" "$WEB" --cache="$WC" --deps > "$TMP/web.cold.xml" 2>/dev/null +"$BIN" "$WEB" --cache="$WC" --deps > "$TMP/web.warm.xml" 2>/dev/null +if cmp -s "$TMP/web.cold.xml" "$TMP/web.warm.xml" && cmp -s "$TMP/web.cold.xml" "$TMP/web.deps.xml"; then + ok "J8: warm == cold == --no-cache on --deps for the JS/TS fixture (isLazy round-trips)" +else + no "J8: JS/TS --deps differs between cold, warm and --no-cache" +fi +if [ -n "$BASE_BIN" ] && [ -x "$BASE_BIN" ]; then + WP="$TMP/web.pre.bin"; rm -f "$WP" + "$BASE_BIN" "$WEB" --cache="$WP" --deps >/dev/null 2>&1 + cp "$WP" "$TMP/web.pre.bin.copy" + "$BIN" "$WEB" --cache="$WP" --deps > "$TMP/web.prewarm.xml" 2>/dev/null + if cmp -s "$TMP/web.prewarm.xml" "$TMP/web.deps.xml" && cmp -s "$WP" "$TMP/web.pre.bin.copy"; then + ok "J8: a pre-change binary's JS/TS cache is accepted unchanged and reads identically — no kParserVer bump is owed" + else + no "J8: a pre-change JS/TS cache reads differently or was rewritten — extraction output changed, kParserVer must be bumped" + fi +else + skip "J8: pre-change-cache arm for JS/TS (set RIPWIRE_BASE_BIN= to run it)" +fi +"$BIN" "$WEB" --no-cache --deps > "$TMP/web.d2.xml" 2>/dev/null +if cmp -s "$TMP/web.deps.xml" "$TMP/web.d2.xml"; then + ok "J8: deterministic (two --no-cache JS/TS runs identical)" +else + no "J8: non-deterministic JS/TS output" +fi + +# THE GO SLICE (arms 14-19). Go's import_declaration rides `(import_declaration) @import.path` and the +# DepDialect::Go normaliser; the Include target is the declaration CLAUSE, exactly as the removed walk branch +# built it, so every row below is the byte-for-byte row the extractor emitted (read off the pre-change binary, +# not predicted). A GROUPED import is ONE row whose text is the whole group cut at 96 bytes — resolve.h's +# resolveGoImport documents that — and splitting it per spec would ADD rows, which is exactly what arm 15 +# would show as a diff. +# ═══════════════════════════════════════════════════════════════════════════════════════════════════ +GO="$TMP/go"; mkdir -p "$GO/cmd" "$GO/pkg/util" "$GO/replaced" +printf 'module example.com/app\n\ngo 1.21\n\nrequire example.com/ext v0.0.0\n\nreplace example.com/ext => ./replaced\n' > "$GO/go.mod" +printf 'package ext\n\nfunc Ext() int { return 1 }\n' > "$GO/replaced/ext.go" +printf 'package util\n\nfunc Helper() int { return 2 }\n' > "$GO/pkg/util/util.go" +cat > "$GO/main.go" <<'GOSRC' +package main + +import "fmt" +import f2 "os" +import . "strings" +import _ "embed" +import ( + "example.com/app/pkg/util" + alias "example.com/ext" + . "math" + _ "net/http/pprof" + "strconv" // trailing comment +) + +import "C" + +func main() { + fmt.Println(util.Helper(), alias.Ext(), f2.Args, strconv.Itoa(1), ToUpper("x"), Abs(1)) +} +GOSRC +# a TAB after the keyword keeps the word `import` (the keyword is dropped at a SPACE); two spaces leave one +# leading space; a trailing `;` is trimmed; a raw-string path keeps its backticks. +printf 'package main\n\nimport\t"os"\nimport "io";\nimport `raw/path`\n\nfunc t() { _ = os.Args }\n' > "$GO/cmd/tab.go" +cat > "$GO/cmd/long.go" <<'GOSRC' +package main + +import ( + "github.com/very/long/module/path/that/goes/on/and/on/and/on/forever/and/ever/amen/pkg1" + "github.com/very/long/module/path/that/goes/on/and/on/and/on/forever/and/ever/amen/pkg2" +) + +func l() {} +GOSRC +# error recovery: a bare `import` swallowing the next lines is still ONE import_declaration, and a broken +# function header does not stop the next import being recovered. +printf 'package main\n\nimport\nfunc b( {\nimport "after/error"\n' > "$GO/cmd/broken.go" +printf 'package main\n\nfunc f( {\nimport "x"\n}\n' > "$GO/cmd/err.go" +# the 96-byte cut lands mid multibyte sequence on purpose: the pre-change bytes are kept, not "fixed". +printf 'package main\nimport "é世界/pkg"; import "z"\nimport ("é世界/pkg/aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"\n)\n' > "$GO/cmd/utf.go" +# a definition named like the last path element of utf.go's first import: --uses keys on an INDEXED name, so +# arm 18 needs one (`fmt`, `os` have no definition in the tree and answer found=0). +printf 'package main\n\nfunc pkg() {}\n' > "$GO/cmd/pkgdef.go" +# arm 16's negative: an import inside a function body is not a file-scope declaration and was never an edge. +printf 'package main\n\nfunc a() {\n\timport "inbody"\n}\n' > "$GO/cmd/nested.go" + +"$BIN" "$GO" --deps --no-cache > "$TMP/go.xml" 2>/dev/null +grep -o ' "$TMP/go.rows" +cat > "$TMP/go.want" <<'GOWANT' + "$WS/svc/pkg/handle.go" +printf 'module example.com/svc\n\ngo 1.21\n' > "$WS/svc/go.mod" +printf 'module example.com/cli\n\ngo 1.21\n\nreplace example.com/svc => ../svc\n' > "$WS/cli/go.mod" +printf 'package main\n\nimport svc "example.com/svc/pkg"\n\nfunc runGoMain() int { return svc.GoHandle() }\n' > "$WS/cli/main.go" +printf 'module example.com/cli2\n\ngo 1.21\n\nreplace example.com/svc => ../svc\n' > "$WS/cli2/go.mod" +printf 'package main\n\nimport (\n\tsvc "example.com/svc/pkg"\n\t"fmt"\n)\n\nfunc runGrouped() int { fmt.Println(); return svc.GoHandle() }\n' > "$WS/cli2/main.go" +printf 'module example.com/clibogus\n\ngo 1.21\n\nreplace example.com/svc => ../svc/NONEXISTENT\n' > "$WS/clibogus/go.mod" +printf 'package main\n\nimport svc "example.com/svc/pkg"\n\nfunc runBogus() int { return svc.GoHandle() }\n' > "$WS/clibogus/main.go" +"$BIN" "$WS/svc" "$WS/cli" "$WS/cli2" --deps --no-cache > "$TMP/gows.xml" 2>/dev/null +if grep -q ']*' "$TMP/gows.xml")" +fi +# the evidence-driven negative: the same import under a replace whose target does not exist resolves to nothing. +"$BIN" "$WS/svc" "$WS/clibogus" --deps --no-cache > "$TMP/gobogus.xml" 2>/dev/null +if grep -q ']*' "$TMP/gobogus.xml")" +fi + +# ── arm 18: the use-site half ───────────────────────────────────────────────────────────────────────── +GUSES="$("$BIN" "$GO" --uses=pkg --no-cache 2>/dev/null)" +if printf '%s' "$GUSES" | grep -q ']*' | head -3)" +fi + +# ── arm 19: Go cache round-trip + determinism ──────────────────────────────────────────────────────── +GCACHE="$TMP/go.cache" +"$BIN" "$GO" --cache="$GCACHE" --deps > "$TMP/gocold.xml" 2>/dev/null +"$BIN" "$GO" --cache="$GCACHE" --deps > "$TMP/gowarm.xml" 2>/dev/null +if cmp -s "$TMP/gocold.xml" "$TMP/gowarm.xml" && cmp -s "$TMP/gocold.xml" "$TMP/go.xml"; then + ok "Go: warm == cold == --no-cache on --deps (the Go Include round-trip is byte-identical)" +else + no "Go: --deps differs between cold, warm and --no-cache" +fi +if [ -n "${BASE_BIN:-}" ] && [ -x "${BASE_BIN:-}" ]; then + "$BASE_BIN" "$GO" --deps --no-cache > "$TMP/gobase.xml" 2>/dev/null + if cmp -s "$TMP/gobase.xml" "$TMP/go.xml"; then + ok "Go: --deps over the fixture is byte-identical to the pre-change binary's" + else + no "Go: --deps differs from the pre-change binary — dependency output changed" + fi + # and the Go half of arm 13: a cache written by the pre-change binary over the Go fixture is ACCEPTED + # (file untouched) and reads identically, so no kParserVer bump is owed for the Go records either. + GPRE="$TMP/gopre.bin"; rm -f "$GPRE" + "$BASE_BIN" "$GO" --cache="$GPRE" --deps >/dev/null 2>&1 + cp "$GPRE" "$TMP/gopre.bin.copy" + "$BIN" "$GO" --cache="$GPRE" --deps > "$TMP/gopre.xml" 2>/dev/null + if cmp -s "$TMP/gopre.xml" "$TMP/go.xml" && cmp -s "$GPRE" "$TMP/gopre.bin.copy"; then + ok "Go: a pre-change binary's cache over the Go fixture is accepted untouched and reads identically — no kParserVer bump owed" + else + no "Go: a pre-change cache reads differently or was rewritten — extraction output changed, kParserVer must be bumped" + fi +else + skip "Go: pre-change comparison (set RIPWIRE_BASE_BIN= to run it)" +fi + # ── well-formed XML ─────────────────────────────────────────────────────────────────────────────────── if command -v xmllint >/dev/null 2>&1; then if xmllint --noout "$TMP/cold.xml" 2>/dev/null; then ok "xml well-formed"; else no "xml malformed"; fi diff --git a/test/legochargefix/pipeline.cpp b/test/legochargefix/pipeline.cpp new file mode 100644 index 000000000..c8c79f6bc --- /dev/null +++ b/test/legochargefix/pipeline.cpp @@ -0,0 +1,59 @@ +// legochargefix/pipeline.cpp — the ranked rows: free functions the render-stage query matches by name. +#include "stages.h" + +int arrangeRenderStageWidgetsLayout( LayoutRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeLayoutWidgets( regionWidth, regionHeight, "Layout" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsLayout( LayoutRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowLayoutWidgets( regionWidth, regionHeight, "Layout" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsLayout( LayoutRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshLayoutWidgets( regionWidth, regionHeight, "Layout" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsLayout( LayoutRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintLayoutWidgets( regionWidth, regionHeight, "Layout" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsCompositor( CompositorRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeCompositorWidgets( regionWidth, regionHeight, "Compositor" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsCompositor( CompositorRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowCompositorWidgets( regionWidth, regionHeight, "Compositor" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsCompositor( CompositorRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshCompositorWidgets( regionWidth, regionHeight, "Compositor" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsCompositor( CompositorRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintCompositorWidgets( regionWidth, regionHeight, "Compositor" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsStyles( StylesRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeStylesWidgets( regionWidth, regionHeight, "Styles" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsStyles( StylesRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowStylesWidgets( regionWidth, regionHeight, "Styles" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsStyles( StylesRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshStylesWidgets( regionWidth, regionHeight, "Styles" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsStyles( StylesRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintStylesWidgets( regionWidth, regionHeight, "Styles" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsScreen( ScreenRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeScreenWidgets( regionWidth, regionHeight, "Screen" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsScreen( ScreenRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowScreenWidgets( regionWidth, regionHeight, "Screen" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsScreen( ScreenRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshScreenWidgets( regionWidth, regionHeight, "Screen" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsScreen( ScreenRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintScreenWidgets( regionWidth, regionHeight, "Screen" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsMessage( MessageRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeMessageWidgets( regionWidth, regionHeight, "Message" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsMessage( MessageRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowMessageWidgets( regionWidth, regionHeight, "Message" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsMessage( MessageRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshMessageWidgets( regionWidth, regionHeight, "Message" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsMessage( MessageRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintMessageWidgets( regionWidth, regionHeight, "Message" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsBinding( BindingRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeBindingWidgets( regionWidth, regionHeight, "Binding" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsBinding( BindingRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowBindingWidgets( regionWidth, regionHeight, "Binding" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsBinding( BindingRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshBindingWidgets( regionWidth, regionHeight, "Binding" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsBinding( BindingRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintBindingWidgets( regionWidth, regionHeight, "Binding" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsTimer( TimerRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeTimerWidgets( regionWidth, regionHeight, "Timer" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsTimer( TimerRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowTimerWidgets( regionWidth, regionHeight, "Timer" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsTimer( TimerRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshTimerWidgets( regionWidth, regionHeight, "Timer" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsTimer( TimerRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintTimerWidgets( regionWidth, regionHeight, "Timer" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsReactive( ReactiveRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeReactiveWidgets( regionWidth, regionHeight, "Reactive" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsReactive( ReactiveRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowReactiveWidgets( regionWidth, regionHeight, "Reactive" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsReactive( ReactiveRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshReactiveWidgets( regionWidth, regionHeight, "Reactive" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsReactive( ReactiveRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintReactiveWidgets( regionWidth, regionHeight, "Reactive" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsScroll( ScrollRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeScrollWidgets( regionWidth, regionHeight, "Scroll" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsScroll( ScrollRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowScrollWidgets( regionWidth, regionHeight, "Scroll" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsScroll( ScrollRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshScrollWidgets( regionWidth, regionHeight, "Scroll" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsScroll( ScrollRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintScrollWidgets( regionWidth, regionHeight, "Scroll" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsFocus( FocusRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeFocusWidgets( regionWidth, regionHeight, "Focus" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsFocus( FocusRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowFocusWidgets( regionWidth, regionHeight, "Focus" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsFocus( FocusRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshFocusWidgets( regionWidth, regionHeight, "Focus" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsFocus( FocusRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintFocusWidgets( regionWidth, regionHeight, "Focus" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsMount( MountRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeMountWidgets( regionWidth, regionHeight, "Mount" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsMount( MountRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowMountWidgets( regionWidth, regionHeight, "Mount" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsMount( MountRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshMountWidgets( regionWidth, regionHeight, "Mount" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsMount( MountRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintMountWidgets( regionWidth, regionHeight, "Mount" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsTheme( ThemeRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeThemeWidgets( regionWidth, regionHeight, "Theme" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsTheme( ThemeRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowThemeWidgets( regionWidth, regionHeight, "Theme" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsTheme( ThemeRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshThemeWidgets( regionWidth, regionHeight, "Theme" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsTheme( ThemeRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintThemeWidgets( regionWidth, regionHeight, "Theme" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsCursor( CursorRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeCursorWidgets( regionWidth, regionHeight, "Cursor" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsCursor( CursorRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowCursorWidgets( regionWidth, regionHeight, "Cursor" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsCursor( CursorRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshCursorWidgets( regionWidth, regionHeight, "Cursor" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsCursor( CursorRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintCursorWidgets( regionWidth, regionHeight, "Cursor" ); return regionWidth + regionHeight; } +int arrangeRenderStageWidgetsRegion( RegionRenderStage& stage, int regionWidth, int regionHeight ) { stage.arrangeRegionWidgets( regionWidth, regionHeight, "Region" ); return regionWidth + regionHeight; } +int reflowRenderStageWidgetsRegion( RegionRenderStage& stage, int regionWidth, int regionHeight ) { stage.reflowRegionWidgets( regionWidth, regionHeight, "Region" ); return regionWidth + regionHeight; } +int refreshRenderStageWidgetsRegion( RegionRenderStage& stage, int regionWidth, int regionHeight ) { stage.refreshRegionWidgets( regionWidth, regionHeight, "Region" ); return regionWidth + regionHeight; } +int repaintRenderStageWidgetsRegion( RegionRenderStage& stage, int regionWidth, int regionHeight ) { stage.repaintRegionWidgets( regionWidth, regionHeight, "Region" ); return regionWidth + regionHeight; } diff --git a/test/legochargefix/stages.h b/test/legochargefix/stages.h new file mode 100644 index 000000000..fd782a5ad --- /dev/null +++ b/test/legochargefix/stages.h @@ -0,0 +1,787 @@ +// legochargefix/stages.h — lego-charge gate fixture (forsectioncollapsecheck.sh arm (13)): a textual-like corpus whose +// --for answer names many interfaces with implementors, so the FULL render (~8 KB) is bigger than the default +// budget while the default answer serves it as a ~150 B counted stub. Generated once; plain C++ on purpose. + +class LayoutRenderStage +{ +public: + virtual ~LayoutRenderStage() = default; + virtual void arrangeLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class LayoutRenderStageImpl1 : public LayoutRenderStage +{ +public: + void arrangeLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class LayoutRenderStageImpl2 : public LayoutRenderStage +{ +public: + void arrangeLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class LayoutRenderStageImpl3 : public LayoutRenderStage +{ +public: + void arrangeLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class LayoutRenderStageImpl4 : public LayoutRenderStage +{ +public: + void arrangeLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveLayoutWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class CompositorRenderStage +{ +public: + virtual ~CompositorRenderStage() = default; + virtual void arrangeCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class CompositorRenderStageImpl1 : public CompositorRenderStage +{ +public: + void arrangeCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class CompositorRenderStageImpl2 : public CompositorRenderStage +{ +public: + void arrangeCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class CompositorRenderStageImpl3 : public CompositorRenderStage +{ +public: + void arrangeCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class CompositorRenderStageImpl4 : public CompositorRenderStage +{ +public: + void arrangeCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveCompositorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class StylesRenderStage +{ +public: + virtual ~StylesRenderStage() = default; + virtual void arrangeStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class StylesRenderStageImpl1 : public StylesRenderStage +{ +public: + void arrangeStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class StylesRenderStageImpl2 : public StylesRenderStage +{ +public: + void arrangeStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class StylesRenderStageImpl3 : public StylesRenderStage +{ +public: + void arrangeStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class StylesRenderStageImpl4 : public StylesRenderStage +{ +public: + void arrangeStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveStylesWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ScreenRenderStage +{ +public: + virtual ~ScreenRenderStage() = default; + virtual void arrangeScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class ScreenRenderStageImpl1 : public ScreenRenderStage +{ +public: + void arrangeScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ScreenRenderStageImpl2 : public ScreenRenderStage +{ +public: + void arrangeScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ScreenRenderStageImpl3 : public ScreenRenderStage +{ +public: + void arrangeScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ScreenRenderStageImpl4 : public ScreenRenderStage +{ +public: + void arrangeScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveScreenWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class MessageRenderStage +{ +public: + virtual ~MessageRenderStage() = default; + virtual void arrangeMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class MessageRenderStageImpl1 : public MessageRenderStage +{ +public: + void arrangeMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class MessageRenderStageImpl2 : public MessageRenderStage +{ +public: + void arrangeMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class MessageRenderStageImpl3 : public MessageRenderStage +{ +public: + void arrangeMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class MessageRenderStageImpl4 : public MessageRenderStage +{ +public: + void arrangeMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveMessageWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class BindingRenderStage +{ +public: + virtual ~BindingRenderStage() = default; + virtual void arrangeBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class BindingRenderStageImpl1 : public BindingRenderStage +{ +public: + void arrangeBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class BindingRenderStageImpl2 : public BindingRenderStage +{ +public: + void arrangeBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class BindingRenderStageImpl3 : public BindingRenderStage +{ +public: + void arrangeBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class BindingRenderStageImpl4 : public BindingRenderStage +{ +public: + void arrangeBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveBindingWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class TimerRenderStage +{ +public: + virtual ~TimerRenderStage() = default; + virtual void arrangeTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class TimerRenderStageImpl1 : public TimerRenderStage +{ +public: + void arrangeTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class TimerRenderStageImpl2 : public TimerRenderStage +{ +public: + void arrangeTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class TimerRenderStageImpl3 : public TimerRenderStage +{ +public: + void arrangeTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class TimerRenderStageImpl4 : public TimerRenderStage +{ +public: + void arrangeTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveTimerWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ReactiveRenderStage +{ +public: + virtual ~ReactiveRenderStage() = default; + virtual void arrangeReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class ReactiveRenderStageImpl1 : public ReactiveRenderStage +{ +public: + void arrangeReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ReactiveRenderStageImpl2 : public ReactiveRenderStage +{ +public: + void arrangeReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ReactiveRenderStageImpl3 : public ReactiveRenderStage +{ +public: + void arrangeReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ReactiveRenderStageImpl4 : public ReactiveRenderStage +{ +public: + void arrangeReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveReactiveWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ScrollRenderStage +{ +public: + virtual ~ScrollRenderStage() = default; + virtual void arrangeScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class ScrollRenderStageImpl1 : public ScrollRenderStage +{ +public: + void arrangeScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ScrollRenderStageImpl2 : public ScrollRenderStage +{ +public: + void arrangeScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ScrollRenderStageImpl3 : public ScrollRenderStage +{ +public: + void arrangeScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ScrollRenderStageImpl4 : public ScrollRenderStage +{ +public: + void arrangeScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveScrollWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class FocusRenderStage +{ +public: + virtual ~FocusRenderStage() = default; + virtual void arrangeFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class FocusRenderStageImpl1 : public FocusRenderStage +{ +public: + void arrangeFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class FocusRenderStageImpl2 : public FocusRenderStage +{ +public: + void arrangeFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class FocusRenderStageImpl3 : public FocusRenderStage +{ +public: + void arrangeFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class FocusRenderStageImpl4 : public FocusRenderStage +{ +public: + void arrangeFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveFocusWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class MountRenderStage +{ +public: + virtual ~MountRenderStage() = default; + virtual void arrangeMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class MountRenderStageImpl1 : public MountRenderStage +{ +public: + void arrangeMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class MountRenderStageImpl2 : public MountRenderStage +{ +public: + void arrangeMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class MountRenderStageImpl3 : public MountRenderStage +{ +public: + void arrangeMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class MountRenderStageImpl4 : public MountRenderStage +{ +public: + void arrangeMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveMountWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ThemeRenderStage +{ +public: + virtual ~ThemeRenderStage() = default; + virtual void arrangeThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class ThemeRenderStageImpl1 : public ThemeRenderStage +{ +public: + void arrangeThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ThemeRenderStageImpl2 : public ThemeRenderStage +{ +public: + void arrangeThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ThemeRenderStageImpl3 : public ThemeRenderStage +{ +public: + void arrangeThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class ThemeRenderStageImpl4 : public ThemeRenderStage +{ +public: + void arrangeThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveThemeWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class CursorRenderStage +{ +public: + virtual ~CursorRenderStage() = default; + virtual void arrangeCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class CursorRenderStageImpl1 : public CursorRenderStage +{ +public: + void arrangeCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class CursorRenderStageImpl2 : public CursorRenderStage +{ +public: + void arrangeCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class CursorRenderStageImpl3 : public CursorRenderStage +{ +public: + void arrangeCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class CursorRenderStageImpl4 : public CursorRenderStage +{ +public: + void arrangeCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveCursorWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class RegionRenderStage +{ +public: + virtual ~RegionRenderStage() = default; + virtual void arrangeRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void reflowRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void refreshRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void repaintRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void measureRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; + virtual void resolveRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) = 0; +}; + +class RegionRenderStageImpl1 : public RegionRenderStage +{ +public: + void arrangeRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class RegionRenderStageImpl2 : public RegionRenderStage +{ +public: + void arrangeRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class RegionRenderStageImpl3 : public RegionRenderStage +{ +public: + void arrangeRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; + +class RegionRenderStageImpl4 : public RegionRenderStage +{ +public: + void arrangeRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void reflowRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void refreshRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void repaintRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void measureRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} + void resolveRegionWidgets( int regionWidth, int regionHeight, const char* stageLabel ) override {} +}; diff --git a/test/listingpagingcheck.sh b/test/listingpagingcheck.sh index 5a352bc84..99f623563 100755 --- a/test/listingpagingcheck.sh +++ b/test/listingpagingcheck.sh @@ -506,5 +506,244 @@ print( " PASS (F) mutation: a drift= that followed the window IS caught by the PY [ $? = 0 ] || fail=1 +# ── ARM B of the paging spec (upstream issue #294, corrected scope): the BUDGETED-bundle continuation ── +# `--for=TASK --limit=N` already pages (the file-grain widening page, forwidencheck owns it). What does +# NOT exist yet: the byte-budgeted bundle (--token-budget) under --offset/--limit — today the pair is +# refused, so an agent whose bundle was cut can only re-run blind. This arm is the SEAM contract for the +# budget-mode continuation once it exists: --offset=N --limit=M must be the EXACT continuation of the +# previous page over the ranked candidate set — page[0:3] + page[3:6] == page[0:6] in candidates, rank +# order, no gap, no overlap (the same contract arm (A) holds for rows, here measured on the sig rows +# of the paged budgeted bundle). RED on the pre-feature binary is the intended state: gate written +# BEFORE the code, red vs the pre-change binary — today both paged runs refuse and this records that. +echo "=== (G) budgeted-bundle candidate continuation: page[0:3] + page[3:6] == page[0:6] ===" +# at a NO-TRIM budget: the page's own legend rides the budget, so a small budget trims within every +# page and the pure-window identity (page[0:3]+page[3:6] == page[0:6]) cannot hold — the identity is +# the WINDOW's; the trim path is the (G4) trimming walk below (the #362 review's ask). +# on src/ (cliff 40 > 6: the single-tier clamp never fires; the repo ROOT clamps every head +# window to the 1-row cliff — a correct page, but not one this pure-window identity can test) +run "$ROOT/src" g_budget_p0 --for="rank symbols" --token-budget=100000 --limit=3 --offset=0 +run "$ROOT/src" g_budget_p3 --for="rank symbols" --token-budget=100000 --limit=3 --offset=3 +run "$ROOT/src" g_budget_p6 --for="rank symbols" --token-budget=100000 --limit=6 --offset=0 + +python3 - "$TMP" <<'PY' +import os, sys, xml.etree.ElementTree as ET + +tmp = sys.argv[1] +problems = [] + +def sigs(name): + path = os.path.join(tmp, name) + if not os.path.exists(path) or os.path.getsize(path) == 0: + problems.append(f"{name}: no output — the paged budgeted bundle does not exist yet " + "(today --token-budget refuses --offset/--limit; that refusal IS this arm's recorded red)") + return None + raw = open(path, encoding="utf-8", errors="replace").read() + try: + root = ET.fromstring(raw) + except ET.ParseError as e: + problems.append(f"{name}: not parseable XML ({e})"); return None + q = [k for k in ("shown", "total", "capped", "has_more", "next_offset") if k not in root.attrib] + if q: + problems.append(f'{name}: paged budgeted root lacks the quintet (missing: {", ".join(q)})'); return None + # the bundle's sig rows are elements (serialize.h's flat lens path) — the arm's first + # cut guessed /, matched nothing, and produced a VACUOUS green (0+0 == 0); a seam that + # compares no rows proves nothing, so assert the rows EXIST too. + rows = [ el.attrib.get("n") or el.attrib.get("name") or el.text or "" for el in root.iter() if el.tag == "d" ] + if not rows: + problems.append(f"{name}: the page carries no rows — the seam comparison below would be vacuous") + return rows + +p0 = sigs("g_budget_p0"); p3 = sigs("g_budget_p3"); p6 = sigs("g_budget_p6") +if p0 is not None and p3 is not None and p6 is not None: + joined, whole = p0 + p3, p6 + if len(joined) != len(whole): + problems.append(f"seam: page[0:3]+page[3:6] gives {len(joined)} rows, page[0:6] gives {len(whole)} — rows lost or duplicated across the seam") + else: + for i, (a, b) in enumerate(zip(joined, whole)): + if a != b: + problems.append(f"seam: row {i} disagrees across the seam (pages say {a!r}, whole says {b!r}) — not the exact continuation") + break + overlap = set(p0) & set(p3) + if overlap: + problems.append(f"seam: pages overlap on {sorted(overlap)[:3]} — a candidate served twice is a page that lies") + if not problems: + print(f" PASS (G) seam: {len(whole)} candidate rows, two pages == the whole, rank order, no overlap") + +# ARM B2 (issue #294 follow-up): the candidate page carries next= — the RESUMABLE ARGV, the same +# contract the file page's next= serves (forwidencheck arm (5) reproduces its page from it). The handle +# must name the offset the machine attribute next_offset= declares, so a client that pastes it walks the +# exact continuation. RED until the page grows the attribute (next_offset= alone is machine-only). +import re as _re +next_attr = None; next_offset_attr = None +for name in ("g_budget_p0", "g_budget_p3"): + path = os.path.join(tmp, name) + if not os.path.exists(path) or os.path.getsize(path) == 0: continue + raw = open(path, encoding="utf-8", errors="replace").read() + # scope to the ROOT OPENING TAG (rows carry their own next= expand hints — the arm's first cut + # read the row's handle and mis-filed it as the page's) + o = raw.find( "", o ) + opentag = raw[ o : oend ] if o >= 0 and oend > o else "" + m = _re.search(r' next="([^"]*)"', opentag); n = _re.search(r' next_offset="(\d+)"', opentag) + t = _re.search(r' next_tier="([^"]*)"', opentag) # ruling suggestion 2: the tier the NEXT page carries + if m: next_attr = m.group(1) + if n: next_offset_attr = n.group(1) + if next_attr and next_offset_attr: + if ( "--offset=" + next_offset_attr ) not in next_attr: + problems.append(f"next= does not name the declared continuation: next={next_attr!r} vs next_offset={next_offset_attr}") + else: + print(f" PASS (G2) next= names the exact continuation (--offset={next_offset_attr})") + break +if t and t.group(1) not in ("head", "below-cliff"): + problems.append(f"next_tier={t.group(1)!r} is not one of the two tier values") +if not t and next_attr: + problems.append("next= present but next_tier= absent — the ruling's suggestion 2 (the handle must make the next page's tier visible)") +if not next_attr: + problems.append("the candidate page root carries no next= — the resumable argv handle the file page's next= contract serves (next_offset= alone is machine-only)") + +# mutation: the seam shape can fail — an overlapping pair of fabricated pages must be DETECTED +mut_a, mut_b = ["a", "b", "c"], ["b", "c", "d"] +if len(set(mut_a) & set(mut_b)) != 2: + problems.append("mutation: an overlapping fabricated page pair was NOT detected — the seam arm cannot fail") +else: + print(" PASS (G) mutation: an overlapping fabricated page pair IS detected") + +for p in problems: print(" FAIL " + p) +sys.exit(1 if problems else 0) +PY +[ $? = 0 ] && ok '(G) budgeted-bundle pages are the exact continuation (candidates, rank order, no overlap)' \ + || no '(G) the budgeted-bundle continuation seam (upstream issue #294) does not hold — see FAIL lines above' + +# ── (G3) END-TO-END WALK (issue #294 ruling, suggestion 5): every page concatenated equals the full ── +# ranked candidate list — no gap, no overlap, same order — and the tier sequence is head* then +# below-cliff* (exactly one boundary). Walked at a budget large enough that no within-page trim fires, +# so the row NAMES are the ranked set itself. The un-paged bundle's served rows (top-40 of the same +# order, same budget) must be a PREFIX of the walk. +echo "=== (G3) end-to-end walk: all pages concatenated == the full ranked candidate list ===" +WALK_BUDGET=100000 +# on the big src/ corpus: enough candidates (878 for this query) that the walk spans multiple pages +# and crosses the tier boundary — the root corpus' 5 candidates fit one page and prove nothing about seams. +run "$ROOT/src" g3_bundle --for="rank symbols" --token-budget=$WALK_BUDGET +offset=0; page=0 +while [ "$page" -lt 12 ]; do + run "$ROOT/src" "g3_walk_$page" --for="rank symbols" --token-budget=$WALK_BUDGET --limit=200 --offset=$offset + shown="$( attr "$TMP/g3_walk_$page" sigs shown )"; has_more="$( attr "$TMP/g3_walk_$page" sigs has_more )" + nextoff="$( attr "$TMP/g3_walk_$page" sigs next_offset )"; tier="$( attr "$TMP/g3_walk_$page" sigs tier )" + total="$( attr "$TMP/g3_walk_$page" sigs total )" + [ -n "$shown" ] || { no "(G3) walk page $page produced no parseable root"; break; } + echo "$tier" >> "$TMP/g3_tiers.txt" + echo "$shown" >> "$TMP/g3_shown.txt" + # rows carry n= AFTER other attributes (", "", raw, flags=re.S ) # the legend DEFINES in prose: never count prose as rows +# candidate identity is the (file, line, name) TRIPLE — same-name definitions are distinct +# candidates even in ONE file (DYNMAP_DEFINE_RANK_32 sits at two lines of dynamic_map.hpp), +# so name or (file, name) alone false-flags them as seam overlap. n=/p=/l= may sit in any +# order on the tag — three sub-extracts per tag. +for tag in re.finditer( r"]*>", raw ): + t = tag.group( 0 ) + n = re.search( r"\sn=\"([^\"]*)\"", t ) + p = re.search( r"\sp=\"([^\"]*)\"", t ) + l = re.search( r"\sl=\"([^\"]*)\"", t ) + print( ( p.group( 1 ) if p else "?" ) + "\x1f" + ( l.group( 1 ) if l else "?" ) + "\x1f" + ( n.group( 1 ) if n else "?" ) ) +' "$TMP/g3_walk_$page" >> "$TMP/g3_rows.txt" + page=$(( page + 1 )) + [ "$has_more" = "1" ] || break + offset="$nextoff" +done +walk_pages="$page" + +python3 - "$TMP" "$walk_pages" "$total" <<'PY' +import os, sys +tmp, pages, total = sys.argv[1], int(sys.argv[2]), int(sys.argv[3]) +problems = [] +rows = [ l.rstrip("\n") for l in open(os.path.join(tmp, "g3_rows.txt")) if l.strip() ] +tiers = [ l.rstrip("\n") for l in open(os.path.join(tmp, "g3_tiers.txt")) if l.strip() ] +shown_sum = sum( int( x ) for x in open(os.path.join(tmp, "g3_shown.txt")) if x.strip() ) +dups = len(rows) - len(set(rows)) +if dups: + problems.append(f"walk: {dups} duplicate candidates across pages — a row served twice is a page that lies") +# the tiling measure is the pages' own shown= arithmetic (candidates SERVED); the ) is counted as served but emits no row, +# so shown-sum == total is the contract, and the row list tiles the named-def subset in order. +# the tile under pseudo-symbols: a candidate slot can be SERVED without printing a row (), +# so shown-sum <= total with the difference consumed by pseudo-slots; the coherence is rows-extracted +# == shown-counted plus no duplicates plus the prefix below. +if shown_sum > total: + problems.append(f"walk: shown sums to {shown_sum} vs total={total} — the pages over-serve the candidate set") +if len(rows) != shown_sum: + problems.append(f"walk: {len(rows)} rows extracted vs shown summing {shown_sum} — the count and the rows disagree") +# tier sequence: all head pages precede all below-cliff pages (exactly one boundary) +seen_below = False +for i, t in enumerate(tiers): + if t not in ("head", "below-cliff"): + problems.append(f"walk page {i}: tier={t!r} is not one of the two values"); break + if t == "below-cliff": seen_below = True + elif seen_below: + problems.append(f"walk page {i}: tier=head AFTER a below-cliff page — the boundary moved mid-walk"); break +# the un-paged bundle's served rows are a PREFIX of the walk (same order, same budget) +bp = os.path.join(tmp, "g3_bundle") +if os.path.exists(bp) and os.path.getsize(bp) > 0: + import re + raw = open(bp, encoding="utf-8", errors="replace").read() + raw = re.sub( r'', '', raw, flags=re.S ) # the legend's prose is not a row + bundle = [] + for m in re.finditer( r']*>', raw ): + t = m.group( 0 ) + n = re.search( r'\sn="([^"]*)"', t ) + pp = re.search( r'\sp="([^"]*)"', t ) + ll = re.search( r'\sl="([^"]*)"', t ) + bundle.append( ( pp.group(1) if pp else "?" ) + "\x1f" + ( ll.group(1) if ll else "?" ) + "\x1f" + ( n.group(1) if n else "?" ) ) + if rows[:len(bundle)] != bundle: + problems.append("the un-paged bundle's served rows are NOT a prefix of the walk — the page and the bundle answer different orders") + else: + print(f" PASS (G3) the un-paged bundle's {len(bundle)} rows are a prefix of the walk (same order)") +for p_ in problems: print(" FAIL " + p_) +print(f" .. (G3) walked {pages} pages, {len(rows)} rows, total={total}, tiers: {' -> '.join(tiers)}") +sys.exit(1 if problems else 0) +PY +[ $? = 0 ] && ok '(G3) every page concatenated == the full ranked candidate list (no gap, no overlap, one tier boundary)' \ + || no '(G3) the end-to-end walk (issue #294 ruling, suggestion 5) does not hold — see FAIL lines above' + +# ── (G4) THE TRIMMING WALK (#362 review, item 2): the same walk at a budget that TRIMS within ── +# every page, asserting NO DUPLICATE candidate across the seam. The resume point is the last +# PRINTED row's candidate index + 1 (a pseudo-symbol consumes a slot without printing a row), so +# a served-count arithmetic lands one short and the next page re-serves a row — the walk catches it. +echo "=== (G4) trimming walk: a budget that trims, no duplicate candidate across the seam ===" +: > "$TMP/g4_rows.txt" +offset=0; page=0 +while [ "$page" -lt 12 ]; do + run "$ROOT/src" "g4_walk_$page" --for="rank symbols" --token-budget=600 --limit=40 --offset=$offset + has_more="$( attr "$TMP/g4_walk_$page" sigs has_more )" + nextoff="$( attr "$TMP/g4_walk_$page" sigs next_offset )" + [ -n "$nextoff" ] || { no "(G4) walk page $page produced no parseable root"; break; } + python3 -c ' +import re, sys +raw = open( sys.argv[1], encoding="utf-8", errors="replace" ).read() +raw = re.sub( r"", "", raw, flags=re.S ) # the legend DEFINES in prose: never count prose as rows +for tag in re.finditer( r"]*>", raw ): + t = tag.group( 0 ) + n = re.search( r"\sn=\"([^\"]*)\"", t ) + p = re.search( r"\sp=\"([^\"]*)\"", t ) + l = re.search( r"\sl=\"([^\"]*)\"", t ) + print( ( p.group( 1 ) if p else "?" ) + "\x1f" + ( l.group( 1 ) if l else "?" ) + "\x1f" + ( n.group( 1 ) if n else "?" ) ) +' "$TMP/g4_walk_$page" >> "$TMP/g4_rows.txt" + page=$(( page + 1 )) + [ "$has_more" = "1" ] || break + offset="$nextoff" +done +TRIM_DUPS=$( python3 - "$TMP/g4_rows.txt" < "$MF/d$i/s$i.sh" + printf 'def ok(x):\n return x\nok("py %s")\n' "$i" > "$MF/d$i/p$i.py" +done +memoBad=0 +for i in 1 2 9 17 18; do + for f in "d$i/s$i.sh" "d$i/p$i.py"; do + out="$( "$BIN" "$MF" --no-cache --callers="$f:ok" 2>/dev/null )" + if printf '%s' "$out" | grep -q 'count="1"' && printf '%s' "$out" | grep -q "p=\"$f:1\""; then :; else + memoBad=1; no "(10) $f:ok should have exactly its own file's module scope as caller: $( printf '%s' "$out" | grep -o ']*>' | head -1 )" + fi + done +done +rm -rf "$MF" +if [ "$memoBad" -eq 0 ]; then ok "(10) 36 same-named defs over two languages: every file's module-scope call binds its own file's ok (10 files sampled)"; fi + [ "$fail" -eq 0 ] && echo "ALL PASS" || echo "SOME FAILED" exit "$fail" diff --git a/test/mapinventorycheck.sh b/test/mapinventorycheck.sh new file mode 100755 index 000000000..0f627ed44 --- /dev/null +++ b/test/mapinventorycheck.sh @@ -0,0 +1,230 @@ +#!/usr/bin/env bash +# mapinventorycheck.sh — the default map's inventory tier (): no indexed file is silently absent from an +# orient map, and the program entry points are named. +# +# WHY. The ranked map prints the top-k (200) symbols, which covers ~30-90 files of a few hundred. On the round-1 +# comparison table that cut was the whole orient miss class (K19): tmux input.c / key-bindings.c, hono's +# validator/, textual's drivers/ and command.py, htop's SignalsPanel.c — every one an indexed file ranked just below +# the cut, with NOTHING in the answer saying the file existed. Entry points are worse: `main` has no in-repo caller, +# so PageRank puts it at the very bottom (htop main #2548 of 3206, tmux main #3220 of 6361) and no re-rank of the +# map can surface it. The tier appends, after the ranked rows and without touching them: rows for program +# entries by name convention, and rows naming (code) or counting (tests/docs/examples/config) every file the +# ranked rows did not show, grouped by directory. +# +# Fixture test/mapinvfix: C, Python, Go, Rust, Java and TS files; a test-dir main, an examples main, a Python main +# with a real caller (NOT an entry), a cli main reached from __main__.py (NOT an entry: the __main__ scope is). +set -u + +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" +TMP="$( mktemp -d )"; trap 'rm -rf "$TMP"' EXIT +FAILED=0 +fail() { printf ' FAIL %s\n' "$*"; FAILED=$(( FAILED + 1 )); } +ok() { printf ' PASS %s\n' "$*" || { FAILED=$(( FAILED + 1 )); printf ' FAIL could not write the PASS line for: %s\n' "$*"; }; return 0; } +[ -x "$BIN" ] || { echo "mapinventorycheck: no binary at $BIN — build first"; exit 2; } +echo "mapinventorycheck: BIN=$BIN" +cd "$ROOT" || exit 2 +FIX=test/mapinvfix +[ -d "$FIX" ] || { echo "mapinventorycheck: fixture $FIX missing"; exit 2; } + +run() { "$BIN" "$FIX" --no-cache "$@" 2>"$TMP/err"; } +run --top-k=3 >"$TMP/map"; rc=$? +[ "$rc" -eq 0 ] && [ -s "$TMP/map" ] || { echo "mapinventorycheck: the map run failed (rc=$rc)"; cat "$TMP/err"; exit 1; } +run --top-k=3 --legend=full >"$TMP/full" +run --top-k=3 --json >"$TMP/json" +run >"$TMP/whole" # default top-k: every file shown +run --top-k=3 --max-tokens=400 >"$TMP/maxtok" +run --top-k=3 --expand=run >"$TMP/expand" +run --top-k=3 --for="where does main start" >"$TMP/for" +run --top-k=3 --stable >"$TMP/stable" +run --top-k=3 --no-inventory --json >"$TMP/noinvjson" +run --top-k=3 >"$TMP/map2" + +# The tier, isolated (one line of minified XML). +INV="$( grep -o '"; else fail "(A) no on a map whose top-k=3 dropped 15 of 18 files"; fi + +# (B) it comes AFTER the ranked rows and the rows are untouched: … +if grep -q ']*>.*' "$TMP/map"; then ok "(B) follows the last ranked group and closes the root"; else fail "(B) is not appended after the ranked rows"; fi + +# (C) entries: exactly the program entries by convention, sorted by path +ENTRIES="$( printf '%s' "$INV" | grep -o ''; then ok "(D) prog.c main → run"; else fail "(D) prog.c main does not list its callee run"; fi +if printf '%s' "$INV" | grep -q ']*/>'; then ok "(D2) the __main__.py scope → main"; else fail "(D2) the __main__.py entry does not list main"; fi + +# (E) a low-ranked code file is NAMED in its directory's row +if printf '%s' "$INV" | grep -q '"; then ok "(F) $d/ rolls up count-only"; else fail "(F) $d/ is not a count-only row"; fi +done + +# (G) arithmetic: listed + Σ n = the header's files=, and listed = the groups the map printed +FILES="$( grep -o '', text ) + shown = re.search( r'\bshown=(\d+)', hdr.group( 0 ) ) if hdr else None + total = re.search( r'\bsymbols=(\d+)', hdr.group( 0 ) ) if hdr else None keys = set(); cur = "" for m in re.finditer( r'<(f|s) ([^>]*)>', text ): attrs = dict( re.findall( r'([a-z_]+)="([^"]*)"', m.group( 2 ) ) ) if m.group( 1 ) == "f": cur = attrs.get( "p", "" ); continue keys.add( attrs.get( "id" ) or f'{cur}::{attrs.get("t")}::{attrs.get("n")}' ) - return ( int( est.group( 1 ) ) if est else -1 ), keys -bareEst, bare = rows( sys.argv[ 1 ] ) -budEst, bud = rows( sys.argv[ 2 ] ) + return ( int( est.group( 1 ) ) if est else -1 ), keys, ( int( shown.group( 1 ) ) if shown else -1 ), ( int( total.group( 1 ) ) if total else -1 ) +bareEst, bare, _, _ = rows( sys.argv[ 1 ] ) +budEst, bud, shown, total = rows( sys.argv[ 2 ] ) problems = [] -if len( bud ) < 20: problems.append( f"budgeted map has only {len(bud)} rows (presence guard)" ) -if not bud <= bare: problems.append( f"{len(bud - bare)} budgeted row(s) absent from the bare map — not a head, a different ranking" ) -if not 0 < budEst <= 4500: problems.append( f"budgeted est_tokens={budEst}, ceiling 4500" ) -if not bareEst > 4500: problems.append( f"bare est_tokens={bareEst} is already under the 4500 ceiling — the recommendation saves nothing" ) -print( ( "FAIL " + "; ".join( problems ) ) if problems else f"OK bare={bareEst} budgeted={budEst} rows={len(bud)}/{len(bare)}" ) +if len( bud ) < 20: problems.append( f"cut map has only {len(bud)} rows (presence guard)" ) +if not bud <= bare: problems.append( f"{len(bud - bare)} cut-map row(s) absent from the plain map — not a head, a different ranking" ) +if not 0 < budEst < bareEst: problems.append( f"cut est_tokens={budEst} is not below the plain map's {bareEst} — the budget cut nothing" ) +if not ( 0 <= shown < total ): problems.append( f"the cut map's header does not disclose the cut (shown={shown} symbols={total})" ) +print( ( "FAIL " + "; ".join( problems ) ) if problems else f"OK plain={bareEst} cut={budEst} rows={len(bud)}/{len(bare)} shown={shown} of {total}" ) PY )" - case "$verdict" in - OK*) ok "(G4) $budget_flag keeps the bare map's head under the ceiling (${verdict#OK })" ;; - *) no "(G4) $budget_flag does not keep the promise the start-here comment makes: ${verdict#FAIL }" ;; - esac -fi +case "$verdict" in + OK*) ok "(G4) --max-tokens=3000 keeps the plain map's head and discloses the cut (${verdict#OK })" ;; + *) no "(G4) --max-tokens=3000 does not keep the promise the orient comment makes: ${verdict#FAIL }" ;; +esac # ── (H) SUMMARY-LINE NUMBERS ───────────────────────────────────────────────────────────────────── # WHY THIS ARM EXISTS. On 2026-09-07 the lineage section's read "34 repositories, 67 papers diff --git a/test/regression.sh b/test/regression.sh index babf92104..1897da0d9 100755 --- a/test/regression.sh +++ b/test/regression.sh @@ -278,7 +278,7 @@ else fi # retired: forspinecheck — the --for caller spine (Change A) was dropped by its own offline kill rule: a spine of up to 2 caller hops from the top 3 seeds named a missed rubric item on 1 of 7 in-sample rows, so the gate that pinned it went with it # retired: cacheexclkeycheck — the per-configuration auto-cache key it pinned is a registered NEGATIVE (docs/EVALS.md, "The auto-cache key ignores --exclude", RUN 2026-09-03: a 158K-file root with >= 12 gate configurations thrashed the 2 GiB sweep); the retry design keeps ONE superset blob per root and will bring its own gate -for _g in a9disclosurecheck abicheck accessshapecheck ackonlycheck adaptivecheck adaptivecutshapecheck affectedcheck agentloopclaudecheck agentloopcodexcheck agentloopeditsuitecheck agentloopfollowupcheck agentloopgradercheck agentlooplockcheck agentloopopencodecheck agentsurfacecheck agenttablecheck aiderbytescheck anchorbodycheck anchorcheck archcheck archmetricscheck argvdiffcheck arisefollowupcheck ariseshimcheck aritycheck artifactcheck astqueryregexcheck astrocheck atcheck atomscheck attrvocabcheck baselinecheck baselinedirtycheck baselineportcheck bashsourcecheck batchcheck binoverridecheck blindspotcheck bm25boundcheck bm25check bodiesshowncheck bodydialectcheck budgetpolicycheck buildtypestampcheck builtinbindcheck bundleidcheck cachefuzzcheck cachehashcheck cacheidentitycheck cacheisolationcheck cachelintcheck cacheoffsetcheck cachereservecheck cachesplitcheck callerscheck callformcheck callsrankordercheck candheadcheck candidatescheck canoncheck capdisclosurecheck capsweepcheck ccheck ccjsoncheck ceilingverdictcheck chacheck chaconecheck chainguardcheck chainidcheck childwalkscalecheck churndecaycheck churnjoincheck churnjsonstampcheck claudeconfigdircheck clicheck clonebandcheck clonecachecheck clonededupcheck cloneidiomcheck clonelexcheck clsrecvcheck cochangeboostcheck cochangecliocheck cochangesurprisecheck codexinstallhonestycheck codexplugincheck codexwrapcheck collectioncapcheck columnarattrcheck columnarcheck columnarcommacheck commentcoherencecheck commenttokencheck communitydrillcheck communitylabelcheck compactlegendcheck compactroutecheck completecheck composelangcheck connectcheck connectcorecheck connectjoincheck constcheck contextratiocheck coplintcheck countfloorcheck cppbenchcheck cppoperatorcheck cppqualcheck cpptmplscopecheck crashsweepcheck crawlescapecheck crossdirincludecheck crossrefcheck crossrefdegradecheck csharpcheck csharpcondcheck cudacheck cyclecutcheck dartcheck deadcheck deadfiltercheck deadprecisioncheck deckcheck deckclaimcheck declinecheck declinedlistcheck decltodefcheck deeptailcheck defaultceilingcheck defectshapecheck defoverdeclcheck degradedhintcheck dependencypincheck deplangscheck depsprecisecheck detailcheck diagnoticecheck didyoumeancheck dispatchordercheck dmmcheck docanchorcheck docdemotecheck docdriftcheck docdriftcommentcheck docmdcachecheck docmentioncheck docscommandscheck doctorcheck donelegendcheck droppedpositivecheck duprowcheck dynmapsimdcheck editcheckanswercheck editcheckcheck editcheckdeclcheck editchecknotecheck edithandlehintcheck editpayloadbinarycheck editplancheck editplanpayloadconfinecheck editplanrecheckcheck editplanrollbackmsgcheck editpreviewcheck editroundtripcheck edittargetfileabscheck eliximportcheck elixircheck elixirnamearitycheck elixirsemanticcheck emitescapecheck emittertruthcheck emptycorpuscheck emptyvaluerefusecheck ensembleavailcheck ensemblecheck enumtablecheck essentialcxcheck estchargecheck evalcheck evictioncheck exemplarcheck exemplarconfcheck exercisescheck expandbodyfirstcheck expandcallscheck expandmodecheck expandrangecheck expandsibscheck expandtokencheck expandtopk0check extentcheck externalvetocheck falseedgecheck fficheck fieldaffinitycheck fieldidcheck fieldnarrowcheck fieldusescheck filerootcheck fileselectorrefusecheck fillordercheck fixedbufsweep flagscheck flagsnoisecheck flagsurfacecheck flagtablecheck flipcheck floormarkcheck fnliteralcheck fnptrcheck forautobodycheck forblowupcheck forbudgetmonotoncheck forcalibfactscheck forcompresscheck fordisclosurecheck forhdrshapecheck forlenscheck formatgatecheck formaxtokenscheck fornotesbudgetcheck fornotesjsoncheck forrankordercheck forrootlegendcheck forsectioncollapsecheck forsigspancheck forwidencheck freshclonecheck freshnesscheck g1configcheck gateabilitycheck gatecountcheck gateexitcheck gdscriptcheck genrecallcheck gitenvhermeticcheck githardencheck gitignorecheck gitquotepathcheck gitstampcheck goinstcheck gointerfacecheck gokindcheck graphlegendbudgetcheck graphqueryrefusecheck grepanchorcheck grepandcheck grepbytescheck grepcheck grepcontextcheck grepcorpuscheck grepfastcheck grepfollowupcheck grepignorecheck grepscancheck grepseamcheck greptiercheck guardmsgcheck hasacheck hazardpatterncheck headbinstagecheck headsnapcachecheck helpbudgetcheck hermesinstallcheck historyoraclecheck hookcheck hostilecheck hotspotsincecheck htmlcolorcheck htmlhostcheck htmlrendercheck identitycheck impactdepthcheck impactimportcheck impactpartitioncheck importcapcheck importnarrowcheck includeanglecheck includeprecisecheck indexoutcheck infraportcheck isolateprovenancecheck javamethodrefcheck javarubycheck jslangcheck jsmetricscheck jsnestedcheck jsoncheck jsonlangcheck jsonparitycheck jsonredactcheck jsonrefusallegendcheck jsonwalkcheck jsshapecheck jsverbscheck jsxcallcheck knownitemcheck kotlincheck landingcheck langcensuscheck langcheck layerquerycheck layoutcheck lb3namecheck legendcostcheck legendcoveragecheck legenddriftcheck legendrefcheck legobundlecheck legocheck liftdisclosurecheck limitstablecheck lintbudgetcheck lintcatalogcheck lintcheck lintdedupcheck lintpayloadcapcheck lintprecisioncheck lintrulescheck lintscopecheck lintselectcheck listingpagingcheck localitycheck localscountcheck loopconservationcheck lpincheck luacheck luarequirecheck macroedgecheck macroreparsecheck manifestcheck mapdatasectioncheck mapdiffcheck matchcapturecheck matchgrammarcheck maxfilesizecheck mcpattrparitycheck mcpaudit4hardencheck mcpclidiffcheck mcpcodexmetacheck mcpcontractcheck mcpdegradedhintcheck mcpeditcheck mcpeditkindcheck mcpeditmodecheck mcpeditpresencecheck mcpeditracecheck mcpflagshipcheck mcpforparitycheck mcpframehonestycheck mcpgrepdegradedcheck mcphandlecheck mcpincrementalcheck mcpmanifestcheck mcprangeedgecheck mcpreadloopcheck mcpredactcheck mcpreloadasancheck mcpreloadcheck mcpremotecheck mcprobustcheck mcpslicecheck mcpstalecheck mcpstdiolinecapcheck mcpstrictschemacheck mcptoolprunecheck mcptoolsubsetcheck mcptranchecheck mcptwinclaimscheck mcpverbscheck mcpw2fixcheck mcpw3fixcheck mcpwatchercheck mdembedcheck mdsectioncheck memguardcheck mentioncapcheck mentioncheck mentionsverbcheck mergechurncheck mergescoutcheck mergescoutlonglinecheck metalcheck meterdisclosurecheck metricscheck modifierguardcheck moduleconstcheck morecontractcheck mrowalkcheck multirootcheck multiswecheck namedfileinputcheck nameinfocheck namingcalibrationcheck namingconsistencycheck naminglenscheck naminglocalscheck narrowcheck narrowlangcheck neighbourcapcheck nestedimportcheck nestedqualcheck nestprofilecheck nextverbcheck noaliascheck nodekindcheck nongitqmetricscheck nonlocalstatecheck notecanoncheck notescheck notesdegradecheck nsfiltercheck nulbytecheck numericrefusecheck objcfieldcheck objcsniffcheck opencodewrapcheck optremarkscheck optremarkshotcheck ordercheck osswitchcheck oswin32logiccheck outlinecheck overbudgetcommentcheck ownerscheck packcallersharecheck packtaskcheck packtaskmonotoncheck packtaskquotacheck padscalecheck paginationcheck pagingsweepcheck panellegendcheck pargatescheck parsehealthcheck partitioncheck pathgapcheck patterncheck perfharnesscheck phpcheck pincensuscheck planlanescheck planlintcheck pmccheck portablebuildcheck portablecachecheck postingscheck ppaltcheck ppdeadrolescheck pranchorcheck prbudgetcheck prcheck prcontextcheck prconvergecheck precedencecheck preproccondcheck preprocdeadscalecheck prmaskanchorcheck prnestedcapcheck probecheck propcostcheck prrefsafecheck prrenamecheck pyimportprecisecheck pymodulealiascheck pyshapecheck qackconcurrencycheck qackorigincheck qbaselineproducercheck qchurncheck qchurnmemocheck qddialscheck qdrefpaircheck qextractionkeycheck qoriginoraclecheck qrevtokencheck qrowlocatorcheck qschemetripcheck qsnapcachecheck qsnapprefetchcheck qsnapproducercheck qualifiedresolvecheck qualitycheck qualitycrosslangcheck qualityexcludecheck qualitykeycheck qualitykindscheck qualityorigincheck qualitypanelcheck qualityscopecheck qualitysignalcheck qualitystalecheck qualitysymcheck qualnewcheck querycheck queryfilescancheck racymtimecheck radixsimdcheck rangecomposecheck rankbycheck reachcheck readabilitycheck readmedriftcheck readmeexamplecheck recallanchorcheck recallboundarycheck recallbudgetcheck recallbufcheck recallevalcheck recallparitycheck recallpassagecheck recallrankdepthcheck recallrelcheck recallshapecheck recalltablecheck recalltotalcheck receiptpostcheck receiverevidencecheck recentscopecheck redactcheck redactfixcheck refusaltailcheck regexbombcheck regexcheck regexguardcheck regexrefusecheck registermacrocheck relevancefloorcheck relinkcheck reportcheck resolvecheck resolverhonestycheck retrievalqualitycheck reusefirstworkflowcheck ripwirepubliccheck rootrelcheck rootrelemitcheck rootspellingcheck routecheck routeedgecheck routehookcheck routeoncecheck routingreportcheck rubyargcheck rubyattrscheck rubybarecallcheck rubyclassrecvcheck rubyconstcheck rubydeclrefcheck rubydescribedclasscheck rubyinheritcheck rubymetricscheck rubyrakejbuildercheck rubyreachcheck rubyrecvcheck rubyrecvnarrowcheck rubyrequirecheck rubyrspectargetcheck rubyscopecheck rubysettercheck rubytypedrecvcheck runhintcheck runtracecheck rustanccheck rustimportprecisecheck rustqualcheck safedeletecheck sarifcheck savecachecheck scipcheck scipjoincheck scorecardcheck scoutheadconflictcheck scoutkeycheck scroundtripcheck seedboundscheck selectorchaincheck selectorhonestycheck selectorrefusecheck selectorscopecheck selfcheckcheck selfcontainedcheck shadowcheck shallowhistorycheck shapingflagcheck shellgateindexcheck showcasecapturecheck sibliftcheck sidecarsymlinkcheck sigredactcheck sincecheck sincecochangecheck sincewindowcheck singledefcheck situdiffcheck situshapecheck skilldescbudgetcheck skillevalcheck skillevalsplitcheck skillinstallcheck skillroutingjudgedcheck skillscanreadcheck skilltruthcheck skipclassifycheck skippedcheck skipreasoncheck slicecheck slicediffcheck sliceflowcheck sliceflowsenscheck spectimingcheck staleackcheck statgatecheck stdqualcheck strkerncheck structlayoutcheck sublistcountcheck substrfiltercheck subtokencheck svectorcheck swiftcheck swiftmemberscheck swiftshapecheck taskechocheck tempfilesymlinkcheck termmargincheck testedreachcheck testgatecheck testgatelegendbudgetcheck testgatepagecheck testgaterefusecheck testmacrocheck testrowruncheck testscopecheck textdocscheck timsortcheck tokenbudgetcheck tomllangcheck toolcallroutecheck tornreadcheck traceasanlinearcheck tracecheck tracehandoffcapcheck tracehopcheck traceminecheck treecheck truncvocabcheck tsimportprecisecheck tsshapecheck type3check type3clonecheck typerefcheck unreachablecheck unresolvedcheck usescheck usesselectorcheck usingdeclcheck utf8scrubcheck vendoredassetcheck vendoredbundlecheck vendorpatchcheck verifycheck versioncheck w2verbscheck w3fixbudgetcheck w3fixlegendcheck weaksignalcheck withgraphcheck withprofilecheck worktreeleakcheck wrapverbscheck writetargetcheck xmlwellformed yamllangcheck zonecheck zoneconsistencycheck zoomcheck; do +for _g in a9disclosurecheck abicheck accessshapecheck ackonlycheck adaptivecheck adaptivecutshapecheck affectedcheck agentloopclaudecheck agentloopcodexcheck agentloopeditsuitecheck agentloopfollowupcheck agentloopgradercheck agentlooplockcheck agentloopopencodecheck agentsurfacecheck agenttablecheck aiderbytescheck anchorbodycheck anchorcheck archcheck archmetricscheck argvdiffcheck arisefollowupcheck ariseshimcheck aritycheck artifactcheck astqueryregexcheck astrocheck atcheck atomscheck attrvocabcheck baselinecheck baselinedirtycheck baselineportcheck bashsourcecheck batchcheck binoverridecheck blindspotcheck bm25boundcheck bm25check bodiesshowncheck bodydialectcheck budgetpolicycheck buildtypestampcheck builtinbindcheck bundleidcheck cachefuzzcheck cachehashcheck cacheidentitycheck cacheisolationcheck cachelintcheck cacheoffsetcheck cachereservecheck cachesplitcheck callerscheck callformcheck callsrankordercheck candheadcheck candidatescheck canoncheck capdisclosurecheck capsweepcheck ccheck ccjsoncheck ceilingverdictcheck chacheck chaconecheck chainguardcheck chainidcheck childwalkscalecheck churndecaycheck churnjoincheck churnjsonstampcheck claudeconfigdircheck clicheck clonebandcheck clonecachecheck clonededupcheck cloneidiomcheck clonelexcheck clsrecvcheck cochangeboostcheck cochangecliocheck cochangesurprisecheck codexinstallhonestycheck codexplugincheck codexwrapcheck collectioncapcheck columnarattrcheck columnarcheck columnarcommacheck commentcoherencecheck commenttokencheck communitydrillcheck communitylabelcheck compactlegendcheck compactroutecheck completecheck composelangcheck connectcheck connectcorecheck connectjoincheck constcheck contextratiocheck coplintcheck countfloorcheck cppbenchcheck cppoperatorcheck cppqualcheck cpptmplscopecheck crashsweepcheck crawlescapecheck crossdirincludecheck crossrefcheck crossrefdegradecheck csharpcheck csharpcondcheck cudacheck cyclecutcheck dartcheck deadcheck deadfiltercheck deadprecisioncheck deckcheck deckclaimcheck declinecheck declinedlistcheck decltodefcheck deeptailcheck defaultceilingcheck defectshapecheck defoverdeclcheck degradedhintcheck dependencypincheck deplangscheck depsprecisecheck detailcheck diagnoticecheck didyoumeancheck dispatchordercheck dmmcheck docanchorcheck docdemotecheck docdriftcheck docdriftcommentcheck docmdcachecheck docmentioncheck docscommandscheck doctorcheck donelegendcheck droppedpositivecheck duprowcheck dynmapsimdcheck editcheckanswercheck editcheckcheck editcheckdeclcheck editchecknotecheck edithandlehintcheck editpayloadbinarycheck editplancheck editplanpayloadconfinecheck editplanrecheckcheck editplanrollbackmsgcheck editpreviewcheck editroundtripcheck edittargetfileabscheck eliximportcheck elixircheck elixirnamearitycheck elixirsemanticcheck emitescapecheck emittertruthcheck emptycorpuscheck emptyvaluerefusecheck ensembleavailcheck ensemblecheck enumtablecheck essentialcxcheck estchargecheck evalcheck evictioncheck exemplarcheck exemplarconfcheck exercisescheck expandbodyfirstcheck expandcallscheck expandmodecheck expandrangecheck expandsibscheck expandtokencheck expandtopk0check extentcheck externalvetocheck falseedgecheck fficheck fieldaffinitycheck fieldidcheck fieldnarrowcheck fieldusescheck filerootcheck fileselectorrefusecheck fillordercheck fixedbufsweep flagscheck flagsnoisecheck flagsurfacecheck flagtablecheck flipcheck floormarkcheck fnliteralcheck fnptrcheck forautobodycheck forblowupcheck forbudgetmonotoncheck forcalibfactscheck forcompresscheck fordisclosurecheck forhdrshapecheck forlenscheck formatgatecheck formaxtokenscheck fornotesbudgetcheck fornotesjsoncheck forrankordercheck forrootlegendcheck forsectioncollapsecheck forsigspancheck forwidencheck freshclonecheck freshnesscheck g1configcheck gateabilitycheck gatecountcheck gateexitcheck gdscriptcheck genrecallcheck gitenvhermeticcheck githardencheck gitignorecheck gitquotepathcheck gitstampcheck goinstcheck gointerfacecheck gokindcheck graphlegendbudgetcheck graphqueryrefusecheck grepanchorcheck grepandcheck grepbytescheck grepcheck grepcontextcheck grepcorpuscheck grepfastcheck grepfollowupcheck grepignorecheck grepscancheck grepseamcheck greptiercheck guardmsgcheck hasacheck hazardpatterncheck headbinstagecheck headsnapcachecheck helpbudgetcheck hermesinstallcheck historyoraclecheck hookcheck hostilecheck hotspotsincecheck htmlcolorcheck htmlhostcheck htmlrendercheck identitycheck impactdepthcheck impactimportcheck impactpartitioncheck importcapcheck importnarrowcheck includeanglecheck includeprecisecheck indexoutcheck infraportcheck isolateprovenancecheck javamethodrefcheck javarubycheck jslangcheck jsmetricscheck jsnestedcheck jsoncheck jsonlangcheck jsonparitycheck jsonredactcheck jsonrefusallegendcheck jsonwalkcheck jsshapecheck jsverbscheck jsxcallcheck knownitemcheck kotlincheck landingcheck langcensuscheck langcheck layerquerycheck layoutcheck lb3namecheck legendcostcheck legendcoveragecheck legenddriftcheck legendrefcheck legobundlecheck legocheck liftdisclosurecheck limitstablecheck lintbudgetcheck lintcatalogcheck lintcheck lintdedupcheck lintpayloadcapcheck lintprecisioncheck lintrulescheck lintscopecheck lintselectcheck listingpagingcheck localitycheck localscountcheck loopconservationcheck lpincheck luacheck luarequirecheck macroedgecheck macroreparsecheck manifestcheck mapdatasectioncheck mapdiffcheck mapinventorycheck matchcapturecheck matchgrammarcheck maxfilesizecheck mcpattrparitycheck mcpaudit4hardencheck mcpclidiffcheck mcpcodexmetacheck mcpcontractcheck mcpdegradedhintcheck mcpeditcheck mcpeditkindcheck mcpeditmodecheck mcpeditpresencecheck mcpeditracecheck mcpflagshipcheck mcpforparitycheck mcpframehonestycheck mcpgrepdegradedcheck mcphandlecheck mcpincrementalcheck mcpmanifestcheck mcprangeedgecheck mcpreadloopcheck mcpredactcheck mcpreloadasancheck mcpreloadcheck mcpremotecheck mcprobustcheck mcpslicecheck mcpstalecheck mcpstdiolinecapcheck mcpstrictschemacheck mcptoolprunecheck mcptoolsubsetcheck mcptranchecheck mcptwinclaimscheck mcpverbscheck mcpw2fixcheck mcpw3fixcheck mcpwatchercheck mdembedcheck mdsectioncheck memguardcheck mentioncapcheck mentioncheck mentionsverbcheck mergechurncheck mergescoutcheck mergescoutlonglinecheck metalcheck meterdisclosurecheck metricscheck modifierguardcheck moduleconstcheck morecontractcheck mrowalkcheck multirootcheck multiswecheck namedfileinputcheck nameinfocheck namingcalibrationcheck namingconsistencycheck naminglenscheck naminglocalscheck narrowcheck narrowlangcheck neighbourcapcheck nestedimportcheck nestedqualcheck nestprofilecheck nextverbcheck noaliascheck nodekindcheck nongitqmetricscheck nonlocalstatecheck notecanoncheck notescheck notesdegradecheck nsfiltercheck nulbytecheck numericrefusecheck objcfieldcheck objcsniffcheck opencodewrapcheck optremarkscheck optremarkshotcheck ordercheck osswitchcheck oswin32logiccheck outlinecheck overbudgetcommentcheck ownerhopcheck ownerscheck packcallersharecheck packtaskcheck packtaskmonotoncheck packtaskquotacheck padscalecheck paginationcheck pagingsweepcheck panellegendcheck pargatescheck parsehealthcheck partitioncheck pathgapcheck patterncheck perfharnesscheck phpcheck pincensuscheck planlanescheck planlintcheck pmccheck portablebuildcheck portablecachecheck postingscheck ppaltcheck ppdeadrolescheck pranchorcheck prbudgetcheck prcheck prcontextcheck prconvergecheck precedencecheck preproccondcheck preprocdeadscalecheck prmaskanchorcheck prnestedcapcheck probecheck propcostcheck prrefsafecheck prrenamecheck pyimportprecisecheck pymodulealiascheck pyshapecheck qackconcurrencycheck qackorigincheck qbaselineproducercheck qchurncheck qchurnmemocheck qddialscheck qdrefpaircheck qextractionkeycheck qoriginoraclecheck qrevtokencheck qrowlocatorcheck qschemetripcheck qsnapcachecheck qsnapprefetchcheck qsnapproducercheck qualifiedresolvecheck qualitycheck qualitycrosslangcheck qualityexcludecheck qualitykeycheck qualitykindscheck qualityorigincheck qualitypanelcheck qualityscopecheck qualitysignalcheck qualitystalecheck qualitysymcheck qualnewcheck querycheck queryfilescancheck racymtimecheck radixsimdcheck rangecomposecheck rankbycheck reachcheck readabilitycheck readmedriftcheck readmeexamplecheck recallanchorcheck recallboundarycheck recallbudgetcheck recallbufcheck recallevalcheck recallparitycheck recallpassagecheck recallrankdepthcheck recallrelcheck recallshapecheck recalltablecheck recalltotalcheck receiptpostcheck receiverevidencecheck recentscopecheck redactcheck redactfixcheck refusaltailcheck regexbombcheck regexcheck regexguardcheck regexrefusecheck registermacrocheck relevancefloorcheck relinkcheck reportcheck resolvecheck resolverhonestycheck retrievalqualitycheck reusefirstworkflowcheck ripwirepubliccheck rootrelcheck rootrelemitcheck rootspellingcheck routecheck routeedgecheck routehookcheck routeoncecheck routingreportcheck rubyargcheck rubyattrscheck rubybarecallcheck rubyclassrecvcheck rubyconstcheck rubydeclrefcheck rubydescribedclasscheck rubyinheritcheck rubymetricscheck rubyrakejbuildercheck rubyreachcheck rubyrecvcheck rubyrecvnarrowcheck rubyrequirecheck rubyrspectargetcheck rubyschemacheck rubyscopecheck rubysettercheck rubytypedrecvcheck runhintcheck runtracecheck rustanccheck rustimportprecisecheck rustqualcheck safedeletecheck sarifcheck savecachecheck scipcheck scipjoincheck scorecardcheck scoutheadconflictcheck scoutkeycheck scroundtripcheck seedboundscheck selectorchaincheck selectorhonestycheck selectorrefusecheck selectorscopecheck selfcheckcheck selfcontainedcheck shadowcheck shallowhistorycheck shapingflagcheck shellgateindexcheck showcasecapturecheck sibliftcheck sidecarsymlinkcheck sigredactcheck sincecheck sincecochangecheck sincewindowcheck singledefcheck situdiffcheck situshapecheck skilldescbudgetcheck skillevalcheck skillevalsplitcheck skillinstallcheck skillroutingjudgedcheck skillscanreadcheck skilltruthcheck skipclassifycheck skippedcheck skipreasoncheck slicecheck slicediffcheck sliceflowcheck sliceflowsenscheck spectimingcheck staleackcheck statgatecheck stdqualcheck strkerncheck structlayoutcheck sublistcountcheck substrfiltercheck subtokencheck svectorcheck swiftcheck swiftmemberscheck swiftshapecheck taskechocheck tempfilesymlinkcheck termmargincheck testedreachcheck testgatecheck testgatelegendbudgetcheck testgatepagecheck testgaterefusecheck testmacrocheck testrowruncheck testscopecheck textdocscheck timsortcheck tokenbudgetcheck tomllangcheck toolcallroutecheck tornreadcheck traceasanlinearcheck tracecheck tracehandoffcapcheck tracehopcheck traceminecheck treecheck truncvocabcheck tsimportprecisecheck tsshapecheck type3check type3clonecheck typerefcheck unreachablecheck unresolvedcheck usescheck usesselectorcheck usingdeclcheck utf8scrubcheck vendoredassetcheck vendoredbundlecheck vendorpatchcheck verifycheck versioncheck w2verbscheck w3fixbudgetcheck w3fixlegendcheck weaksignalcheck withgraphcheck withprofilecheck worktreeleakcheck wrapverbscheck writetargetcheck xmlwellformed yamllangcheck zonecheck zoneconsistencycheck zoomcheck; do [ -f "$ROOT/test/$_g.sh" ] || continue if RIPWIRE_BIN="$BIN" bash "$ROOT/test/$_g.sh" >/dev/null 2>&1; then ok "absorb gate ($_g.sh)" diff --git a/test/rubyschemacheck.sh b/test/rubyschemacheck.sh new file mode 100755 index 000000000..7df2d0050 --- /dev/null +++ b/test/rubyschemacheck.sh @@ -0,0 +1,196 @@ +#!/usr/bin/env bash +# rubyschemacheck.sh — the Rails schema capture gate: RUBY db/schema.rb COLUMNS ARE Section DEFS. A +# Ruby file whose tree holds a `create_table "x", … do |t| … end` call is a rendered schema BY CONTENT +# (no path heuristic), and each `t. "name"` / `t. :name` receiver-call in +# the table block mints ONE SymKind::Section def at Lang::Ruby — the same data-kind slot as a doc +# heading / YAML key (model.h) — with span = the name token. Rails-generated attribute uses +# (`product.price`, pinned here as explicit-receiver model calls) resolve to the column as a def — +# DEFINITIONS ONLY by maintainer decision: the defs answer --uses/--grep/--whereis but admit NO call +# edges, only the restart share every def gets (graph.h buildGraph's byName skips Section-Ruby; §3 pins that). The DSL +# CALLS keep their reference posture exactly like the attr family (--uses=string/define/timestamps +# stay defs=0 external=1). The four id spellings and t.timestamps name columns too, per the rules +# below; a composite/symbol primary_key mints no implicit id. +# +# Fixture test/rubyschemafix — schema text from a real, running Rails 8.1 app's `bin/rails +# db:schema:dump` (verified db:schema:load-able), with the original domain tables scrubbed to spike_* +# names; two spellings no dump ever emits are restored BY HAND and marked in-file: `id: :uuid` (the +# dump itself notes "Could not dump … Unknown type 'uuid'") and a literal `t.timestamps` (rendered +# schemas expand it to t.datetime pairs) — runtime-semantics and shape notes in USECASES.md beside the +# fixture. +# name the 9 `t.string "name"` columns (→ defs=19 with the 9 attr/def defs + the yaml key) +# id the multi-def floor: EVERY table carries its key — 14 (12 implicit + id: :uuid + …) +# event_id `primary_key: "event_id"` renames the key off `id` +# ref `id: false` table's only column; NO id def there +# created_at/updated_at both the rendered t.datetime pair AND the literal t.timestamps call +# quote_symbol t.string :name + t.references :owner (floor) + t.index ["name"] (floor) + +# helper.string "unbound_column" — a column call on a DIFFERENT bare identifier +# names nothing (the receiver must be the block parameter, `t`) +# migration_string/migration_symbol/migration_columns the migration INTERFERENCE floors: a +# class-wrapped create_table (string- OR symbol-named) mints NOTHING — the schema is the only +# source of column names, indexing a migration beside its schema would double every def and its +# — and add_column/remove_column are argument data, never defs (a column added +# then dropped never registers) +# floor_case where("name = ?") — a string fragment stays opaque, never a def or a use +# all_four / pair_def_column / triple_* the def+attr+column(+yaml) collision matrix; the picker +# styles: `self.name` in the instance method PairDefColumn#lookup pins its own def (locality), +# `rec.name` on a local splits over the method/attr defs (graph_ambiguous), and +# `SingleColumn.new.name` binds nothing (no in-tree def on that model; never a column) +# unrelated_client an untyped `response.ref` (ref is a column-only name): a use, never a caller +# config/spike_names.yml the cross-language name key (counted, never edgeable) +# +# Usage: test/rubyschemacheck.sh | RIPWIRE_BIN=asan/ripwire test/rubyschemacheck.sh +# Exit: 0 = clean · 1 = an arm failed · 2 = usage / missing prerequisite + +set -u +ROOT="$( cd "$( dirname "$0" )/.." && pwd )" +BIN="${1:-${RIPWIRE_BIN:-$ROOT/build/ripwire}}" +[ "${BIN#/}" = "$BIN" ] && BIN="$ROOT/$BIN" +FIX="$ROOT/test/rubyschemafix" +TMP="$( mktemp -d )"; trap 'rm -rf "$TMP"' EXIT +fail=0 +ok(){ printf ' PASS %s\n' "$*" || { fail=1; printf ' FAIL could not write the PASS line for: %s\n' "$*"; }; return 0; } +no(){ printf ' FAIL %s\n' "$*"; fail=1; } + +[ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first (cmake --build build -j)"; exit 2; } +[ -d "$FIX" ] || { echo "no test/rubyschemafix — fixture missing"; exit 2; } +echo "rubyschemacheck: BIN=$BIN FIX=$FIX" + +useshead(){ "$BIN" "$FIX" --uses="$1" --no-cache 2>/dev/null | grep -oE "of=\"[^\"]*\" defs=\"[0-9]+\" external=\"[0-9]+\" count=\"[0-9]+\"" | head -1; } +undefinable(){ "$BIN" "$FIX" --uses="$1" --no-cache 2>&1 | grep -q "matched no indexed definition"; } +callershead(){ "$BIN" "$FIX" --callers="$1" --no-cache 2>/dev/null | grep -oE "of=\"[^\"]*\" defs=\"[0-9]+\" count=\"[0-9]+\"" | head -1; } + +# ── 1. CAPTURE: columns are Section defs; the id rule; the timestamps pair ────────────────────────── +[ "$( useshead name )" = 'of="name" defs="19" external="0" count="3"' ] \ + && ok 'capture: name — 9 columns + 9 attr/def defs + 1 yaml key = 19; only the 3 consumers use it (the where() fragment is opaque)' \ + || no "capture: name: $( useshead name )" +[ "$( useshead id )" = 'of="id" defs="14" external="0" count="1"' ] \ + && ok 'capture: id — the multi-def floor: 12 implicit + the id: :uuid pair + the id-pair… 14 keys on 14 tables (only id_false and the renamed table have none)' \ + || no "capture: id: $( useshead id )" +[ "$( useshead event_id )" = 'of="event_id" defs="1" external="0" count="0"' ] \ + && ok 'capture: `primary_key: "event_id"` mints the renamed key def, and NO implicit id beside it' \ + || no "capture: event_id: $( useshead event_id )" +[ "$( useshead ref )" = 'of="ref" defs="1" external="0" count="1"' ] \ + && ok 'capture: the `id: false` table'"'"'s own column mints; the table has NO id def (its one use is the untyped unrelated_client.rb site)' \ + || no "capture: ref: $( useshead ref )" +[ "$( useshead created_at )" = 'of="created_at" defs="2" external="0" count="1"' ] \ + && ok 'capture: created_at — the rendered t.datetime column AND the literal t.timestamps call each mint one; the consumer call reaches them' \ + || no "capture: created_at: $( useshead created_at )" +[ "$( useshead updated_at )" = 'of="updated_at" defs="2" external="0" count="0"' ] \ + && ok 'capture: updated_at — both spellings def; no consumer calls it (floor proof the count is per-row, not padded)' \ + || no "capture: updated_at: $( useshead updated_at )" +[ "$( useshead origin )" = 'of="origin" defs="1" external="0" count="0"' ] \ + && ok 'capture: a COMPOSITE primary_key (["origin","destination"]) mints the rendered columns — and NO implicit id (the id defs=14 arm above is the regression guard)' \ + || no "capture: origin: $( useshead origin )" +[ "$( useshead destination )" = 'of="destination" defs="1" external="0" count="0"' ] \ + && ok 'capture: the composite key'"'"'s second column mints too' \ + || no "capture: destination: $( useshead destination )" +undefinable 'price > 0' \ + && ok "floor: t.check_constraint \"price > 0\", name: … — a constraint EXPRESSION names no column (would otherwise mint n=\"price > 0\")" \ + || no 'floor: price > 0 — a check_constraint expression minted a def' + +# ── 2. FLOORS: the DSL calls stay references; non-column DSL names no column ─────────────────────── +undefinable owner \ + && ok 'floor: t.references :owner names no column — owner stays undefinable (references is the reference, its arg is data)' \ + || no 'floor: owner — t.references minted a def' +[ "$( useshead index )" = 'of="index" defs="0" external="1" count="1"' ] \ + && ok 'floor: t.index ["name"] — the CALL is an external reference (DSL posture), the array arg defines nothing' \ + || no "floor: index: $( useshead index )" +undefinable email \ + && ok 'migration: a CLASS-wrapped string-named create_table ("spike_migrated") mints nothing — the same columns the migration created are already minted from the rendered schema; indexing both would DOUBLE the defs+PageRank' \ + || no 'migration: email — a class-wrapped (string-named) migration table minted a def' +undefinable widget_name \ + && ok 'migration: a symbol-named migration table (create_table :spike_widgets) mints nothing — both the class-nest and the string-name gates refuse' \ + || no 'migration: widget_name — a symbol-named migration table minted a def' +undefinable bla \ + && ok 'migration: add_column "bla" is argument data — the DSL call is a reference, its args are not defs (only the schema is the source of names)' \ + || no 'migration: bla — add_column minted a def' +undefinable gonna_die \ + && ok 'migration: a column added then REMOVED never registers (remove_column is read nowhere)' \ + || no 'migration: gonna_die — a removed column minted a def' +undefinable unbound_column \ + && ok 'receiver: a column call on a DIFFERENT bare identifier (`helper.string "unbound_column"`) names no column — the receiver must BE the block parameter (`t`) the create_table call bound' \ + || no 'receiver: unbound_column — a non-t receiver minted a def' +for dsl in string define datetime timestamps create_table; do + [ "$( useshead "$dsl" | grep -oE 'defs="[0-9]+" external="[0-9]+"' )" = 'defs="0" external="1"' ] \ + && ok "posture: the DSL call \`$dsl\` itself is still no def (reference capture posture)" \ + || no "posture: $dsl gained a def — the DSL call must stay a reference: $( useshead "$dsl" )" +done + +# ── 3. DEFINITIONS ONLY: columns admit NO call edges (maintainer decision, graph.h buildGraph byName) ── +[ "$( callershead id )" = 'of="id" defs="14" count="0"' ] \ + && ok 'defs-only: `IdDefault.new.id` matches the 14 defs but has NO caller row — columns take no edges' \ + || no "defs-only: id callers: $( callershead id )" +[ "$( callershead created_at )" = 'of="created_at" defs="2" count="0"' ] \ + && ok 'defs-only: `Timestamps.new.created_at` matches both defs but carries no call edge' \ + || no "defs-only: created_at callers: $( callershead created_at )" +"$BIN" "$FIX" --uses=ref --no-cache 2>/dev/null | grep -q 'p="unrelated_client.rb:7"' \ + && [ "$( callershead ref )" = 'of="ref" defs="1" count="0"' ] \ + && ok "defs-only false-binding: the untyped 'response.ref' in unrelated_client.rb:7 is a use of the column-only name ref, never its caller (RED without buildGraph's Section-Ruby byName skip: it binds by name alone)" \ + || no "defs-only: ref callers: $( callershead ref )" +"$BIN" "$FIX" --callers=name --no-cache 2>/dev/null >"$TMP/callers_name" +[ "$( callershead name )" = 'of="name" defs="19" count="2"' ] \ + && grep -q '' "$TMP/callers_name" \ + && grep -q '' "$TMP/callers_name" \ + && ok 'defs-only: the 2 name callers are METHODS (render_label by name alone, lookup pinned by its own class) — the 9 columns contribute zero edges to the 19-def set; SingleColumn.new.name binds nothing (no in-tree def on that model)' \ + || no "defs-only: name callers: $( callershead name )" + +# ── 4. AMBIGUITY + LOCALITY: the resolver splits honestly, pins on evidence ───────────────────────── +"$BIN" "$FIX" --uses=name --no-cache 2>/dev/null >"$TMP/uses_name" +grep -q 'of="name" defs="19" external="0" count="3"' "$TMP/uses_name" \ + && grep -q 'graph_ambiguous="1"' "$TMP/uses_name" \ + && ok 'ambiguity: the name gauge counts ONLY the 1 site that still splits over real (method/attr) defs (rec.name, consumer_ambiguous.rb:8) — the id/created_at column sites left the gauge once columns stopped taking edges, and self.name in PairDefColumn#lookup is pinned by its class' \ + || no 'ambiguity: name gauge/rows wrong' +grep -q 'p="consumer_ambiguous.rb:8"' "$TMP/uses_name" \ + && ok 'ambiguous arm: rec.name on a plain local SPLITS over the whole def set (its row is in the gauge)' \ + || no 'ambiguous arm: consumer_ambiguous.rb:8 row missing' +grep -q 'p="pair_def_column.rb:15"' "$TMP/uses_name" \ + && [ "$( "$BIN" "$FIX" --callees=lookup --no-cache 2>/dev/null | grep -oE ']*>' | tr -d '\n' )" = '' ] \ + && ok 'locality arm: self.name in the instance method PairDefColumn#lookup pins exactly its own class'"'"'s def name (pair_def_column.rb:6, no via=) and its use row is NOT in the ambiguous gauge — never a column' \ + || no "locality arm: pair_def_column.rb:15 row missing or lookup does not pin pair_def_column.rb:6 alone" +grep -q 'p="column_consumers.rb:7"' "$TMP/uses_name" \ + && ok 'binding arm: SingleColumn.new.name is a real use row (explicit model receiver)' \ + || no 'binding arm: column_consumers.rb:7 row missing' + +# ── 5. MAP KINDS: the defs carry t="sec" with the DSL row family merged by name ───────────────────── +"$BIN" "$FIX" --no-cache 2>/dev/null >"$TMP/map" +# The schema.rb file block, extracted once; every kind arm asserts INSIDE it (the yaml Section key is a +# different file and must never satisfy a schema assertion). +python3 - "$TMP/map" >"$TMP/schema_block" <<'PYEOF' +import re, sys +xml = open(sys.argv[1], encoding='utf-8').read() +m = re.search(r']*>(.*?)', xml, re.S) +sys.stdout.write(m.group(1) if m else "") +PYEOF +grep -q '"$TMP/m1" 2>/dev/null +"$BIN" "$FIX" --no-cache >"$TMP/m2" 2>/dev/null +if cmp -s "$TMP/m1" "$TMP/m2"; then ok "deterministic (two --no-cache runs byte-identical)"; else no "non-deterministic"; fi +"$BIN" "$FIX" --cache="$TMP/c.bin" >"$TMP/cold" 2>/dev/null +"$BIN" "$FIX" --cache="$TMP/c.bin" >"$TMP/warm" 2>/dev/null +if cmp -s "$TMP/cold" "$TMP/warm"; then ok "warm == cold (schema defs survive the cache round-trip)"; else no "warm != cold"; fi +if command -v xmllint >/dev/null 2>&1; then + if xmllint --noout "$TMP/m1" 2>/dev/null; then ok "xml well-formed"; else no "xml malformed"; fi +else + ok "xml well-formed (xmllint absent — skipped)" +fi + +[ "$fail" -eq 0 ] && echo "ALL PASS" || { echo "SOME CHECKS FAILED"; exit 1; } \ No newline at end of file diff --git a/test/rubyschemafix/USECASES.md b/test/rubyschemafix/USECASES.md new file mode 100644 index 000000000..6cc6aba0e --- /dev/null +++ b/test/rubyschemafix/USECASES.md @@ -0,0 +1,57 @@ +# rubyschemafix — fixture fact sheet (the Rails schema-column capture) + +What this fixture is, and which claims in it are runtime-proven vs stated floors. + +## Provenance + +`schema.rb` is the rendered schema of a real, running Rails 8.1 application: its `bin/rails +db:schema:dump` output, with the original domain tables and all identifying markers stripped and the +remaining tables renamed to `spike_*`. The dump was verified to load back into that application +(`bin/rails db:schema:load` succeeded), so the shape vocabulary here — `force: :cascade`, `id: false`, +`primary_key:`, the option-pair spellings, the string/symbol column spellings, `null: false` — comes +from a working Rails install, not from a hand-invented guess. + +Two spellings no real dump ever emits are restored BY HAND in `schema.rb` and marked in-file: + +- `spike_id_uuids` with `id: :uuid` — the source app's dump FAILED for this table ("Could not dump + table … Unknown type 'uuid' for column 'id'": the dump host lacked the uuid extension). The table's + shape is the standard Rails rendering for a uuid key; the `id: :uuid` pair is the spelling ripwire's + id rule reads (the pair KEY names `id`, the uuid TYPE is not modelled — stated floor). +- `spike_timestamps_literal` with a bare `t.timestamps` — rendered schemas expand it into the two + `t.datetime` pairs (present above it as `spike_timestamps`); the DSL CALL itself exists only in + migrations. The column names it mints (`created_at`, `updated_at`) are part of Active Record's + documented timestamps contract. + +The model files mirror the source application's runtime shapes (`self.table_name`, `attribute`, +`def name`; a quoted `where("name = ?")` in `floor_case.rb` is the runtime pattern of a raw SQL +fragment). They are indexed as text; nothing here executes at gate time. + +## What the gate pins (all measured on the binary, per the repo's honesty rule) + +| Query | Pinned answer | Why | +|---|---|---| +| `--uses=name` | defs=19 external=0 count=3 | 9 columns + 9 attr/def defs + 1 yaml key; only the 3 consumers use it — the `where("name = ?")` fragment stays opaque | +| `--uses=id` | defs=14 external=0 count=1 | the multi-def floor: EVERY table carries its key (12 implicit, `id: :uuid`, and the explicit `id: false`-less ones); only `id: false` + the renamed table have none | +| `--uses=created_at` | defs=2 | the rendered `t.datetime "created_at"` column AND the literal `t.timestamps` call | +| `--uses=event_id` | defs=1 | `primary_key: "event_id"` renames the key off `id` | +| `--callers=created_at` | defs=2 count=0 | DEFINITIONS ONLY: the call site matches both defs but columns carry NO call edges (§3) | +| `graph_ambiguous` | 2 on `name` | only the sites that still split over real method/attr defs; the column sites exited the gauge when edges were switched off (§4) | + +## Stated floors (silence is stated, never silent) + +- `t.references :owner` / `t.belongs_to` / `t.polymorphic` / `t.index` name no column (the fixture + spells `references` + `index` in `spike_quote_symbol`; `owner` stays undefinable). +- A symbol-named table (`create_table :users`) is a migration spelling, not a rendered-schema shape — + no name-string node ⇒ no schema surface. +- Duplicate column names across tables stay SEPARATE defs (`id` on N tables → defs=N); the map row + merges them by name with `overloads`, never hiding the multiplicity. +- MIGRATIONS never interfere (the three migration fixtures, all pinned undefinable): a class-wrapped + `create_table "x"` / `create_table :x` — the migration shape — is refused by the class/module-nest + gate, so the schema is the ONE source of column names and indexing a migration beside its schema + cannot double any def; `add_column` / `remove_column` / `change_column` are argument data and read + nowhere — a column added then dropped (`gonna_die`) never registers. +- `where("price > ?")` string fragments are opaque to any static extractor. +- The DSL calls (`string`, `datetime`, `create_table`, …) stay external-surface references — no def + named `string` appears; only the NAME args gain defs (same posture as the attr family). +- A column call's receiver must BE the block parameter: `helper.string "unbound_column"` (a different + bare identifier) names no column — the gate accepts only `t.` (the `|t|` handle) forms. \ No newline at end of file diff --git a/test/rubyschemafix/all_four.rb b/test/rubyschemafix/all_four.rb new file mode 100644 index 000000000..423054b34 --- /dev/null +++ b/test/rubyschemafix/all_four.rb @@ -0,0 +1,14 @@ +# all-four arm: def + attribute + column in ONE model, and the yaml source in +# config/spike_names.yml. This is the canonical collision; runtime dispatch winner +# is the def, the static picker order (def > attr > column > yaml) is pinned in C. +module Spike + class AllFour < ApplicationRecord + self.table_name = "spike_all_fours" + + attribute :name + + def name + "def" + end + end +end diff --git a/test/rubyschemafix/column_consumers.rb b/test/rubyschemafix/column_consumers.rb new file mode 100644 index 000000000..b25eba7fa --- /dev/null +++ b/test/rubyschemafix/column_consumers.rb @@ -0,0 +1,17 @@ +# consumer arm: EXPLICIT model receivers — `Spike::SingleColumn.new.name` resolves to the +# column def set by name; `Spike::IdDefault.new.id` sits on the multi-table implicit id +# floor (14 id defs). Definitions-only per the review round: these calls match the defs +# but columns take NO call edges, so the gate's §3 arms are count="0" assertions. +module Spike + def self.single_name + SingleColumn.new.name + end + + def self.default_id + IdDefault.new.id + end + + def self.literal_stamp + Timestamps.new.created_at + end +end diff --git a/test/rubyschemafix/column_yaml.rb b/test/rubyschemafix/column_yaml.rb new file mode 100644 index 000000000..f794405e8 --- /dev/null +++ b/test/rubyschemafix/column_yaml.rb @@ -0,0 +1,6 @@ +# pair arm (column, yaml): column only; the yaml source lives in config/spike_names.yml. +module Spike + class ColumnYaml < ApplicationRecord + self.table_name = "spike_column_yamls" + end +end diff --git a/test/rubyschemafix/config/spike_names.yml b/test/rubyschemafix/config/spike_names.yml new file mode 100644 index 000000000..9e2e762a9 --- /dev/null +++ b/test/rubyschemafix/config/spike_names.yml @@ -0,0 +1,5 @@ +# spike_names.yml — the cross-language Section def for `name`. +# The all-four collision (def + attribute + column + yaml) in all_four.rb; the YAML +# key is counted, never edgeable (Ruby<->Yaml is not langCompatible). +name: + display: "Spike" \ No newline at end of file diff --git a/test/rubyschemafix/consumer_ambiguous.rb b/test/rubyschemafix/consumer_ambiguous.rb new file mode 100644 index 000000000..ffe21c986 --- /dev/null +++ b/test/rubyschemafix/consumer_ambiguous.rb @@ -0,0 +1,11 @@ +# consumer arm, spam: a call site with NO locality evidence — `rec.name` where rec is +# an ordinary local. Every in-repo def named `name` is a candidate (the 9 columns, the +# attr Vars, the method defs, the yaml key); the resolver SPLITS and says so (amb=K), +# never silently picks. The call edges land on the def set, not on nothing. +module Spike + class ConsumerAmbiguous < ApplicationRecord + def render_label( rec ) + rec.name + end + end +end \ No newline at end of file diff --git a/test/rubyschemafix/floor_case.rb b/test/rubyschemafix/floor_case.rb new file mode 100644 index 000000000..1e8dc422f --- /dev/null +++ b/test/rubyschemafix/floor_case.rb @@ -0,0 +1,12 @@ +# floor arm: a raw SQL string fragment. The `name` inside it is opaque to any +# static extractor — this is a DISCLOSED floor (no def, no use), proven only that +# the query runs. +module Spike + class FloorCase < ApplicationRecord + self.table_name = "spike_single_columns" + + def self.find_named( value ) + where( "name = ?", value ).to_a + end + end +end diff --git a/test/rubyschemafix/id_default.rb b/test/rubyschemafix/id_default.rb new file mode 100644 index 000000000..55f9e7ad3 --- /dev/null +++ b/test/rubyschemafix/id_default.rb @@ -0,0 +1,6 @@ +# id arm: implicit default id. +module Spike + class IdDefault < ApplicationRecord + self.table_name = "spike_id_defaults" + end +end diff --git a/test/rubyschemafix/id_false.rb b/test/rubyschemafix/id_false.rb new file mode 100644 index 000000000..732241389 --- /dev/null +++ b/test/rubyschemafix/id_false.rb @@ -0,0 +1,8 @@ +# id arm: create_table carried `id: false` — there is NO id column; the model +# declares its own primary key. +module Spike + class IdFalse < ApplicationRecord + self.table_name = "spike_id_false" + self.primary_key = "ref" + end +end diff --git a/test/rubyschemafix/id_renamed.rb b/test/rubyschemafix/id_renamed.rb new file mode 100644 index 000000000..08c02c84c --- /dev/null +++ b/test/rubyschemafix/id_renamed.rb @@ -0,0 +1,6 @@ +# id arm: `t.primary_key "event_id"` renames the key off `id`. +module Spike + class IdRenamed < ApplicationRecord + self.table_name = "spike_id_renamed" + end +end diff --git a/test/rubyschemafix/id_uuid.rb b/test/rubyschemafix/id_uuid.rb new file mode 100644 index 000000000..d0d21eaaa --- /dev/null +++ b/test/rubyschemafix/id_uuid.rb @@ -0,0 +1,6 @@ +# id arm: `id: :uuid` — the key is present, still named `id`, uuid type. +module Spike + class IdUuid < ApplicationRecord + self.table_name = "spike_id_uuids" + end +end diff --git a/test/rubyschemafix/migration_columns.rb b/test/rubyschemafix/migration_columns.rb new file mode 100644 index 000000000..c70a1fdbd --- /dev/null +++ b/test/rubyschemafix/migration_columns.rb @@ -0,0 +1,13 @@ +# migration arm, column lifecycle: an OLD migration adds "bla", a LATER one removes it — +# only the rendered schema is the source of column names. `add_column` / `remove_column` +# are not read anywhere (the DSL call is a reference, its arguments are data), so a column +# added then dropped never registers, and a column that STILL exists is registered by the +# schema, not by the migration that added it. `bla` and `gonna_die` stay undefinable. +class MoreColumns < ActiveRecord::Migration[8.1] + def change + add_column :spike_single_columns, "bla", :string + change_column :spike_single_columns, :ref, :text + add_column :spike_single_columns, "gonna_die", :string + remove_column :spike_single_columns, "gonna_die" + end +end \ No newline at end of file diff --git a/test/rubyschemafix/migration_string.rb b/test/rubyschemafix/migration_string.rb new file mode 100644 index 000000000..a25572c7e --- /dev/null +++ b/test/rubyschemafix/migration_string.rb @@ -0,0 +1,14 @@ +# migration arm, STRING-named table: a real migration wraps its create_table in a class body. +# The schema capture must NOT mint from it — the same columns the migration created are already +# minted from the rendered db/schema.rb, and indexing both would DOUBLE every def on corpora +# that carry migrations. The suppression is the +# class/module-nest gate: a rendered schema's tables live at file level or under +# ActiveRecord::Schema[].define, never inside a class. `email` here stays undefinable; the +# table's implicit id is NOT added to the id multi-def floor (the gate pins both). +class CreateUsers < ActiveRecord::Migration[8.1] + def change + create_table "spike_migrated" do |t| + t.string "email" + end + end +end \ No newline at end of file diff --git a/test/rubyschemafix/migration_symbol.rb b/test/rubyschemafix/migration_symbol.rb new file mode 100644 index 000000000..5345d1766 --- /dev/null +++ b/test/rubyschemafix/migration_symbol.rb @@ -0,0 +1,11 @@ +# migration arm, SYMBOL-named table: the other common spelling. Already floored by the +# string-name gate (a symbol-named create_table is a migration spelling, not a rendered +# schema), kept as a fixture so the floor is pinned from BOTH sides — the class-nest gate +# and the name-string gate both refuse, and the widget columns stay undefinable. +class CreateWidgets < ActiveRecord::Migration[8.1] + def change + create_table :spike_widgets do |t| + t.string "widget_name" + end + end +end \ No newline at end of file diff --git a/test/rubyschemafix/pair_def_column.rb b/test/rubyschemafix/pair_def_column.rb new file mode 100644 index 000000000..f564004bd --- /dev/null +++ b/test/rubyschemafix/pair_def_column.rb @@ -0,0 +1,18 @@ +# pair arm (def, column): the class def shadows the generated attribute reader. +module Spike + class PairDefColumn < ApplicationRecord + self.table_name = "spike_pair_def_columns" + + def name + "def" + end + + # locality-resolved arm: an INSTANCE method's `self.name`, in the class whose own + # `def name` sits in this same file — Ruby's method lookup pins THAT def (no via=), + # never the schema column or the yaml key. (A class-side `def self.lookup` would + # call the class object's own `name`, Module#name, and bind nothing in the tree.) + def lookup + self.name + end + end +end diff --git a/test/rubyschemafix/quote_double.rb b/test/rubyschemafix/quote_double.rb new file mode 100644 index 000000000..824666252 --- /dev/null +++ b/test/rubyschemafix/quote_double.rb @@ -0,0 +1,6 @@ +# quote-spelling arm: column declared t.string "name" (double quotes). +module Spike + class QuoteDouble < ApplicationRecord + self.table_name = "spike_quote_double" + end +end diff --git a/test/rubyschemafix/quote_single.rb b/test/rubyschemafix/quote_single.rb new file mode 100644 index 000000000..2b7cf31f4 --- /dev/null +++ b/test/rubyschemafix/quote_single.rb @@ -0,0 +1,6 @@ +# quote-spelling arm: column declared `t.string 'name'` (single quotes). +module Spike + class QuoteSingle < ApplicationRecord + self.table_name = "spike_quote_single" + end +end diff --git a/test/rubyschemafix/quote_symbol.rb b/test/rubyschemafix/quote_symbol.rb new file mode 100644 index 000000000..22cdf94ff --- /dev/null +++ b/test/rubyschemafix/quote_symbol.rb @@ -0,0 +1,6 @@ +# quote-spelling arm: column declared t.string :name (symbol). +module Spike + class QuoteSymbol < ApplicationRecord + self.table_name = "spike_quote_symbol" + end +end diff --git a/test/rubyschemafix/schema.rb b/test/rubyschemafix/schema.rb new file mode 100644 index 000000000..ced49257a --- /dev/null +++ b/test/rubyschemafix/schema.rb @@ -0,0 +1,90 @@ +# This file is auto-generated from the current state of the database. Instead +# of editing this file, please use the migrations feature of Active Record to +# incrementally modify your database, and then regenerate this schema definition. +# +# This file is the source Rails uses to define your schema when running `bin/rails +# db:schema:load`. When creating a new database, `bin/rails db:schema:load` tends to +# be faster and is potentially less error prone than running all of your +# migrations from scratch. Old migrations may fail to apply correctly if those +# migrations use external dependencies or application code. +# +# It's strongly recommended that you check this file into your version control system. + +ActiveRecord::Schema[8.1].define(version: 2000_01_01_000001) do + create_table "spike_all_fours", force: :cascade do |t| + t.string "name" + end + + create_table "spike_column_yamls", force: :cascade do |t| + t.string "name" + end + + create_table "spike_id_defaults", force: :cascade do |t| + end + + create_table "spike_id_false", id: false, force: :cascade do |t| + t.string "ref", null: false + end + + create_table "spike_id_renamed", primary_key: "event_id", force: :cascade do |t| + end + +# Could not dump table "spike_id_uuids" because of following StandardError +# Unknown type 'uuid' for column 'id' +# Restored by hand: the standard Rails rendering for a uuid table — `id: :uuid` (the value is a +# simple_symbol), the spelling the capture's id rule reads. + create_table "spike_id_uuids", id: :uuid, force: :cascade do |t| + end + + create_table "spike_pair_attr_columns", force: :cascade do |t| + t.string "name" + end + + create_table "spike_pair_def_columns", force: :cascade do |t| + t.string "name" + end + + create_table "spike_plain", force: :cascade do |t| + end + + create_table "spike_quote_double", force: :cascade do |t| + t.string "name" + end + + create_table "spike_quote_single", force: :cascade do |t| + t.string 'name' + end + +create_table "spike_quote_symbol", force: :cascade do |t| + t.string :name + t.references :owner + t.index ["name"], name: "idx_spike" + helper.string "unbound_column" + t.check_constraint "price > 0", name: "check_positive" + end + + create_table "spike_composite_keys", primary_key: ["origin", "destination"], force: :cascade do |t| + t.string "origin" + t.string "destination" + end + + create_table "spike_single_columns", force: :cascade do |t| + t.string "name" + end + + create_table "spike_timestamps", force: :cascade do |t| + t.datetime "created_at", null: false + t.datetime "updated_at", null: false + end + +# Migration spelling, added by hand: the rendered dump expands `t.timestamps` into the two datetime +# columns above; the DSL CALL itself (the rule under test) names both columns literally. + create_table "spike_timestamps_literal", force: :cascade do |t| + t.timestamps + end + + create_table "spike_triples", force: :cascade do |t| + t.string "name" + end + +end diff --git a/test/rubyschemafix/single_column.rb b/test/rubyschemafix/single_column.rb new file mode 100644 index 000000000..1c89b79d4 --- /dev/null +++ b/test/rubyschemafix/single_column.rb @@ -0,0 +1,6 @@ +# single-source arm: column `name` only (no def, no attr). +module Spike + class SingleColumn < ApplicationRecord + self.table_name = "spike_single_columns" + end +end diff --git a/test/rubyschemafix/timestamps.rb b/test/rubyschemafix/timestamps.rb new file mode 100644 index 000000000..59fd081ba --- /dev/null +++ b/test/rubyschemafix/timestamps.rb @@ -0,0 +1,6 @@ +# timestamps arm: `t.timestamps` yields created_at / updated_at columns. +module Spike + class Timestamps < ApplicationRecord + self.table_name = "spike_timestamps" + end +end diff --git a/test/rubyschemafix/triple_plain.rb b/test/rubyschemafix/triple_plain.rb new file mode 100644 index 000000000..d920b8458 --- /dev/null +++ b/test/rubyschemafix/triple_plain.rb @@ -0,0 +1,14 @@ +# triple arm, spelling (a): plain `def name` alongside `attribute :name` +# and a real `name` column. Ruby lookup is by ancestry — the class-level def +# wins over the module-generated attribute reader regardless of declaration order. +module Spike + class TriplePlain < ApplicationRecord + self.table_name = "spike_triples" + + attribute :name + + def name + "def" + end + end +end diff --git a/test/rubyschemafix/triple_reversed.rb b/test/rubyschemafix/triple_reversed.rb new file mode 100644 index 000000000..c9a2881ec --- /dev/null +++ b/test/rubyschemafix/triple_reversed.rb @@ -0,0 +1,13 @@ +# triple arm, spelling (a) reversed: `def name` declared BEFORE `attribute :name`. +# Same winner as triple_plain — proof that lookup is by ancestry, not declaration order. +module Spike + class TripleReversed < ApplicationRecord + self.table_name = "spike_triples" + + def name + "def" + end + + attribute :name + end +end diff --git a/test/rubyschemafix/triple_super.rb b/test/rubyschemafix/triple_super.rb new file mode 100644 index 000000000..8f58e4612 --- /dev/null +++ b/test/rubyschemafix/triple_super.rb @@ -0,0 +1,15 @@ +# triple arm, spelling (b): `def name; super; end` + `attribute :name` + +# column. Bare `super` reaches the generated attribute reader, which reads the +# column-backed @attributes store. There is no separate "column method" — the +# column IS the storage. Expected chain: def -> super -> generated attr reader -> column. +module Spike + class TripleSuper < ApplicationRecord + self.table_name = "spike_triples" + + attribute :name + + def name + super + end + end +end diff --git a/test/rubyschemafix/unrelated_client.rb b/test/rubyschemafix/unrelated_client.rb new file mode 100644 index 000000000..6ef5baa73 --- /dev/null +++ b/test/rubyschemafix/unrelated_client.rb @@ -0,0 +1,9 @@ +# false-binding arm (definitions only): an UNTYPED receiver calls a method spelled like a +# COLUMN-ONLY name (`ref`, the `id: false` table's one column, schema.rb). Nothing proves the +# receiver is that table's model, so the call is a use of the name and never the column's caller. +# Without buildGraph's Section-Ruby byName skip this site binds to the column by name alone. +class UnrelatedClient + def fetch( response ) + response.ref + end +end diff --git a/test/scroundtripcheck.sh b/test/scroundtripcheck.sh index c60402b98..727c7c22f 100755 --- a/test/scroundtripcheck.sh +++ b/test/scroundtripcheck.sh @@ -155,7 +155,13 @@ if grep -q '"sc":"Widget"' "$TMP/map.json" && ! grep -q '"id":"qual.cpp::' "$TMP # that had just printed the path, on a document carrying no legend at all — a repetition AND an undefined # first-screen attribute on the one --expand shape with nothing else to read. # L1 (2026-09-19): (E2) reads the FULL legend's composition prose, so this run asks for the full legend. -"$BIN" test/nestedqualfix --expand=Outer --no-cache --legend=full >"$TMP/wf.xml" 2>/dev/null "$WFQ/outer.hpp" +"$BIN" "$WFQ" --expand=Outer --no-cache --legend=full >"$TMP/wf.xml" 2>/dev/null ]*id="outer.hpp::' "$TMP/wf.xml"; then ok "(E2) whole-file --expand anchor rows carry sc= and no path-repeating id=" @@ -167,12 +173,12 @@ if grep -q 'mode="whole-file"' "$TMP/wf.xml"; then && ok "(E2) the whole-file root defines the composition (sc= is not an undefined first-screen attribute)" \ || no "(E2) the whole-file serving carries no legend defining sc=/the composition" # the composed id still RESOLVES, which is the whole contract - "$BIN" test/nestedqualfix --expand='outer.hpp::Outer::Outer' --no-cache >"$TMP/wf2.xml" 2>/dev/null "$TMP/wf2.xml" 2>/dev/null wrapper, no p= repetition) — pinned so the design # stays a decision rather than an omission nobody re-checked. @@ -205,7 +211,7 @@ else no "(E3) the probe did not serve the bundle shape (root says $E3_MODE) — this arm proves nothing" fi # …and the positive control: a REAL whole-file serving still selects expand-file. -"$BIN" test/nestedqualfix --expand=Outer --legend=compact --no-cache >"$TMP/e3b.xml" 2>/dev/null "$TMP/e3b.xml" 2>/dev/null "$R/alt.cpp" cat > "$R/DOC.md" <<'EOF' # Design @@ -123,10 +126,13 @@ grep -q 'head_labels="index"' "$W" \ && ok '§A7: head_labels="index" — HEAD rows are the parsed answer, and say so' \ || { no '§A7: head_labels="index" missing'; grep -o ']*>' "$W"; } -# ── §A7(ii) the ref-blob side keeps its LEXICAL heuristic — and now reads the same shape correctly ────── -grep -q '^feat-doc def DOC\.md:' "$TMP/rows" \ - && ok '§A7: a branch DOC quoting the signature still reads kind="def" (the lexical residual is intact, and now handles noexcept)' \ - || { no '§A7: the feat-doc DOC.md row is not a def row (the blob-side heuristic still misreads the house style)'; cat "$TMP/rows"; } +# ── §A7(ii) the ref-blob side reads the house style correctly ────────────────────────────────────────── +# Fix round 2 (rule 4, by design): this arm asserted the lexical residual — "a branch DOC quoting the signature still reads +# kind="def"" — and that residual is what the parser removed. Its two halves now: the doc's quoted signature is a ref on the +# branch (as on HEAD), and the noexcept-terminated definition on the branch is a def (the chooser keeps the house style). +{ grep -q '^feat-doc ref DOC\.md:' "$TMP/rows" && ! grep -q '^feat-doc def DOC\.md:' "$TMP/rows" && grep -q '^feat-doc def alt\.cpp:1$' "$TMP/rows"; } \ + && ok '§A7: on the branch, the DOC quoting the signature reads kind="ref" (as on HEAD) and the noexcept-terminated alt.cpp definition reads kind="def"' \ + || { no '§A7: the branch DOC.md row is still a def, or the noexcept-terminated branch definition lost its label'; cat "$TMP/rows"; } # ── §A7(iii) the scan denominator names its noun ──────────────────────────────────────────────────────── { grep -q 'refs_scanned="' "$W" && ! grep -q ']* refs="' "$W"; } \ @@ -134,16 +140,16 @@ grep -q '^feat-doc def DOC\.md:' "$TMP/rows" \ || { no '§A7: whereis root still spells the scan denominator refs='; grep -o ']*>' "$W"; } # determinism + well-formedness of the changed verb -# TWIN on the default listing: the same def rows (HEAD's exactly the index def site, the branch doc's lexical def), no +# TWIN on the default listing: the same def rows (HEAD's exactly the index def site, the branch's alt.cpp definition), no # ref row printed, and the ref rows of the whole list — core.cpp:11 among them — counted exactly by . WD="$TMP/wd.xml" "$BIN" "$R" --whereis=parseArgs --limit=200 >"$WD" 2>/dev/null tr '<' '\n' <"$WD" | sed -n 's/^hit ref="\([^"]*\)".* p="\([^"]*\)" l="\([0-9]*\)" kind="\([a-z]*\)".*/\1 \4 \2:\3/p' >"$TMP/rowsd" NREFALL="$( grep -c ' ref ' "$TMP/rows" )" { [ "$( grep ' def ' "$TMP/rowsd" )" = "$( grep ' def ' "$TMP/rows" )" ] && ! grep -q ' ref ' "$TMP/rowsd" \ - && [ "$( sed -n 's/^HEAD //p' "$TMP/rowsd" | grep '^def ' )" = "def core.cpp:4" ] && grep -q '^feat-doc def DOC\.md:' "$TMP/rowsd" \ + && [ "$( sed -n 's/^HEAD //p' "$TMP/rowsd" | grep '^def ' )" = "def core.cpp:4" ] && grep -q '^feat-doc def alt\.cpp:1$' "$TMP/rowsd" \ && grep -q '^HEAD ref core\.cpp:11$' "$TMP/rows" && grep -q "]*>' "$WD"; } "$BIN" "$R" --whereis=parseArgs --limit=200 --whereis-listing=all >"$TMP/w2.xml" 2>/dev/null diff --git a/test/selfcheckcheck.sh b/test/selfcheckcheck.sh index 4167ee89e..a6b1e3f75 100755 --- a/test/selfcheckcheck.sh +++ b/test/selfcheckcheck.sh @@ -152,6 +152,7 @@ ALLOW = { "isDefinitionNotDeclaration": "read-only span comparison (model.h); no state — the predicate std::all_of walks above", "isFunctionLikeKind": "read-only enum comparison (graph.h, a local lambda); no state — the F2 body-test scope guard", "any": "read-only member predicate (pathgaps.h PathSearchGaps::any: calls > 0); no state — the gap-cone postcondition", + "inCheckout": "read-only member predicate (crossref.h WhereHit::inCheckout: fromWorktree || ref == \"HEAD\"); no state — labelJobRows' precondition that a parse job holds no checkout row", } # (T) ASSUMED-THEN-TESTED allowlist: "site -> reason", one line each. A site lands here only when the equality # really is a true invariant (the re-test below is dead defensive code that should eventually be deleted, not diff --git a/test/shapingflagcheck.sh b/test/shapingflagcheck.sh index 5ac0b39c8..b8b8d01ff 100755 --- a/test/shapingflagcheck.sh +++ b/test/shapingflagcheck.sh @@ -111,12 +111,16 @@ git status --porcelain 2>/dev/null | grep -vE '^\?\? (build|asan|tsan)' | LC_ALL # decide whether the ride-along note='s bytes should be charged to the search/verdict, the SAME predicate # `noteAppliesToBundle` already evaluates further down for the SAME two verbs (--expand/--outline). It does # not change which verbs honour --top-k, only fixes an existing verb's ceiling math, so this is a re-pin. +# expand-lean-k64 (2026-10-08, 21->22 / 14->15): ONE new read of each, the same kind as V1's. `leanEligible` (src/main.cpp) +# is the multi-definition twin of V1's exact-name predicate — it reads cfg.topKExplicit and cfg.maxTokens to decide whether +# the lean default applies (an explicit --top-k and a --max-tokens-sized map both stand it aside). --expand/--outline +# already honour both flags in kShapingVerbs, so this is a re-pin, not a column change. MAXSITES="$( grep -c 'cfg\.maxTokens\|c\.maxTokens' src/main.cpp src/verbs_*.h src/mcpserver.h 2>/dev/null | awk -F: '{s+=$2} END{print s+0}' )" TOPSITES="$( grep -c 'cfg\.topK\|c\.topK' src/main.cpp src/verbs_*.h src/mcpserver.h 2>/dev/null | awk -F: '{s+=$2} END{print s+0}' )" -[ "$MAXSITES" = 21 ] && ok "(A) --max-tokens has 21 read sites outside cli.h (grep 'cfg\\.maxTokens' src/main.cpp src/verbs_*.h src/mcpserver.h)" \ - || no "(A) --max-tokens read sites moved 21 -> $MAXSITES: a verb gained or lost the budget, so kShapingVerbs' honorsMaxTokens column must be re-decided (and this number re-pinned)" -[ "$TOPSITES" = 14 ] && ok "(A) --top-k has 14 read sites outside cli.h" \ - || no "(A) --top-k read sites moved 14 -> $TOPSITES: re-decide kShapingVerbs' honorsTopK column and re-pin this number" +[ "$MAXSITES" = 22 ] && ok "(A) --max-tokens has 22 read sites outside cli.h (grep 'cfg\\.maxTokens' src/main.cpp src/verbs_*.h src/mcpserver.h)" \ + || no "(A) --max-tokens read sites moved 22 -> $MAXSITES: a verb gained or lost the budget, so kShapingVerbs' honorsMaxTokens column must be re-decided (and this number re-pinned)" +[ "$TOPSITES" = 15 ] && ok "(A) --top-k has 15 read sites outside cli.h" \ + || no "(A) --top-k read sites moved 15 -> $TOPSITES: re-decide kShapingVerbs' honorsTopK column and re-pin this number" # no OTHER file may read them: a third file would be a verb family this table has never heard of. # 2026-08-29 main.cpp split: src/verbs_*.h are SECTIONS of main.cpp's own TU (RIPWIRE_MAIN_TU-guarded), # so they count as main.cpp in this derivation — the counts above sweep them, the exclusion below too. diff --git a/test/showcase_capture.py b/test/showcase_capture.py index 745f4c257..08c6015b9 100644 --- a/test/showcase_capture.py +++ b/test/showcase_capture.py @@ -352,6 +352,7 @@ def add(section, cmd, what, **opts): S1 = "understand a codebase cold" add(S1, f"{BIN} .", "The default ranked symbol map — start here when landing cold in a repo.") add(S1, f"{BIN} . --top-k=5", "Same map, capped to the 5 highest-ranked symbols.") +add(S1, f"{BIN} . --top-k=5 --no-inventory", "The same capped map without its inventory tier: --top-k=5 leaves files out, so the default map appends (program entries, and the files the ranked rows left out) after its rows; --no-inventory drops that tier, and the ranked rows are the same.") add(S1, f"{BIN} . --top-k=0 --expand=rankGraphTeleport", "NEW since the last capture: --top-k=0 means PAYLOAD-ONLY — no ranked map rides along with the body you asked for.") add(S1, f"{BIN} . --top-k=0", "--top-k=0 with NO payload verb asked for — REFUSES (exit 1) naming the payload verbs, never an empty map.") add(S1, f"{BIN} . --max-tokens=1500", "SHAPE the map to fit ~1500 tokens (binary-search top-K).") diff --git a/test/taskroutecheck.sh b/test/taskroutecheck.sh index 197f49e31..a2d0cc21b 100755 --- a/test/taskroutecheck.sh +++ b/test/taskroutecheck.sh @@ -11,6 +11,7 @@ fail=0 ok(){ printf ' PASS %s\n' "$*" || { fail=1; printf ' FAIL could not write the PASS line for: %s\n' "$*"; }; return 0; } no(){ printf ' FAIL %s\n' "$*"; fail=1; } +echo "taskroutecheck: BIN=$BIN" [ -x "$BIN" ] || { echo "no ripwire binary at $BIN — build first"; exit 2; } REPO="$TMP/router repo" mkdir -p "$REPO" @@ -643,6 +644,26 @@ for f in --verify --connect --expand --grep --grep-context --edit-check --from-t --edit-plan --dry-run --handles --legend --doctor --agent=codex --test-gate --slice --slice-flow --at --uses --seams \ --rank-by= --limit=; do "$BIN" --help=all 2>&1 | grep -q -- "$f" || no "recommended flag absent from --help: $f"; done +# ── K50: a NON-ASCII prompt is not a sanitizer error (lowerAscii, the name-shape and FILE:LINE predicates) ────────────── +# `for( const unsigned char c : text )` over a string_view of `char` converts every byte >= 0x80 implicitly; G1's +# implicit-integer-sign-change makes that a hard abort on the sanitizer build, so every --help-task prompt with an +# accented letter, CJK or an emoji died there (K50). The arms below are RED only on a sanitizer binary (run the gate +# with BIN=asan/ripwire); on a plain build they prove the answer is still a router document and is unchanged by the +# non-ASCII bytes, which is also what a regression to a locale-sensitive tolower would break. +for kp in 'where is the résumé parser called from' 'what is at router.cpp:é9' 'how does `Ünï.cöde` work' 'explain computeBudget and 路由 plus 🙂'; do + routeRaw "$kp" >"$TMP/k50.out"; k50rc=$? + if [ "$k50rc" = 0 ] && grep -q ', no sanitizer report): $kp" + else + no "K50: a non-ASCII prompt died or reported (rc=$k50rc): $kp :: $( head -c 200 "$TMP/err" )" + fi +done +# The negative control: the non-ASCII bytes are noise to the router — the same prompt without them routes identically. +K50A="$( route 'how do alphaNode, betaNode and gammaNode all come together' )" +K50B="$( route 'how do alphaNode, betaNode and gammaNode all come together, café' )" +[ "${K50A%%resolved_symbols*}" = "${K50B%%resolved_symbols*}" ] && case "$K50B" in *'intent="connect-symbols"'*) ok "K50: a trailing non-ASCII word leaves the --connect route unchanged";; *) no "K50: the control did not route --connect: $K50B";; esac \ + || no "K50: a trailing non-ASCII word changed the route: [$K50A] vs [$K50B]" + # ── byte-compat: the verify-claim template must emit the SHIPPED --verify grammar byte-exactly ───────── # (PLAN 2026-08-13 addendum: gate against the real verb's PARSER, never a copy of its syntax.) # Route a claim through --help-task, unquote the recommended argv robustly (a real XML parse + a real diff --git a/test/tokenbudgetcheck.sh b/test/tokenbudgetcheck.sh index ef91a893c..8a97a432b 100755 --- a/test/tokenbudgetcheck.sh +++ b/test/tokenbudgetcheck.sh @@ -236,6 +236,34 @@ GATE_EST="$( grep -oE 'est_tokens=[0-9]+' "$TMP_TB_B_ERR" | grep -oE '[0-9]+' )" && ok "gate value ($GATE_EST) == map header's est_tokens ($HDR_EST) — one counter, not two" \ || no "gate/header est_tokens MISMATCH (header=$HDR_EST gate=$GATE_EST) — a second counter crept in" +# ── #9b (map-inventory-d7, 2026-10-08): the SAME equality on the boundary where the compact reprice drops a digit. The gate +# used to recount the compacted body's size AFTER the layer had rewritten the est digits (a 5-digit full price repriced to 4 +# digits is 2 bytes shorter in the root and the header), so there it said one token less than the header (7608d1f6: +# src --top-k=233, header 9965, gate 9964). The arm LOCATES that boundary on today's src/ (it moves with the tree): the +# smallest top-k whose FULL price reaches 10000, then every k up to 8 past it whose compact header is still 4 digits. +full_est(){ "$BIN" src --top-k="$1" --legend=full --token-budget=1 --no-cache 2>&1 >/dev/null | grep -oE 'est_tokens=[0-9]+' | grep -oE '[0-9]+'; } +lo=1; hi=4000 +if [ "$( full_est $hi )" -ge 10000 ] 2>/dev/null; then + while [ $(( hi - lo )) -gt 1 ]; do mid=$(( ( lo + hi ) / 2 )); if [ "$( full_est $mid )" -ge 10000 ] 2>/dev/null; then hi=$mid; else lo=$mid; fi; done + crossed=0; bad="" + for k in $( seq "$hi" $(( hi + 8 )) ); do + H="$( "$BIN" src --top-k="$k" --token-budget=999999 --no-cache 2>/dev/null | grep -oE 'est_tokens=[0-9]+' | head -1 | grep -oE '[0-9]+' )" + [ -n "$H" ] && [ "$H" -lt 10000 ] 2>/dev/null || continue + crossed=$(( crossed + 1 )) + G="$( "$BIN" src --top-k="$k" --token-budget=1 --no-cache 2>&1 >/dev/null | grep -oE 'est_tokens=[0-9]+' | grep -oE '[0-9]+' )" + [ "$G" = "$H" ] || bad="$bad k=$k(header=$H,gate=${G:-?})" + done + if [ "$crossed" -eq 0 ]; then + no "#9b premise: no top-k in $hi..$(( hi + 8 )) has a 5-digit full price and a 4-digit compact one — the arm tested nothing" + elif [ -z "$bad" ]; then + ok "#9b gate == header on all $crossed digit-crossing top-k (from $hi): the gate reads the printed price" + else + no "#9b gate/header MISMATCH where the compact reprice drops a digit:$bad" + fi +else + no "#9b premise: src --top-k=4000 prices under 10000 tokens — no digit boundary to probe" +fi + # ── #10: determinism — the same command re-run twice gives the same exit code + the same est_tokens ─── "$BIN" src --token-budget=1 --no-cache >/dev/null 2>"$TMP_TB_B2_ERR"; rc_tiny2=$? GATE_EST2="$( grep -oE 'est_tokens=[0-9]+' "$TMP_TB_B2_ERR" | grep -oE '[0-9]+' )" diff --git a/test/truncvocabcheck.sh b/test/truncvocabcheck.sh index 503aaf4c8..b8e013af8 100755 --- a/test/truncvocabcheck.sh +++ b/test/truncvocabcheck.sh @@ -502,5 +502,124 @@ else ok "G4: xmllint unavailable (skipped)" fi +# ── (H) ARM A of the paging spec (upstream issue #294, scope corrected by measurement) ────────── +echo +echo "── (H) ARM A: the BUDGETED bundle continuation carries the pageview quintet ──" +# SCOPE, corrected by what the tree already does (measured 2026-09-19, on this binary): +# * `--for=TASK --limit=N` (+offset=M) is the FILE-GRAIN WIDENING PAGE — it exists, it carries the +# full pageview quintet, and forwidencheck owns it. Re-gating it here would violate §6.4 (the gate +# that already owns the subject takes the arm; a duplicate arm is coverage-shaped noise). +# * The MCP `for` verb serves that widening page today (limit=5 → ); the MCP `explore` verb refuses limit ("unknown field"). +# * `--for --token-budget=N --offset=M` REFUSES (stdout empty). The byte-budgeted compact bundle — +# the thing whose cut IS disclosed (over_ceiling= reason=) but with NO continuation handle — is the +# actual open gap of issue #294. THIS arm guards exactly that: the budgeted bundle, paged by +# candidate offset, must carry the pageview quintet, and offset=0 must be byte-identical to the +# un-paged budgeted answer (the regression floor: existing callers must see identical output). +# RED on the pre-feature binary is the intended state — gate written BEFORE the code, red vs the +# pre-change binary, same discipline as runtracecheck.sh. It greens when the continuation ships. +paged(){ local name="$1"; shift; run "$@" > "$TMP/page_$name.xml"; } +paged for_budget_p0w "$BIG" --for="rank symbols" --token-budget=800 --limit=6 --offset=0 +paged for_budget_page "$BIG" --for="rank symbols" --token-budget=800 --offset=5 +paged pack_budget_page "$BIG" --pack-task="rank symbols by pagerank" --token-budget=800 --offset=5 +run "$BIG" --for="rank symbols" --token-budget=800 --offset=0 > "$TMP/page_for_budget_alone.xml" # offset=0 ALONE = the un-paged bundle +run "$BIG" --for="rank symbols" --token-budget=800 > "$TMP/page_for_budget_nopage.xml" + +python3 - "$TMP" <<'PY' +import os, sys, xml.etree.ElementTree as ET + +tmp = sys.argv[1] +QUINTET = ("shown", "total", "capped", "has_more", "next_offset") +problems = [] + +def load(name): + path = os.path.join(tmp, f"page_{name}.xml") + if not os.path.exists(path) or os.path.getsize(path) == 0: + problems.append(f"{name}: the paged run produced no output — the page arm asserted nothing " + "(today: --token-budget refuses --offset; that refusal IS the red this arm exists to record)") + return None + raw = open(path, encoding="utf-8", errors="replace").read() + try: + return ET.fromstring(raw), raw + except ET.ParseError as e: + problems.append(f'{name}: paged output is not parseable XML ({e}) — a paged task-lens answer must be a document') + return None + +paged_roots = {} +for name in ("for_budget_p0w", "for_budget_page", "pack_budget_page"): + loaded = load(name) + if loaded is None: continue + paged_roots[name] = loaded[0] + root, _ = loaded + a = root.attrib + missing = [k for k in QUINTET if k not in a] + if missing: + problems.append(f'{name}: paged budgeted root lacks the pageview quintet (missing: {", ".join(missing)}) — ' + "the continuation contract pageview.h owns for every row verb is absent on the budgeted bundle") + continue + if a["capped"] not in ("0", "1"): + problems.append(f'{name}: capped="{a["capped"]}" is not the 0|1 bit (rule 3)') + if int(a["shown"]) > int(a["total"]): + problems.append(f'{name}: shown={a["shown"]} > total={a["total"]} — a page cannot show more than the candidate set') + # issue #294 ruling, suggestions 1+3: above_cliff (the head-tier count, next to total=) and the + # at= index stamp (which index answered; a consumer compares across pages instead of mixing two + # rankings silently). above_cliff must be < total unless the whole set is head-tier. + if "above_cliff" not in a: + problems.append(f'{name}: paged root lacks above_cliff= — the ruling\'s suggestion 1 (decide from page 1 whether the remainder is worth fetching)') + elif int(a["above_cliff"]) >= int(a["total"]) and int(a["total"]) > 0: + problems.append(f'{name}: above_cliff={a["above_cliff"]} >= total={a["total"]} — the boundary is not inside the set') + if "at" not in a: + problems.append(f'{name}: paged root lacks at= — the index stamp the ruling\'s suggestion 3 names (a stale continuation must be detectable, not silent)') + # ARM C of the paging spec, housed here because THIS gate owns the rule-5 vocabulary (the + # §B8.3 sweep): within a candidate page, a body byte-trim must still disclose shown=/total=/capped= + # on — the candidate cursor advances the SET, rule 5 discloses the within-page byte trim. + for el in root.iter(): + ea = el.attrib + if ea.get("capped") == "1" and el.tag.lower().endswith("bodies"): + if "shown" not in ea or "total" not in ea: + problems.append(f'{name}: <{el.tag} capped="1"> within the page carries no ' + "shown=/total= — the within-page byte trim is undisclosed (rule 5)") + +# ARM F of the paging spec (CONTINGENT on the #294 ruling; it encodes the PROPOSED answer): the cliff +# boundary is computed ONCE over the full ranked candidate set — the boundary value reported must be +# IDENTICAL on every page of the same query. If maintainers pick the per-page re-cliff alternative, this +# assert is rewritten to that contract before any code lands (the spec does not assume its own question). +cliff_attrs = {} +for name, root in paged_roots.items(): + for el in root.iter(): + for key, val in el.attrib.items(): + if key in ("cliff_rank", "cliffrank", "cliff"): + cliff_attrs.setdefault(key, set()).add(val) +if cliff_attrs: + for key, vals in cliff_attrs.items(): + if len(vals) > 1: + problems.append(f"cliff {key} moved across pages of the same query ({sorted(vals)}) — the boundary " + "must be computed once over the full candidate set, not re-decided per page") + +alone = os.path.join(tmp, "page_for_budget_alone.xml") +nopage = os.path.join(tmp, "page_for_budget_nopage.xml") +if os.path.exists(alone) and os.path.getsize(alone) > 0 and os.path.exists(nopage) and os.path.getsize(nopage) > 0: + if open(alone, encoding="utf-8", errors="replace").read() != open(nopage, encoding="utf-8", errors="replace").read(): + problems.append("for_budget_alone: --offset=0 ALONE must be byte-identical to the un-paged budgeted " + "answer (the un-paged bundle is the byte-pinned ladder, forbudgetmonotoncheck's ground — " + "the page contract ADDS a windowed mode, it must not touch the un-paged bytes)") + +# mutation: the assertion shape can fail — a bare paged root with no continuation handle must be SEEN +mut = ET.fromstring('').attrib +if sorted(k for k in QUINTET if k not in mut) != ["has_more", "next_offset"]: + problems.append("mutation: a bare paged root with no has_more=/next_offset= was NOT detected — the arm cannot fail") +else: + print(' PASS (H) mutation: a bare paged root (no has_more=/next_offset=) IS detected') + +for p in problems: print(" FAIL " + p) +print(" .. (H) budgeted-bundle paged captures checked: 3, plus the offset=0 byte-identity floor") +sys.exit(1 if problems else 0) +PY +if [ $? = 0 ]; then + ok '(H) paged budgeted-bundle roots carry the full pageview quintet; offset=0 is byte-identical to the un-paged budgeted answer' +else + no '(H) the budgeted-bundle continuation contract (upstream issue #294, corrected scope) does not hold — see FAIL lines above' +fi + [ $fail = 0 ] && echo "ALL PASS" || echo "FAILURES" exit $fail diff --git a/test/usesselectorcheck.sh b/test/usesselectorcheck.sh index 2117c262c..cde67bc80 100755 --- a/test/usesselectorcheck.sh +++ b/test/usesselectorcheck.sh @@ -142,7 +142,9 @@ grep -q 'srcmut_sigchange' "$TMP/err" && no "the constant nonsense suggestion (s # fetch a canonical id; --outline always rides the classic 200-row map (no V1 lean default applies to # it) and its rows carry id= unconditionally, so it is the stable lookup path here. # row 6 (2026-09-12): the row prints the short id sc=; the canonical id composes as ::sc::n -CANON_ID="$( "$BIN" "$ROOT" --outline='src/notes.h:empty' --no-cache 2>/dev/null | python3 -c ' +# expand-lean-k64 (2026-10-08): --outline of definitions it fully serves no longer rides the map by default; the lookup asks +# for the classic ranked map explicitly (--top-k=200, the old default), which is what carries the id= rows read here. +CANON_ID="$( "$BIN" "$ROOT" --outline='src/notes.h:empty' --top-k=200 --no-cache 2>/dev/null | python3 -c ' import re, sys doc, name, scope = sys.stdin.read(), sys.argv[1], ( sys.argv[2] if len( sys.argv ) > 2 else None ) for f in re.finditer( r"]*>(.*?)", doc, re.S ): diff --git a/test/wrapverbscheck.sh b/test/wrapverbscheck.sh index 903b595b4..270ceccf7 100755 --- a/test/wrapverbscheck.sh +++ b/test/wrapverbscheck.sh @@ -289,27 +289,35 @@ else fi echo -echo "=== 8. one-shot --for recipes budget with --token-budget, never --max-tokens ===" +echo "=== 8. one-shot --for recipes name no budget (never --max-tokens, never a --token-budget) ===" # --for does not read --max-tokens: it warns on stderr and emits the full, unbudgeted result. # --token-budget is the flag that actually shapes --for's output. The wrap recipes are the tool's # own advice, so they must not teach the inert pairing (found live 2026-08-20: wrap claude # recommended `--for="" --max-tokens=2000`, which produced an unbudgeted map + a warning). -for _agent in claude codex cursor windsurf gemini opencode aider; do +for _agent in claude codex cursor windsurf gemini opencode openclaw hermes aider; do _out="$( "$BIN" wrap "$_agent" 2>/dev/null )" - if echo "$_out" | grep -- '--for=' | grep -q -- '--max-tokens'; then + if ! echo "$_out" | grep -q -- '--for='; then + no "wrap $_agent printed no --for= line — the --max-tokens check below would pass on nothing" + elif echo "$_out" | grep -- '--for=' | grep -q -- '--max-tokens'; then no "wrap $_agent pairs --for with --max-tokens — inert advice, --for ignores that flag" else ok "wrap $_agent never pairs --for with --max-tokens" fi done -# the three recipes that ship a budgeted one-shot --for line must budget it with --token-budget= -for _agent in claude opencode aider; do +# K29 (2026-10-08): the one-shot --for recipes name NO budget: all five CLI-first agents (claude, opencode, codex, +# openclaw, hermes) print the same line, plus aider's repo-map line. A budget handed to an agent is a ceiling it +# pastes for every task: in round-2 mechanical (provisional) scoring --token-budget=2000 answered fewer of the +# measured questions than the default, and the owner rule is lean AND answered. The default budget shapes the +# answer; the recipe must not narrow it. +for _agent in claude opencode codex openclaw hermes aider; do _out="$( "$BIN" wrap "$_agent" 2>/dev/null )" - if echo "$_out" | grep -- '--for=' | grep -q -- '--token-budget='; then - ok "wrap $_agent one-shot --for recipe carries --token-budget=" + if ! echo "$_out" | grep -q -- ' --for=""'; then + no "wrap $_agent lost its one-shot --for=\"\" recipe line" + elif echo "$_out" | grep -- '--for=' | grep -q -e '--token-budget' -e '--max-tokens'; then + no "wrap $_agent one-shot --for recipe carries a budget flag — it must recommend the default budget" else - no "wrap $_agent one-shot --for recipe lost its --token-budget= budget" + ok "wrap $_agent one-shot --for recipe is present and names no budget flag" fi done