[helly25]

Adapt matching-pipeline scheduling to observed output volume while retaining ordered, credited output. Small or unknown output uses batch publication; substantial output can overlap traversal and matching on the existing persistent workers.

The 1,000-file heavy-output workload improves about 29% on Linux and 33% on Mac, with roughly half the peak process memory. Ordinary results include small costs and unstable Linux populations; the report retains those alongside the gains.

AG;DR

Implementation:

• Use reusable contiguous output transactions with 64 inline bytes, 4 KiB per-slot grants and an 8 MiB shared excess-byte allowance. Charge both old and replacement allocations during growth. The current ordered head may exceed shared credit so an arbitrarily large file can complete binary/read-error validation.
• Start generations on the existing persistent matcher workers and borrow/release results in traversal order. Cancellation wakes byte-credit waiters and drains producers before ownership is recycled. Serial archive and traversal-error boundaries drain preceding host output.
• Bound collecting input by owned text capacity, admitting one oversized record alone. Grow slot containers on demand and recycle cleared predecessor input storage before waking workers.
• Keep small/unknown output and output-free filters batched. An average of at least 1 KiB already-produced output per visited entry enables overlap for the next batch; later small output disables it. Both paths enforce the same byte credit and drain early if a producer blocks.
• Share an N-unit CPU allowance across matcher generations and coordinator collection/publication. Producers return units during output-credit waits and reacquire after dropping the credit mutex, including cancellation. The coordinator returns its unit before awaiting output. No per-file CPU admission lock or timer is introduced.
• Preserve native/rg whole-file suppression, headings, contexts and callback order. Update generated --jobs help and the concurrency design.
• Publish the new report and evidence capsule through release-site.json. The complete local release-site build renders and validates 158 HTML pages and preserves the mapped capsule hash. This fixes the release-site CI failure from the first evidence commit.
• Correct GID conformance fixtures to query actual fixture ownership, which need not match the process group. The unchanged baseline reproduces the distinction; production matching is unchanged.

Qualification:

• All required current-head checks pass at a020d8df08b9a834baea92adb2a872c3eabfb1eb, including done, pre-commit and Trunk, in CI run 42547463433. Patch coverage is 98.20% of lines and 98.67% of branches. Both platform test jobs, all sanitizers, clang-tidy and benchmark comparison/expression jobs pass.
• Based on main after Fix historical Linux backfills on a compatible modern runtime #1150 (52950f937ad371e1450538faf7983024b324688d), with no other PR dependencies. The measured baseline (efffe2529f26c1c7ca61763fa5fbb17efd573f6f) differs only in six workflow/backfill/docs files, with no runtime/toolchain changes.
• Runtime head 4245f78c00f483309b22f08d6ac00fdca54214c7 passes the 270-target root full-feature release invocation and the 73-target standalone extension/API invocation on this Mac and drogon. These are separate invocations, not a count of unique source tests.
• The evidence-only published head 9955b3047e6649abe49d8d78b635dcdccdd33eed passes the combined 343-target release invocation on both hosts. The lean CLI SHA-256 remains identical to each measured runtime image; no timing population is transferred across a changed executable.
• Six focused targets pass ASan/TSan on both hosts and MSan on Linux. Local Linux MSan keeps LLVM22 instrumentation/libc++ and uses the existing LLVM23 ld.lld only because that host lacks the older linker's libxml2 ABI; CI has no override.
• Local clang-tidy passes the affected sources after fixing explicit special-member declarations and scope-only const leases. Formatting and repository-policy hooks pass.
• Each host completes 76 native cells plus 32 crossover/mixed-output cells, with one/three workers, three sessions, nine rounds retaining seven, discarded warmups, rotated participants and exact independent output/exit checks. Linux pins physical cores; Mac makes no affinity claim. A 13-cell memory-backend screen and four first-visible-output diagnostics supplement native results.
• Heavy-output native elapsed changes are -28.88% Linux and -33.16% Mac; peak RSS changes 38.80 -> 18.52 MiB and 79.30 -> 38.12 MiB. First-visible pipe output changes 5.967 -> 3.098 ms and 15.019 -> 10.492 ms. Pipe observations include parent overhead and are not native throughput measurements.
• Ordinary results are mixed. Linux many-directory rg filename output costs 3.78%; small-heavy-small transitions retain 3-4% costs. Mac ordinary line/filename cells are around 0-1%. No universal speedup is claimed.
• A Linux population reports a 26.07% ordinary name cost. Confirmation cohorts change its sign. All allocated sections/addresses of the two CPU builds match except the build-ID note; separate instruction diagnostics show +0.02%/+0.69% retired user instructions for flat/many-directory names. The original observations and independently copied-image controls are retained, without assigning a specific host cause or claiming zero ordinary elapsed cost.
• CI compares 360 Linux cells with main: typical normalized change is +0.4%, with three advisory alarms (+15.97%, +23.20%, +17.33%). Raw XFF changes in those cells are +7.31%, +4.46%, +2.17%, while references change -7.47%, -15.22%, -12.92%. These warnings are retained, not disabled or explained away as a proven host cause. Mac CI has no compatible retained merged baseline, so its cross-run regression status is unknown.
• A scoped paired confirmation uses the exact CI fixtures/scenarios, three sessions of nine rounds retaining seven, independent baseline-image controls and complete NUL-record oracles. Linux changes in the three alarm cases are -1.15%, approximately 0%, and +3.37%; Mac changes are -0.06%, +2.20%, and +0.08%. These do not reproduce a 15%+ local slowdown; they do not establish zero ordinary cost or negate the CI reference-adjusted observations. No runtime source or public head was changed for this confirmation.
• Fixed scheduling, notification, input-recycling, credit-size and CPU-accounting variants, including failed/partial/contaminated populations, remain in the lossless evidence capsule. Independent recomputation verifies 116,424 native measured children across 29 distinct populations. The capsule round-trips all 988 file records and 775 unique payloads.

The detailed qualification report explains mechanisms, all controls, footprint and reproduction. The lean CLI grows 94,448 bytes (0.74%) on Linux and 25,360 bytes (0.60%) on Mac.

Limitations and remaining work:

• Directory I/O retains its independent existing allowance. This is not one process-wide N-operation budget.
• Output credit is not a whole-process memory cap. Source/decoder buffers, derived caches and renderer temporaries remain separate. The ordered-head oversize exception is explicit.
• Output volume does not characterize every computationally expensive low-output matcher. The 1 KiB threshold remains internal; no new scheduling flag is exposed.
• Broader O18/JC source/temporary admission and active-I/O preemption remain open in TODO.md. The separately identified ScanLines span-read candidate is not folded into this PR.
• Required CI completed successfully in this PR context. After merging, the resulting main workflows and publication still require inspection.