diff --git a/TODO.md b/TODO.md index f1dd364d6c..bb7a6e66d4 100644 --- a/TODO.md +++ b/TODO.md @@ -1453,9 +1453,11 @@ this as intentional accounting. - [x] O27b: Screen direct mapped-node construction on current main. All 28 fixture oracles and focused tests pass on both hosts, but the screen does not justify adoption or full timing. Both decisions and the lossless evidence are in the [O27 report](docs/benchmarks/archive-index-insertion-20261107.md). -- [ ] O27c: Independently measure transparent lower_bound plus hinted ancestor insertion. - Preserve duplicate overwrite, synthesis flags, child counts and inode order. Public - construction/destruction benchmarks include parsing/source costs; hot-listing setup does not. +- [x] O27c: Screen transparent lower_bound plus hinted archive ancestor insertion on both hosts. + All 28 construction oracles and focused tests pass. Linux shape medians improve 2.89-4.35%, + but Mac flat/100,000 costs 13.97% with same-image repeat -0.79%; do not adopt the runtime. + Retain the isolated prototype and every raw case in the [hinted-insertion report](docs/benchmarks/archive-ancestor-hint-20261107.md). + All three requested O27 mechanisms are now evaluated; further index work needs a new hypothesis. - [x] O29: [Qualify borrowed archive member keys and bounded parent-length checks](docs/benchmarks/archive-member-keys-20261105.md). Both full 333-target release suites, relevant sanitizers, actual changed-TU lint, exact ordinary release images and 15/15 lines plus 2/2 branches pass. All 14 targeted operations diff --git a/docs/benchmarks/archive-ancestor-hint-20261107.json.gz b/docs/benchmarks/archive-ancestor-hint-20261107.json.gz new file mode 100644 index 0000000000..ba5671c5e6 Binary files /dev/null and b/docs/benchmarks/archive-ancestor-hint-20261107.json.gz differ diff --git a/docs/benchmarks/archive-ancestor-hint-20261107.md b/docs/benchmarks/archive-ancestor-hint-20261107.md new file mode 100644 index 0000000000..a47f0b9011 --- /dev/null +++ b/docs/benchmarks/archive-ancestor-hint-20261107.md @@ -0,0 +1,80 @@ +# Archive ancestor insertion hints, 2026-11-07 + +Keep the production index unchanged. Reusing an ordered ancestor lookup for hinted +insertion improves the Linux constructor screen, but Mac results are mixed and include +a substantial unchanged-flat-path cost. This trial does not qualify portable adoption. + +## Isolated mechanism + +The candidate retains owned ancestor keys, default mapped-node construction and its +subsequent assignment. Member insertion, duplicate overwrite, direct-child counts, +synthesis flags and inode order stay unchanged. Only the missing-ancestor search +changes: `lower_bound` supplies the insertion position to `try_emplace`, avoiding a +second ordered lookup. This differs from the earlier rejected borrowed-key and direct +mapped-value-construction trials; neither change is included here. + +A nonempty ancestor is a strict prefix of the already-stored member. Under the map's +`std::less<>` ordering, that member is greater than the ancestor, so `lower_bound` +cannot return `end()`. The lower bound plus the reverse comparator determines equality. +No other map mutation happens before the hinted insertion. + +## Screen and provenance + +The unchanged frozen baseline is main `9bf1aad5`, reused byte-for-byte from the preceding +O27 experiment. Candidate/baseline configurations, compiler version (Clang 23.1.4) and +recorded source hashes match except for the intended index implementation. Both use +`clang_release` and `xff_full`. The exact baseline image is repeated as a control. + +Each host runs every one of the 28 public construction/destruction cases: flat, +wide-short/wide-long, deep-short/deep-long and chain archives, with 10 through 100,000 +members where supported; the chain cases stop at 1,000. This includes real public +archive parsing/source/index/destruction work, not a hot private map lookup. Complete +fixture metadata/path/directory oracles run before timing. Every oracle passes. + +This is one initial screen with three raw repetitions per case/participant; retain +the fastest two and average them. The table shows the median case change per shape, +not elapsed-time weighting. Linux is pinned to one allowed CPU; Mac is unpinned. +No host builds or sanitizer checks run concurrently with timing. Positive is slower. + +| Shape | Mac candidate | Mac same-image repeat | Linux candidate | Linux same-image repeat | +| :--------- | ------------: | --------------------: | --------------: | ----------------------: | +| flat | +3.46% | -1.80% | -4.35% | +0.38% | +| wide-short | +10.53% | -1.09% | -3.20% | +0.59% | +| wide-long | +2.65% | +0.14% | -2.89% | +0.55% | +| deep-short | +1.98% | -4.83% | -3.93% | +0.35% | +| deep-long | -3.71% | -0.86% | -2.89% | +0.48% | +| chain | -4.40% | -0.74% | -4.28% | -0.88% | + +The Mac flat/100,000 case costs 13.97% while the identical baseline repeat changes +-0.79%. The modified ancestor branch does not execute for flat members; no causal +explanation is claimed from source inspection alone. Keep that counterexample and +the noisy tiny cells, including Mac deep-short/10 at +30.66% (repeat -21.61%). +Linux's consistently favorable shape medians do not erase the Mac costs. + +## Correctness and decision boundary + +Both hosts pass optimized `archive_fs_test`, `archive_directory_metadata_test` and +`archive_openbytes_ownership_test`. The additional in-memory regression checks +synthesized ancestors alongside lexically adjacent member keys and their contents; +existing tests cover explicit/synthesized replacement, duplicate/direct-child counts, +inodes, metadata and retained ownership. Test source, Mac XML and the Linux complete +build/test log are retained. This is focused prototype validation, not a claim of a +full release/sanitizer/current-head CI run. + +The first Mac attempt failed before any timing because copied baseline files lacked +executable permissions. Only their mode was corrected, failed setup logs remain, +and the complete corrected screen is separate. No source or measured image bytes +changed as a recovery, and no successful screen was repeated until favorable. + +The predeclared next stage required a promising cross-platform screen. That condition +is not met, so no full three-session nine-repeat/retain-seven population, native +82-case CLI/88-case listing controls, sanitizer expansion or runtime PR follows. +Retain the candidate rather than hiding the negative result or changing production +code on a Linux-only improvement. + +## Evidence + +[`archive-ancestor-hint-20261107.json.gz`](archive-ancestor-hint-20261107.json.gz) is 115,195 bytes, SHA-256 +`8359b99d815469270bbdb90aef7ec8dd11f9156ef37d6b7c5a879672badfaad7`. Its 53 exact source/raw/log records include both complete populations, +copy controls, failed setup and per-record byte counts/hashes. Measured executables +remain in the local/remote task directories with image identities recorded. diff --git a/docs/performance-analysis.md b/docs/performance-analysis.md index e7c7bdbb35..af23192928 100644 --- a/docs/performance-analysis.md +++ b/docs/performance-analysis.md @@ -4820,3 +4820,19 @@ Empty valid output, too-small destinations, corrupt data and exact-size mismatch retain committed coverage. Current integration PR checks remain authoritative. This small ownership cleanup does not complete the broader immutable-catalog/first-parse investigation or imply a user-visible command improvement. + +### O27c: reject hinted archive ancestor insertion after both-host screen + +[The isolated hinted-insertion report](benchmarks/archive-ancestor-hint-20261107.md) +retains owned ancestor keys and mapped-value construction, changing only the ordered +lookup and missing-node insertion hint. All 28 complete construction fixture oracles +and three optimized archive targets pass on both hosts. One three-repetition screen +retaining two improves Linux shape medians 2.89-4.35%, but Mac results are mixed: +wide-short +10.53% and flat/100,000 +13.97%, against same-image repeats -1.09% and -0.79%. +The ancestor branch does not execute for flat members; no causal explanation is claimed. + +Stop at the predeclared screen rather than selecting the Linux result or repeating +until favorable. No full three-session population, native/listing expansion or new +full-root/sanitizer qualification is claimed. Preserve both complete populations, +source/image identities, the failed pre-timing executable-mode setup and raw controls. +O27a, O27b and O27c are now separately evaluated without changing the production index. diff --git a/release-site.json b/release-site.json index eef458b398..ddfae9d65f 100644 --- a/release-site.json +++ b/release-site.json @@ -136,7 +136,8 @@ "docs/benchmarks/shared-stage-workers-20261107.md": "docs/benchmarks/shared-stage-workers-20261107.html", "docs/benchmarks/archive-index-insertion-20261107.md": "docs/benchmarks/archive-index-insertion-20261107.html", "docs/benchmarks/transactional-rg-records-20261107.md": "docs/benchmarks/transactional-rg-records-20261107.html", - "docs/benchmarks/language-json-storage-20261107.md": "docs/benchmarks/language-json-storage-20261107.html" + "docs/benchmarks/language-json-storage-20261107.md": "docs/benchmarks/language-json-storage-20261107.html", + "docs/benchmarks/archive-ancestor-hint-20261107.md": "docs/benchmarks/archive-ancestor-hint-20261107.html" }, "generated_html": { "XFF.md": { @@ -160,6 +161,7 @@ ], "files": { "docs/benchmarks/transactional-rg-records-20261107.json.gz": "docs/benchmarks/transactional-rg-records-20261107.json.gz", - "docs/benchmarks/language-json-storage-20261107.json.gz": "docs/benchmarks/language-json-storage-20261107.json.gz" + "docs/benchmarks/language-json-storage-20261107.json.gz": "docs/benchmarks/language-json-storage-20261107.json.gz", + "docs/benchmarks/archive-ancestor-hint-20261107.json.gz": "docs/benchmarks/archive-ancestor-hint-20261107.json.gz" } }