Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ repos:
hooks:
- id: check-added-large-files
# Retain the exact lossless cross-platform performance evidence archives.
exclude: ^docs/benchmarks/((executor-copy-audit|bounded-read-ahead)-20261101|(directory-task-slots|packed-visit-snapshots|bounded-match-results|useful-match-workers|direct-evaluator-workers|derived-content-handoff|immutable-default-catalogs|prepared-diff-configuration|prepared-hash-operands|aggregate-comparison-totals|byte-shingle-representations|archive-input-buffers)-20261102|(digest-and-record-storage|caller-owned-streams|direct-source-materialization|hash-field-binding|hash-field-binding-tag|forward-cursor-skipping|raw-record-reuse|similarity-prefix-view|similarity-reference-reuse|owned-word-moves|smaller-shingle-probes|native-zip-seek-evaluation|encoded-record-reuse|threshold-aware-similarity)-20261103|(column-row-reuse|word-set-insertion|field-context-owned-results|selected-summary-output|fixed-summary-metrics|direct-summary-cells|compact-comparison-entries|comparison-category-sequence|native-name-paths|uniform-native-case|compact-traversal-metadata|eager-native-paths|native-case-experiments)-20261104|(folded-ascii-globs|bulk-directory-metadata|comparison-retention-followups|comparison-storage-followups|archive-prefix-reuse|archive-directory-range|archive-directory-allocation|archive-directory-observations|native-symlink-observations|native-metadata-followups|symlink-target-demand|native-pgo|readlink-short-target|cctz-empty-format|readlink-input-path|readlink-owned-transfer|archive-lookup-trials|archive-member-keys)-20261105|archive-directory-child-capacity-20261106|shared-stage-workers-20261107|archive-index-insertion-20261107|transactional-rg-records-20261107|immutable-zip-locator-evaluation-20261107|queued-read-cancellation-20261107|archive-btree-historical-decision-20261107|compiled-field-stat-demand-20261107|unchanged-site-payloads-20261107|archive-read-buffer-clear-20261107|archive-read-buffer-requalification-20261108|language-list-reserve-20261107|language-list-reserve-requalification-20261108|ordered-output-credit-20261107|adaptive-ordered-output-20261109|direct-line-scan-spans-20261109-(linux|macos))\.json\.gz$
exclude: ^docs/benchmarks/((executor-copy-audit|bounded-read-ahead)-20261101|(directory-task-slots|packed-visit-snapshots|bounded-match-results|useful-match-workers|direct-evaluator-workers|derived-content-handoff|immutable-default-catalogs|prepared-diff-configuration|prepared-hash-operands|aggregate-comparison-totals|byte-shingle-representations|archive-input-buffers)-20261102|(digest-and-record-storage|caller-owned-streams|direct-source-materialization|hash-field-binding|hash-field-binding-tag|forward-cursor-skipping|raw-record-reuse|similarity-prefix-view|similarity-reference-reuse|owned-word-moves|smaller-shingle-probes|native-zip-seek-evaluation|encoded-record-reuse|threshold-aware-similarity)-20261103|(column-row-reuse|word-set-insertion|field-context-owned-results|selected-summary-output|fixed-summary-metrics|direct-summary-cells|compact-comparison-entries|comparison-category-sequence|native-name-paths|uniform-native-case|compact-traversal-metadata|eager-native-paths|native-case-experiments)-20261104|(folded-ascii-globs|bulk-directory-metadata|comparison-retention-followups|comparison-storage-followups|archive-prefix-reuse|archive-directory-range|archive-directory-allocation|archive-directory-observations|native-symlink-observations|native-metadata-followups|symlink-target-demand|native-pgo|readlink-short-target|cctz-empty-format|readlink-input-path|readlink-owned-transfer|archive-lookup-trials|archive-member-keys)-20261105|archive-directory-child-capacity-20261106|shared-stage-workers-20261107|archive-index-insertion-20261107|transactional-rg-records-20261107|immutable-zip-locator-evaluation-20261107|queued-read-cancellation-20261107|archive-btree-historical-decision-20261107|compiled-field-stat-demand-20261107|unchanged-site-payloads-20261107|archive-read-buffer-clear-20261107|archive-read-buffer-requalification-20261108|language-list-reserve-20261107|language-list-reserve-requalification-20261108|ordered-output-credit-20261107|adaptive-ordered-output-20261109|(direct-line-scan-spans|prepared-member-name)-20261109-(linux|macos))\.json\.gz$
- id: check-merge-conflict
- id: check-yaml
# BINARY_TYPES below: archive containers are binary but can START with text (a phar begins with
Expand Down
24 changes: 17 additions & 7 deletions TODO.md
Original file line number Diff line number Diff line change
Expand Up @@ -585,6 +585,10 @@ EP/JC/P work; candidates are not speedup claims until measured on both platforms
+27% on Mac and +11% on Linux. Eliminate those costs without arbitrary count/size thresholds,
preserving identity, budgets, duplicate semantics, exact-content oracles and the rejected
root-offset variant. Native source identity and streamed-ZIP sizes remain separate open work.
- [ ] O19b4: Qualify reuse of validated plain-TAR payload locators independently of name
preparation. The 10,000-member native search still spends roughly 8-13 seconds replaying
headers. Preserve format validation, exact source identity, duplicate selection, nested and
sequential-source fallbacks, and shared read budgets; do not bypass decoder checks.
- [ ] O19b: Qualify archive seeking/index/replay and decoder admission independently. Investigate
native/memory ZIP seeking before concurrency; preserve sequential-source and shared-budget fallbacks.
- [x] O19c: Qualify caller-owned reads for immutable memory and archive-member cursors, including
Expand All @@ -611,13 +615,11 @@ EP/JC/P work; candidates are not speedup claims until measured on both platforms
separate hardware diagnostics are retained in
[the allocation decisions](docs/benchmarks/archive-directory-allocation-20261105.md);
microbenchmark gains alone do not qualify either runtime implementation.
- [ ] O19g3: Prepare the requested member's normalized name once per streamed open,
preserving null-header, normalization, error and duplicate-selection behavior. The
current `MemberSource::Open` repeats `NormalizeMemberName(member_)` while walking
preceding headers. Retain cold/first-member/missing-member controls and measure on
both hosts before adoption. The scanner qualification also exposes roughly eight
seconds for the broad 10,000-member, single-worker TAR invocation; reuse observed plain-TAR payload locators
as a separate, format-validated O19b experiment rather than bypass decoder checks.
- [x] O19g3: Qualify once-per-open requested member-name normalization, preserving nullable
C pathnames, normalization, duplicate/nonregular selection, source ownership and errors.
[Both-host measurements and complete data](docs/benchmarks/prepared-member-name-20261109.md)
retain first/missing/reopened/fresh-wrapper controls, slower results, and the Linux
missing-member microbenchmark tradeoff; no universal speedup is claimed.
- [x] O19i: Bound initial archive-directory result storage to reduce small-listing vector
reallocations, retaining tiny/root/chain costs and complete native controls. Full both-host
qualification and the held O19g variants are in
Expand Down Expand Up @@ -1589,3 +1591,11 @@ this as intentional accounting.
harness versions, slower cases and all raw observations in
[the decision report](docs/benchmarks/language-list-reserve-20261107.md).
Generated immutable catalog representation remains a separate O14 candidate.

### Shared local measurement ownership

- [ ] Coordinate performance measurements and compiler work across simultaneous worktrees/tasks
with an explicit host-level lease. Per-campaign locks alone do not prevent contamination;
retain contention records, wait for foreign work rather than terminating it, and qualify
each affected cohort only after an uncontended rerun. The member-name report records a
concrete benchmark overlap and an independently started MSan compiler.
Binary file not shown.
Binary file not shown.
Loading
Loading