Repository navigation
perf(statistics): scale large aggregate buckets (gates F8 mace all-time) - #13
Merged
Merged
Conversation
… resolve, batched inserts) Behavior-preserving perf fixes for the F8 large all-time buckets (gates the ~20735-replay mace all-time recalc). 01-REVIEW findings 3/4/5. - Finding 3: replace the per-player/per-replay O(|identities|) bestPlayerIdentity scan with a steam_id + lower(name) lookup index built once per bucket; resolves the SAME identity (same steam>nickname>display priority, same first-in-array tie-break). Applied to player and commander paths. - Finding 4: add migration 0010 functional index on lower(canonical_players. display_name) (mirrors idx_player_nicknames_nickname); restructure the name-fallback resolve from a per-occurrence correlated `where exists` (OR across cp x pn) into one set-based pass of two index-sargable EXISTS halves (identical predicate). loadPlayerIdentities fan-out left as-is (bounded; dedup would risk tie-break drift). - Finding 5: convert replaceAggregateRows/replaceCommanderRows/replaceBountyRows from per-row INSERT loops to a single multi-row `insert ... select from unnest` per table (byte-identical rows; jsonb columns JSON-serialized per element). Tests: identity-index cross-checks the index vs the prior per-player scan on an adversarial seeded set across a timestamp sweep; batched-write asserts one insert per table and a bounded (1) set-based fallback resolve. Existing real-pg parity harness (per-replay vs set-based, per-type, all-time, is_show) stays green, proving byte-identity. Full pnpm verify green: 100% coverage, OpenAPI diff empty. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Behavior-preserving perf fix for large aggregate recalc buckets. After F8 (#12), the
maceall-time bucket is ONE recalc pass over ~20,735 replays; the parity-driver flagged it stalls. Output is byte-identical — this is perf only.Addresses the three perf findings from the Phase 1 review (3/4/5):
bestPlayerIdentityscanned the entire identities array per player per replay (O(replays × players × |identities|)). Replaced withbuildPlayerIdentityIndex(steam_id + lower(name) maps, original array order preserved) +bestPlayerIdentityIndexed, scoring only the few reachable candidates with the unchanged priority function and first-in-array tie-break. Player + commander paths.0010adds the functional indexidx_canonical_players_display_name_lower(mirrors the existinglower(nickname)index).ensureNameFallbackIdentitiesresolve goes from a per-occurrence correlatedwhere existsto two index-sargable EXISTS halves with a provably identical predicate.replaceAggregateRows/replaceCommanderRows/replaceBountyRowsnow do oneinsert … select * from unnest(…)per table instead of a per-row await loop inside the transaction.Byte-identity proof
identity-index.test.tsdifferentially cross-checks the index vs a verbatim copy of the OLD scan over an adversarial seeded corpus across a timestamp sweep.batched-write.test.tsasserts exactly one insert per table and a bounded set-based fallback resolve.Code review: APPROVE (no blockers; 3 behavior-equivalent medium notes documented in PERF-REVIEW.md).
pnpm verifygreen: 100% coverage, OpenAPI contract diff empty.Deploy:
pnpm db:migrateapplies index0010(additive, index-only, no recalc needed).🤖 Generated with Claude Code