Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
91 changes: 91 additions & 0 deletions .planning/quick/260615-f13b-deaths-undercount/260615-f13b-PLAN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
---
quick_id: 260615-f13b
slug: deaths-undercount
status: in-progress
date: 2026-06-15
branch: fix/f13a-deaths-debug
---

# Quick Task 260615-f13b — Deaths under-count: counter death signal read from stale `parser_events`

## Symptom (proven, staging post-fix #18)

- sg all-time `deaths.total` = **282720**; parser's own d>0 signal = **293744**
(distinct (lower-name, replay) with artifact `d>0`). Recalc loses ~**11024**
death-games vs the parser's own signal — a server-2 aggregation bug.
- Exemplar player **Zero** (`canonical_players.id =
000a2ac8-443d-434c-9d30-ea8613c4c0ed`): recalc sg all-time games=894,
deaths=621; parser has Zero with d>0 in **653** distinct sg replays ⇒ **32**
death-games dropped while their GAME is counted.

## Root cause — CONFIRMED by instrumentation on real data (not code reading)

The recalc has two death sources in `service.ts`:
1. **counter path** — `player_counter` event with `deaths_total`/`deaths_by_teamkills`
(`applyCounterEvent`), and
2. **victim kill-row path** — another player's kill-row names the victim's eid
(`applyVictimDeath`).

Both read from the `events` array, which the repository builds from the
**`parser_events` table** (`loadParserEvents`). But:

- `parser_events` is re-persisted ONLY by the single-replay audit path
(`recalculateParserResult` → `persistParserArtifact` → `replaceParserEvents`).
The **bulk full-run** (`full-run-recalculation.ts`) NEVER re-persists events —
it reads whatever `parser_events` already exist.
- For ~90% of current sg replays (1850 / 2059 with artifact `d` players), the
persisted `parser_events` contain **0 `player_counter` rows**, while
`raw_snapshot` carries the compact counters. (Staging count, query in
`findings.sql`.) The counter death path therefore finds nothing.
- The player evidence (`resolvedPlayers`) AND games are built from
`raw_snapshot`, so the game is counted; only the death — when it has no
independent victim kill-row (null-killer / suicide / environmental / bleed-out
`d>0` with no attacker) — is lost.

**Instrumentation evidence** (`instrument.ts`, real exported rows, real
`buildPlayerIdentityIndex` / `bestPlayerIdentityIndexed` / `playerIdentityMatchPriority`
/ `calculatePlayerAndSquadAggregates`): across 16 of Zero's d>0 replays, Zero
**resolves in all 16** (so the F12-timestamp/nickname-active hypothesis is
REFUTED — he matches via the time-ungated `display_name` priority-1 branch even
for pre-2026-05-10 replays), yet only 6 deaths are credited (3 via counter, 3 via
victim) and **10 are DROPPED** because for those replays `parser_events` has no
counter event for Zero's eid AND no kill-row names his eid as victim.

Concrete dropped replay: **9eb1a2ad-60b8-4692-8520-47d8362c207c (2021-05-14)**,
current prid `3662f461-e6e1-460f-a985-9902e17c6f9e`. raw_snapshot Zero =
`{d:1, nkd:1, eid:203}`; `parser_events` for this prid = 140 kill / 18
destroyed_vehicle / 6 teamkill / 1 diagnostic, **0 player_counter**; no kill-row
with `victim_entity_id=203` ⇒ death dropped, game kept.

So the bug is: **the death counter signal is read from the stale `parser_events`
table instead of the authoritative `raw_snapshot`.**

## Fix (minimal, service-layer + repository wiring)

Read the per-player counter death signal from `raw_snapshot` (the same source
already used for player resolution and games) and fold it through the existing
capped fold, instead of depending on `parser_events.player_counter`.

- Carry the artifact's compact counters (`d` → total, `td` → by_teamkills) onto
the resolved player evidence in `resolvedPlayers` (repository), keyed by the
resolved `playerId`/`squadId`.
- In `calculatePlayerAndSquadAggregates` (service), tally those counter deaths
into the per-replay scratch maps alongside the existing victim path, then
`foldCappedDeaths` as today. The `player_counter` EVENT path stays as a
no-op-when-absent fallback (kept for the audit path / OCAP events), but the
authoritative signal now comes from the artifact.

### Invariants preserved
- One-life cap: ≤1 `total` and ≤1 `by_teamkills` per replay (`foldCappedDeaths`).
- `by_teamkills ≤ total` (counter total = `max(d, td)`, same as `counterDeaths`).
- Games unaffected (player/replay set already from `raw_snapshot`).
- Parser / OCAP contract untouched; no schema or API change.

## Tests
- Regression unit test in `aggregates.test.ts`: a replay whose player carries an
artifact counter death but NO `player_counter` event and NO victim kill-row
must yield `deaths.total = 1` (red before fix, green after).

## Gates
- `pnpm lint`, `pnpm typecheck`, `pnpm test`; `service.ts` 100% unit coverage.
Integration coverage gate needs Docker — note if unavailable.
107 changes: 107 additions & 0 deletions .planning/quick/260615-f13b-deaths-undercount/260615-f13b-SUMMARY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,107 @@
---
quick_id: 260615-f13b
slug: deaths-undercount
status: complete
date: 2026-06-15
branch: fix/f13a-deaths-debug
---

# Quick Task 260615-f13b — Summary

## Confirmed runtime root cause (instrumented on real staging data)

The recalc reads the death **counter** signal from the `parser_events` table,
which is **stale relative to `raw_snapshot`**. The bulk full-run
(`full-run-recalculation.ts`) never re-persists `parser_events` — only the
single-replay audit path (`recalculateParserResult` → `persistParserArtifact` →
`replaceParserEvents`) does. So for ~90% of current sg replays (1850 / 2059 with
artifact `d` players) `parser_events` has **0 `player_counter` rows**, while
`raw_snapshot` carries the compact counters. The death counter path
(`applyCounterEvent`) therefore finds nothing.

Player resolution AND games are built from `raw_snapshot`, so the game is
counted; only the death — when it has no independent victim kill-row (null-killer
/ suicide / environmental / bleed-out `d>0` with no attacker) — is lost. This is
the platform-wide ~11024 death-game undercount.

**Instrumentation** (`instrument.ts`, real exported rows, real
`buildPlayerIdentityIndex` / `bestPlayerIdentityIndexed` / `playerIdentityMatchPriority`
/ `calculatePlayerAndSquadAggregates`): across 16 of Zero's d>0 replays, Zero
**resolved in all 16** — so the F12-timestamp / `isNicknameActive` hypothesis is
**REFUTED**; he matches via the time-ungated `display_name` priority-1 branch even
for pre-2026-05-10 replays. Pre-fix only 6/16 deaths credited (3 counter, 3
victim), **10 DROPPED**; post-fix **16/16, 0 dropped**.

Concrete dropped replay: **9eb1a2ad-60b8-4692-8520-47d8362c207c (2021-05-14)**,
current prid `3662f461-e6e1-460f-a985-9902e17c6f9e`. raw_snapshot Zero =
`{d:1, nkd:1, eid:203}`; `parser_events` = 140 kill / 18 destroyed_vehicle /
6 teamkill / 1 diagnostic, **0 player_counter**; no kill row with
`victim_entity_id=203`. Logged decision: `deathCredited=0, branch=DROPPED`.

The 4 prior code-reading hypotheses failed because the divergence is
runtime-only: the event-table state cannot be inferred from `raw_snapshot` by
reading code — it required querying both and comparing.

## The fix (minimal, service + repository)

Read the per-player death counter from the authoritative `raw_snapshot` (same
source already used for resolution and games), not the stale events table.

- `service.ts` — `AggregatePlayerEvidence` gains optional `counterDeaths?:
DeathStats`. New `tallyArtifactCounterDeaths` folds it into the per-replay
scratch maps (player + squad) alongside the existing event paths. Exported
`artifactCounterDeaths(d, td)` builds the `DeathStats` with the SAME semantics
as the event-based `counterDeaths` (`total = max(d, td)`, `by_teamkills = td`);
`counterDeaths` now delegates to it (DRY).
- `repository.ts` — `resolvedPlayers` attaches `counterDeaths =
artifactCounterDeaths(player.d, player.td)` onto each resolved evidence row.

### Invariants preserved
- One-life cap: ≤1 `total` and ≤1 `by_teamkills` per replay (`foldCappedDeaths`,
unchanged). Additive with the event paths — any overlap collapses to 1.
- `by_teamkills ≤ total` (the `max` cap).
- Games unaffected (player/replay set already from `raw_snapshot`).
- Parser / OCAP contract, schema, and API untouched.

## Tests (red → green)
- `service/tests/aggregates.test.ts` — 4 new unit tests: artifact counter credits
a death with no event and no victim row; teamkill variant (`by_teamkills ≤
total`); no double-count when artifact counter + victim row both present;
N-distinct-replays via artifact counter alone. 3 fail without the fix
(verified), all pass with it.
- `repository/tests/postgres.test.ts` — 1 new integration test: raw_snapshot
`d:1, nkd:1`, parser_events with NO counter and NO victim row → death credited
(`total:1`) through the real DB recalc.

## Gates
- `pnpm lint` — green. `pnpm typecheck` — green. `pnpm format` — green.
- `pnpm test` (unit) — 661 passed (was 657 + 4 new); `aggregates.test.ts` 18/18.
- Coverage: `service.ts` is **100%** (stmts/branches/functions/lines) from the
service-module unit suite. The full `pnpm test:coverage` gate runs the
integration files (testcontainers PostgreSQL/RabbitMQ/S3); **Docker is
unavailable in this environment**, so the repo-wide 100% gate could not be
exercised. The `repository.ts` change (`resolvedPlayers` → `counterDeaths`) is
covered by the new `postgres.test.ts` integration test (runs under Docker in
CI). Per `solidstats-server-ts-tests` coverage-suppression guidance for the
Docker-unavailable case, the unit-coverable changed file is confirmed 100%.
- No OpenAPI / schema / parser-contract change (Phase-1 API gate N/A).

## Config
- `eslint.config.js` — added `.planning/**` to ignores (the instrumentation
script and fixtures are GSD artifacts, not application source / outside the
tsconfig project), consistent with the existing `.agents/**` / `.claude/**`
ignores.

## Artifacts
- `instrument.ts` + `replays.json` / `identities-zero.json` / `fixture-replays.csv`
— the reproduction and real exported fixtures.
- `findings.sql` — the read-only staging diagnostics.

## Acceptance
- [x] Root cause confirmed by RUNNING (instrument logs the dropped replay's exact decision).
- [x] F12-timestamp / nickname-active hypothesis explicitly refuted with logged evidence.
- [x] Fix reads death from authoritative raw_snapshot; games unaffected.
- [x] One-life cap + `by_teamkills ≤ total` preserved.
- [x] Regression tests (unit red→green + integration) added.
- [x] lint / typecheck / format / unit tests green; `service.ts` 100%.
- [x] No parser, schema, or API-contract change.
63 changes: 63 additions & 0 deletions .planning/quick/260615-f13b-deaths-undercount/findings.sql
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
-- 260615-f13b deaths under-count — read-only staging diagnostics (KUBECONFIG=/tmp/kube-tunnel.yaml,
-- kubectl -n solid-stats-staging exec postgres-0 -- psql -U solid -d solid_stats -tAc "<sql>")

-- 1. Zero (000a2ac8-...): 653 distinct sg replays with artifact d>0, 894 total replays.
-- Recalc deaths=621, games=894 => 32 death-games dropped.
with players as (
select pr.id as prid, r.id as rid, r.replay_timestamp, p
from parser_results pr join replays r on r.id = pr.replay_id
cross join lateral jsonb_array_elements(pr.raw_snapshot->'players') as p
where pr.status='current' and r.game_type='sg' and r.replay_timestamp is not null
)
select count(distinct rid) filter (where lower(p->>'n')='zero' and coalesce((p->>'d')::int,0) > 0) as d_pos_replays,
count(distinct rid) filter (where lower(p->>'n')='zero') as all_replays
from players;

-- 2. The drop classification: of Zero's 653 d>0 replays, 621 are credited
-- (counter event OR victim kill row), 32 dropped (NEITHER). All 32 have
-- counter_evt_count = 0 in parser_events (the stale-events root cause).
with zr as (
select pr.id as prid, r.id as rid, (p->>'eid') as eid
from parser_results pr join replays r on r.id=pr.replay_id
cross join lateral jsonb_array_elements(pr.raw_snapshot->'players') p
where pr.status='current' and r.game_type='sg' and r.replay_timestamp is not null
and lower(p->>'n')='zero' and coalesce((p->>'d')::int,0)>0
), flags as (
select zr.*,
exists(select 1 from parser_events pe where pe.parser_result_id=zr.prid and pe.event_type='player_counter'
and pe.observed_player_ref=zr.eid and coalesce((pe.payload->>'deaths_total')::int,0)>0) as has_counter_death,
exists(select 1 from parser_events pe where pe.parser_result_id=zr.prid
and pe.event_type in ('kill','teamkill','unknown_kill') and (pe.payload->>'victim_entity_id')::int = zr.eid::int) as has_victim_death
from zr
)
select count(*) as total_d_pos,
count(*) filter (where has_counter_death or has_victim_death) as counted_deaths,
count(*) filter (where not has_counter_death and not has_victim_death) as dropped_deaths
from flags;

-- 3. SYSTEMIC: of 2064 current sg parser_results, 1850 with artifact d players have
-- ZERO player_counter events in parser_events; only 209 have them. The bulk
-- full-run never re-persists parser_events from raw_snapshot, so the counter
-- death path is empty for ~90% of replays — this is the platform-wide undercount.
with cur as (
select pr.id,
(select count(*) from jsonb_array_elements(pr.raw_snapshot->'players') p where p ? 'd') as art_d_players,
(select count(*) from parser_events pe where pe.parser_result_id=pr.id and pe.event_type='player_counter') as counter_evt
from parser_results pr join replays r on r.id=pr.replay_id
where pr.status='current' and r.game_type='sg' and r.replay_timestamp is not null
)
select count(*) as total_current_sg,
count(*) filter (where art_d_players>0) as have_artifact_d,
count(*) filter (where art_d_players>0 and counter_evt=0) as d_but_no_counter_events,
count(*) filter (where art_d_players>0 and counter_evt>0) as d_with_counter_events
from cur;

-- 4. Concrete dropped replay 9eb1a2ad-... (2021-05-14), current prid 3662f461-...:
-- raw_snapshot Zero = {d:1, nkd:1, eid:203} (171/193 players carry compact counters);
-- parser_events = 140 kill / 18 destroyed_vehicle / 6 teamkill / 1 diagnostic, 0 player_counter;
-- no kill row with victim_entity_id=203 => death dropped, game kept.
select pe.event_type, count(*)
from parser_events pe
join parser_results pr on pr.id = pe.parser_result_id
where pr.replay_id = '9eb1a2ad-60b8-4692-8520-47d8362c207c' and pr.status='current'
group by pe.event_type order by 2 desc;
16 changes: 16 additions & 0 deletions .planning/quick/260615-f13b-deaths-undercount/fixture-replays.csv
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
9eb1a2ad-60b8-4692-8520-47d8362c207c,2021-05-14,DROPPED
7173a75e-0763-4625-a057-a4f489ae6ad4,2021-10-15,DROPPED
90ce7e09-0300-478f-9e57-839460a022ed,2022-02-05,DROPPED
ed1ebdc2-f49c-47b2-abef-20e53136ef7a,2022-02-11,DROPPED
d0691cfb-ffb5-481d-9cdb-995f9c1e971a,2022-02-18,DROPPED
20562a48-15ca-4991-86a3-0d51926919ac,2024-07-06,COUNTER
a2c50976-135d-472b-9005-5eaffccaad7e,2025-11-28,DROPPED
a70e8d46-278a-486b-a9a0-77f13d9cdebc,2026-01-17,DROPPED
fecb06fd-bd33-4a38-a9bd-0a54358f19e2,2026-02-13,DROPPED
25a7da23-cec2-4f6b-93a3-3eaa99fb0bba,2026-02-20,DROPPED
d628177a-58c9-460c-b53d-83ab6c2df465,2026-05-02,VICTIM
752f7a0c-7b84-4370-b82e-f92c45b0d7f8,2026-05-02,DROPPED
73f5237f-b6b2-4124-9a74-411dc4ea839b,2026-05-09,VICTIM
d813d47f-cd90-4eeb-8ee8-f161b421214b,2026-05-09,VICTIM
f17e01d0-868c-4416-a4f0-23567b58ad68,2026-05-22,COUNTER
8192b572-e24f-4b69-a472-f2440d96bce1,2026-05-24,COUNTER
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
[{"player_id":"000a2ac8-443d-434c-9d30-ea8613c4c0ed","display_name":"Zero","steam_id":null,"nickname":"Zero","nickname_observed_from":"2026-05-10T00:30:57+00:00","nickname_observed_to":null}]
Loading