Repository navigation
perf(ws): collapse snapshot apply into a single dict walk (#263) - #282
Conversation
Before: orderbook_snapshot payloads validated into `list[tuple[Decimal, Decimal]]`, then `_apply_snapshot_inplace` called `dict(...)` on each side — a second per-level walk that did the same tuple unpacks pydantic-core had just done. For a 200-level book that is 400 redundant tuple unpacks per snapshot. Fix: - New `_levels_to_dict` BeforeValidator on `OrderbookSnapshotPayload.yes`/`.no` that coerces each `[price, count]` pair into a fully-typed `dict[Decimal, Decimal]` in a single Python walk. Field type is `Annotated[dict[Decimal, Decimal], BeforeValidator(...)]` so pydantic-core does not re-walk to coerce items the validator already produced as Decimals. - `_apply_snapshot_inplace` adopts `msg.msg.yes` / `msg.msg.no` directly — no second walk, no rebuild. The msg envelope is recv-loop-owned and discarded after dispatch so aliasing is safe. Microbench (200-level book, M4): - apply-only: 8.76 us -> 1.68 us (5.2x) - validate+apply walk count: 2 -> 1 Other call sites updated: `scripts/bench_orderbook_delta.py`, `tests/ws/test_models.py` (snapshot shape assertions), and `tests/ws/test_orderbook.py` (helper docstring) all rewritten against the new dict shape. Direct constructor still accepts a pre-built `dict[Decimal, Decimal]` (used by the bench script and the new regression tests). Tests: - tests/ws/test_orderbook.py::TestSnapshotPayloadDictShape — five regression tests covering dict shape post-validate, identity-adoption by _apply_snapshot_inplace (no rebuild), direct-constructor dict acceptance, empty defaults, and snapshot+delta correctness. - tests/ws/test_models.py — existing snapshot parse / empty-book / decimal-typing tests rewritten against the dict shape. Closes #263
Code Review — PR #282: collapse snapshot apply into a single dict walkOverviewThis PR eliminates a redundant O(n) per-level Python walk when applying WebSocket orderbook snapshots. Previously, Pydantic validated wire data into Bug: misleading comment overstates the optimization
yes_levels = dict(msg.msg.yes)
no_levels = dict(msg.msg.no)
Two ways to resolve this: Option A (fix the comment): Update the docstring to say "a shallow copy (pointer-only, no Decimal re-construction)" instead of "adopts the map directly with no rebuild." Option B (complete the optimization): Remove the # _apply_snapshot_inplace lines 128-129
yes_levels = msg.msg.yes
no_levels = msg.msg.noThe recv-loop discards the snapshot message after Style: multi-paragraph docstrings violate CLAUDE.md
The Missing test coverage for
|
| Path | Covered? |
|---|---|
None → {} |
No |
dict with all-Decimal keys+values (fast-path, same object returned) |
No |
dict with non-Decimal keys+values (slow-path, re-coerced) |
No |
| iterable of 2-element pairs (wire shape, happy path) | Yes (via existing snapshot tests) |
The fast-path is subtle: _levels_to_dict returns the same dict object when all keys/values are already Decimal. If _apply_snapshot_inplace ever adopts directly (Option B above), this identity matters — a test should verify it.
Minor: tighten the return type
def _levels_to_dict(value: Any) -> Any:Can be -> dict[Decimal, Decimal] — BeforeValidator accepts any callable; the return type is not constrained to Any. Makes mypy more useful for callers.
Summary
| Real optimization? | Yes — Decimal construction moves from two passes to one |
| Comment accuracy | Off — dict() copy is still O(n), not "adopted directly" |
| Style compliance | Minor violations (multi-paragraph docstrings) |
| Test gaps | _levels_to_dict fast-path, None input, dict-with-strings input |
| Correctness | No behavioral bugs; the shallow copy is safe |
The core change is sound. The main ask before merging: fix the misleading "adopts directly / no second iteration" claim (either update the comment or update the code to match it), and add the three missing _levels_to_dict unit tests.
Closes #263
Part of #273 (post-v2.4 audit), Wave 2.