Skip to content

WS: validation failure on sequenced frame advances seq watermark — silent orderbook desync #241

Closed

Description

@TexasCoding

Summary

Money risk. A single malformed orderbook_delta or order_group frame (server bug, schema drift, network truncation) silently corrupts local orderbook state with no resync trigger. The seq-gap-driven safety net is bypassed.

_process_frame advances the seq tracker (tracked = True) BEFORE pre-validating the orderbook payload. When OrderbookSnapshotMessage.model_validate / OrderbookDeltaMessage.model_validate raises ValidationError, the recv loop catches it as "malformed WS frame" + continue — but the seq watermark already advanced. The next delta with seq=N+1 matches expected=N+1 and gap detection never fires. The existing rollback only catches KalshiBackpressureError from dispatch, not ValidationError from pre-validation.

order_group_updates has the analogous path through dispatch.py:222-232 (silent swallow, seq already advanced).

Location

  • kalshi/ws/client.py:297-336 — _process_frame track-then-validate ordering
  • kalshi/ws/client.py:257-263 — recv_loop swallows ValidationError
  • kalshi/ws/dispatch.py:222-232 — order_group_updates mirror
  • kalshi/ws/sequence.py:75-89 — track() advances watermark on first call

Evidence

ok = await self._seq_tracker.track(sid, seq, msg_type if msg_type else channel)
if not ok: return
tracked = True
...
if msg_type == "orderbook_snapshot" and self._orderbook_mgr:
    snapshot = OrderbookSnapshotMessage.model_validate(data)  # may raise ValidationError
    self._orderbook_mgr._apply_snapshot_inplace(snapshot, sid=snapshot.sid)
elif msg_type == "orderbook_delta" and self._orderbook_mgr:
    delta = OrderbookDeltaMessage.model_validate(data)  # may raise ValidationError
    ...
try:
    await self._dispatcher.dispatch(...)
except KalshiBackpressureError as exc:
    if tracked and sid is not None and self._seq_tracker:
        self._seq_tracker.rollback(sid, prev_seq)

Recommended fix

Wrap the pre-validate + apply + dispatch block in a try/except that rolls back the seq watermark on any exception (ValidationError, KeyError, programming bugs), not just KalshiBackpressureError. Same shape as the existing rollback branch, broader except clause. Add a regression test that injects a malformed sequenced delta and asserts the next frame triggers gap recovery.

Severity & category

high / correctness, ws

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingwsWebSocket-related

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions