Skip to content

Pagination cycle detector only catches adjacent cursor repeats #352

Closed

Description

@TexasCoding

Problem

The pagination cycle guard in _list_all only stores the most recent cursor. It catches the realistic last-page-replayed shape (A → A → A → …) but misses any cycle of length ≥ 2. A pagination cache invalidation that rotates A → B → A → B → …, or a load-balancer randomly returning one of two stale cursors from two backend pods, would loop forever — or until max_pages / total_timeout saves the caller, neither of which is set by default on most list_all callers.

The docstring even claims this guard handles "server-side pagination bugs." Short cycles ARE server-side pagination bugs and aren't caught.

Evidence

  • kalshi/resources/_base.py:343-371 (sync path):
last_cursor: str | None = None
pages_fetched = 0
while max_pages is None or pages_fetched < max_pages:
    page = self._list(...)
    yield from page.items
    pages_fetched += 1
    if not page.cursor:
        break
    if page.cursor == last_cursor:
        raise KalshiError(
            f"Cursor loop detected on {path}: server returned "
            f"cursor {page.cursor!r} which was already used. "
            f"This indicates a server-side pagination bug."
        )
    last_cursor = page.cursor
    current_params["cursor"] = page.cursor
  • kalshi/resources/_base.py:587 — identical pattern in the async _list_all (AsyncResource).

Suggested fix

Replace last_cursor: str | None with seen: set[str] = set() on both sync and async paths. On each new cursor, if page.cursor in seen: raise KalshiError(...). Memory is O(pages) — typically hundreds, trivial for any production workload — and the cost of detecting real short-cycle bugs is well worth it. Update the docstring (currently advertises "O(1) regardless of page count") to match. Add a regression test in tests/test_base_helpers.py using a respx mock that cycles A → B → A.

Source

Round-3 independent audit (reviewer: http_transport).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    polishCode-quality and DX improvements; non-functional

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions