resources: validate typed list-response envelopes for REQUIRED-array endpoints (perps + prediction) #404

Context

Surfaced by the multi-LLM review of PR #403 (perps/margin API). List endpoints across the SDK extract their array with data.get(key) or [] and then [Model.model_validate(x) for x in raw], never validating the response envelope. When the spec marks the array REQUIRED, a missing/null array is silently coerced to [], masking server-side spec drift instead of surfacing it. This is an SDK-wide convention (the prediction API does the same), so the fix should be repo-wide to avoid divergence — not perps-only.

Problem

The array is pulled straight off the raw dict with no typed-envelope validation, and or [] swallows a missing/null REQUIRED key:

Perps (non-paginated list endpoints — all return a plain builtins.list[...]):

• kalshi/perps/resources/markets.py:70-71 and :137-138 — raw = data.get("markets") or []. Spec GetMarginMarketsResponse.markets is REQUIRED (specs/perps_openapi.yaml:2062-2070).
• kalshi/perps/resources/markets.py:117-118 and :186-187 — raw = data.get("candlesticks") or []. Spec GetMarginMarketCandlesticksResponse.candlesticks is REQUIRED (specs/perps_openapi.yaml:2522-2535). Note this response also requires a top-level ticker field, which the resource discards entirely.
• kalshi/perps/resources/funding.py:80, :99-101, :129, :150-152 — data.get("funding_rates", []) / data.get("funding_history", []). Spec GetMarginHistoricalFundingRatesResponse.funding_rates and GetMarginFundingHistoryResponse.funding_history are both REQUIRED (specs/perps_openapi.yaml:2489-2498 and :2457-2466).
• kalshi/perps/resources/order_groups.py:86-87 and :239-240 — raw = data.get("order_groups", []). Spec GetOrderGroupsResponse.order_groups is OPTIONAL (no required block — specs/perps_openapi.yaml:1497-1504), so this one must stay tolerant.

Prediction API (same pattern, for parity):

• kalshi/resources/markets.py:340-341 / :593-594 (candlesticks — data.get("candlesticks", [])), :445-446 / :699-700 (bulk_candlesticks — data.get("markets", [])), :460-461 / :714-715 (bulk_orderbooks — data.get("orderbooks", [])).
• The shared paginator helper kalshi/resources/_base.py:370 and :613 does the same: raw_items = data.get(items_key) or [] (the comment there even acknowledges it: # .get(key, []) misses explicit null; or [] coerces both.).

Concrete failure: if Kalshi ships a contract change that renames markets → data (or returns null for a required array), every one of these methods silently returns []. The caller sees "no markets" rather than an error, and the contract-drift tests (which only diff request shapes + model fields, not live extraction) won't catch it. There is no envelope model to anchor a typed Model.model_validate(data) that would raise on the missing required key.

Proposed fix

Minimal, repo-wide-consistent:

1. Add typed list-response envelope models (model_config = {"extra": "allow"}, matching the response convention) for the REQUIRED-array endpoints, with the array field non-optional so Pydantic raises on a missing/null key:
   • perps: GetMarginMarketsResponse{markets: list[MarginMarket]}, GetMarginMarketCandlesticksResponse{ticker: str, candlesticks: list[MarginMarketCandlestick]}, GetMarginHistoricalFundingRatesResponse{funding_rates: list[MarginFundingRate]}, GetMarginFundingHistoryResponse{funding_history: list[MarginFundingHistoryEntry]} in the matching kalshi/perps/models/*.py.
   • prediction: an envelope for the candlesticks / bulk-candlesticks / bulk-orderbooks arrays (or reuse existing models where one already fits).
2. In each resource method, replace raw = data.get(key) or [] with envelope = Envelope.model_validate(data); raw = envelope.<field> for REQUIRED arrays. The bulk-orderbook path keeps its existing per-item _orderbook_from_item transform but iterates the validated list.
3. Leave OPTIONAL arrays tolerant: GetOrderGroupsResponse.order_groups stays data.get("order_groups") or [] (or an envelope with an Optional field defaulting to []).
4. Register the new envelope models in _contract_map.py / BODY_MODEL_MAP as appropriate so the drift harness covers them.

Scope decision worth confirming on the issue: whether to also route the Page-based _list helper (_base.py:370/:613) through envelope validation, or leave the shared paginator as-is and only harden the non-paginated list methods. The paginated path is lower-risk (a Page with items=[] is a legitimate empty page), so the conservative cut is non-paginated REQUIRED-array methods first.

Acceptance criteria

• Typed envelope models exist (extra="allow") for the four REQUIRED-array perps responses, with the array field non-optional.
• perps/resources/markets.py (list, candlesticks), perps/resources/funding.py (historical_rates, history) validate via the envelope instead of data.get(key) or []; MarginMarketCandlestick callers preserve the required ticker.
• Prediction markets.py candlesticks / bulk_candlesticks / bulk_orderbooks validate via an envelope for their REQUIRED arrays.
• order_groups.py list (sync + async) remains tolerant of a missing/null order_groups (OPTIONAL) — covered by a regression test.
• Regression tests: each REQUIRED-array method raises pydantic.ValidationError (or a wrapped KalshiError) when the array key is absent or null; the OPTIONAL order-groups method returns [] for the same input.
• New envelope models registered in the contract map; full suite + mypy strict pass.

Notes

Deferred from the PR #403 review because the headline perps fixes (f8cc960 ws, da953b0 klear, f2ef973 rest/models) shipped first; f2ef973 hardened GetMarginOrdersResponse.cursor and order-size guards but did not touch the list-envelope extraction in markets/funding/order_groups (verified against current HEAD). Tradeoff: silently masking drift is convenient (callers never crash on an empty list) but defeats the SDK's spec-conformance posture — the whole point of the extra="forbid" request side and the drift harness. Keep the OPTIONAL order_groups path tolerant; only REQUIRED arrays should hard-fail. Part of the perps EPIC #387. "Needs verification" caveat: whether to harden the shared _base.py paginator is a design call for the issue thread (this draft recommends leaving it for a follow-up).