Page.to_dataframe / Page.to_polars silently corrupt Decimal columns (str instead of Decimal) #190

Summary

Critical: Page.to_dataframe() / Page.to_polars() serialize Decimal fields as str, silently corrupting numeric operations.

DollarDecimal and FixedPointCount are declared with a PlainSerializer(_decimal_to_str) that runs unconditionally (when_used defaults to 'always'). Page.to_dataframe and Page.to_polars call item.model_dump(mode='python'), which still invokes the serializer — so every Decimal field reaches the DataFrame as a Python str. pandas/polars then build an object-dtype column of strings:

df = client.orders.list(...).to_dataframe()
df['yes_price'].iloc[0]   # → '0.5600' (str, not Decimal)
df['yes_price'].sum()     # → '0.56000.5600' (string concatenation!)
df['yes_price'].mean()    # raises
df.groupby('side').agg('sum')  # silent garbage on Decimal columns

For an SDK marketed at trading workloads where users pull positions, fills, and orders into DataFrames for P&L and risk math, this is silent money corruption. A user who computes total notional via df['cost'].sum() gets concatenated strings rather than a TypeError, and any downstream comparison against a numeric threshold short-circuits in the wrong direction.

Location

• kalshi/models/common.py:53-68 — Page.to_dataframe / Page.to_polars
• kalshi/types.py:31-33,36-40 — _decimal_to_str + DollarDecimal PlainSerializer

Evidence

# kalshi/models/common.py
def to_dataframe(self) -> pandas.DataFrame:
    import pandas as pd
    records = [item.model_dump(mode='python') for item in self.items]  # ← Decimal → str
    return pd.DataFrame(records)

# kalshi/types.py
def _decimal_to_str(value: Decimal) -> str:
    return str(value)

DollarDecimal = Annotated[
    Decimal,
    BeforeValidator(_coerce_decimal),
    PlainSerializer(_decimal_to_str),  # when_used defaults to 'always' → fires for mode='python' too
]

Reproduction:

from decimal import Decimal
from kalshi.models.orders import Order
o = Order(... yes_price=Decimal('0.5600') ...)
o.model_dump(mode='python')['yes_price']  # → '0.5600' (str)

Recommended fix

Smallest fix: change the serializer to when_used='json' so model_dump(mode='python') returns the live Decimal:

PlainSerializer(_decimal_to_str, when_used='json')

Add a regression test asserting:

• df['yes_price'].dtype is not object; or isinstance(df['yes_price'].iloc[0], Decimal) for the pandas object-Decimal column.
• df['yes_price'].sum() returns a Decimal, not concatenated strings.
• JSON serialization for request bodies still produces the wire string (model_dump(mode='json') and model_dump_json() unchanged).

Cover both to_dataframe() and to_polars() paths.

Severity & category

critical / correctness, performance