Repository navigation
Test isolation: integration conftest leaks KALSHI_DEMO into unit from_env tests (4 full-suite failures) #445
Description
Summary
Four unit tests fail in the full pytest tests/ run but pass in isolation:
tests/test_client.py::TestKalshiClientFromEnv::test_from_env_demo_flag
tests/test_client.py::TestKalshiClientFromEnv::test_from_env_base_url_override
tests/test_async_client.py::TestAsyncKalshiClientFromEnv::test_from_env_demo_flag
tests/test_async_client.py::TestAsyncKalshiClientFromEnv::test_from_env_base_url_override
This is pre-existing (reproduces on main, independent of #443/PR #444) and is an environment-variable test-isolation leak.
Root cause
tests/integration/conftest.py has a session-scoped autouse fixture that does:
if "KALSHI_DEMO" not in os.environ:
_monkeypatch_session.setenv("KALSHI_DEMO", "true")_monkeypatch_session is a session-scoped MonkeyPatch, so it only undoes at session end — not between tests. Once the first test under tests/integration/ triggers fixture setup (the integration tests pytest.skip() in their bodies, but autouse fixtures still set up first), KALSHI_DEMO=true is set for the remainder of the session and leaks into later unit tests.
test_from_env_base_url_override sets KALSHI_API_BASE_URL but does not delenv("KALSHI_DEMO"), so from_env sees the leaked KALSHI_DEMO=true alongside an explicit base_url and raises the "Conflicting environment: demo=True together with explicit base_url" error.
Reproduction (deterministic)
uv run pytest tests/integration tests/test_client.py::TestKalshiClientFromEnv \
tests/test_async_client.py::TestAsyncKalshiClientFromEnv -q -p no:randomly
# -> 4 failed, 8 passed, 313 skippedExcluding tests/integration makes all four pass.
Suggested fixes (any one)
- Make the demo-env autouse fixture function-scoped (or use a fresh
MonkeyPatchper test), soKALSHI_DEMOdoesn't persist across the session. - Have the
TestKalshiClientFromEnv/TestAsyncKalshiClientFromEnvfrom_envtestsmonkeypatch.delenv("KALSHI_DEMO", raising=False)(they already delenvKALSHI_API_BASE_URL). - Add a session-end teardown that explicitly restores
KALSHI_DEMO.
Option 1 fixes the class of bug (any unit test reading KALSHI_DEMO is currently exposed), so it's preferable.
Activity
TexasCoding commentedon Jun 9, 2026
Correction + fix incoming (PR #444).
The leaked variable is
KALSHI_PRIVATE_KEY_PATH, notKALSHI_DEMOas I first wrote. The actual failure is:Mechanism:
tests/integration/conftest.py::_bridge_env_vars(session-scoped autouse) bridges a developer's ambientKALSHI_DEMO_PRIVATE_KEY_PATH→KALSHI_PRIVATE_KEY_PATHfor the whole session.test_from_env_demo_flag/test_from_env_base_url_overridesetKALSHI_PRIVATE_KEYbut — unlike their passing siblings (test_from_env_with_pem_string,test_from_env_with_key_path) — neverdelenv("KALSHI_PRIVATE_KEY_PATH"), sofrom_env()sees both and raises. (This is why it only surfaces on machines with demo creds exported, including the maintainer's; CI without demo creds wouldn't bridge a path and wouldn't trip it.)Fix: make the two non-hermetic tests match their siblings by dropping the ambient PEM path —
monkeypatch.delenv("KALSHI_PRIVATE_KEY_PATH", raising=False)in bothtest_client.pyandtest_async_client.py. The session-scoped bridge itself is left as-is (session scope is required for the session-scoped integration clients); the right boundary to harden is the unit test's own env, consistent with the rest of the class. Will be closed by PR #444.test: make from_env tests hermetic against bridged PEM path (closes #445