Skip to content

fix: select latest SQLite checkpoint when timestamps tie - #920

Open
Kuang-xianxin wants to merge 1 commit into
apache:mainfrom
Kuang-xianxin:codex/sqlite-latest-checkpoint-ties
Open

Kuang-xianxin wants to merge 1 commit into
apache:mainfrom
Kuang-xianxin:codex/sqlite-latest-checkpoint-ties

Conversation

@Kuang-xianxin

@Kuang-xianxin Kuang-xianxin commented Sep 7, 2026

Copy link
Copy Markdown

SQLitePersister.load(partition_key, None) and its async counterpart can return an older checkpoint when several saves share a timestamp. SQLite's default CURRENT_TIMESTAMP has second precision, and the current query orders only by that timestamp. In a deterministic reproduction, saving sequence 11 followed by sequence 12 and loading after reopening the database returns sequence 11. Across applications it can also return the wrong app.

Changes

  • Use descending rowid as the secondary ordering key in the latest-across-apps query, after created_at.
  • Apply the same fix to synchronous SQLite and aiosqlite persistence.
  • Add deterministic regressions for same-app and cross-app saves, default and explicit partitions, partition isolation, database reopening, and explicit historical lookup. A separate control verifies that timestamps remain the primary ordering key.

This uses the rowid already present in tables created by these persisters; no schema migration or change to app-specific sequence selection is needed. The tie-breaker follows insertion order in the persister-managed table, not a portable identifier for externally rewritten database rows.

Related: #807 explicitly notes the timestamp-tie limitation in its test-only PR. This PR fixes the behavior in both implementations; it does not duplicate those construction/lifecycle tests or depend on that PR.

How I tested this

Python 3.13.14 on Windows, using actual SQLite/aiosqlite databases:

python -m pytest tests/core/test_persistence.py tests/integrations/persisters/test_b_aiosqlite.py -q -k load_latest
Before the fix: 8 failed, 2 passed

python -m pytest tests/core/test_persistence.py tests/integrations/persisters/test_b_aiosqlite.py tests/core/test_application.py -q --timeout=20
After the fix: 199 passed

Pre-commit passes on all four changed files: Black 23.11.0, isort 5.12.0, flake8 6.1.0, whitespace/EOF, AST, and ASF headers. The frontend hook was explicitly skipped because this changes only Python persistence; the full external-database test matrix was not run.

Notes

Checklist

  • PR has an informative and human-readable title
  • Changes are limited to a single goal
  • Applicable pre-commit checks pass (frontend hook excluded as described above)
  • Any change in functionality is tested
  • Existing database format and explicit historical lookup are preserved

Use insertion order to disambiguate same-second checkpoints in sync and async SQLite persisters. Add durable reopening and partition regressions.

Assisted-by: Codex
@github-actions github-actions Bot added area/core Application, State, Graph, Actions area/storage Persisters, state storage area/integrations External integrations (LLMs, frameworks) labels Sep 7, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/core Application, State, Graph, Actions area/integrations External integrations (LLMs, frameworks) area/storage Persisters, state storage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant