Skip to content

ai: Fix HEADS conversation-index invalidation and isolate IMAP mock caches - #9

Merged
latentoperator merged 1 commit into
mainfrom
wt/t_9361843a
Oct 3, 2026
Merged

latentoperator merged 1 commit into
mainfrom
wt/t_9361843a

Conversation

@latentoperator

Copy link
Copy Markdown
Owner

What ships

Two pre-existing test-determinism defects that failed the Linux standalone CI job intermittently at dd28188 (post-merge run 37075651253, attempts 1 and 2). Both reproduce in isolation and are unrelated to the v0.10.8 sync merge; the merge tree is byte-identical to the reviewed head, and a green retry did not repair either root cause.

HEADS conversation-index invalidation (src/agent/storage.rs, src/agent/jobs.rs): the in-process conversation index keyed on the store root's dev/ino and coarse mtime/ctime. On filesystems whose timestamps come from the coarse clock, a turn directory created inside the same tick as the previous poll leaves every one of those fields unchanged, so list() kept serving the index it built before the turn existed and agent.jobsList returned jobs: []. Store::revision() now also carries the root's link count, which moves on every subdirectory create and remove even inside one tick. The store's ownership, symlink and path safety checks are untouched.

IMAP mock isolation (src/providers/imap/** test fixtures): every IMAP test runs in one process, and both the connection pool (POOL, 45 s) and the mailbox cache (BOXES, 60 s) are keyed by account — settings + credential + oauth, which includes the ephemeral loopback port. When two independent mock lifetimes were handed the same port inside those lifetimes, the later mock reused the earlier server's mailbox list, skipped CAPABILITY+LIST, and then read SELECT where it expected CAPABILITY. Each fixture build now claims a distinct synthetic account (testSession), so no two fixtures can share a cache key; fixtures that deliberately reuse a connection build their parameters once and keep that same value. No process-global cache is cleared, no assertion is relaxed or removed, and the synthetic LOGIN bytes on the wire are unchanged.

Regression proof

Three new deterministic, sleep-free tests:

  • agent::jobs::tests::head_index_sees_turns_created_within_one_root_timestamp_tick — polls an empty store, seeds a turn, rebuilds the cached key from the live timestamps with the pre-seed link count (the same-tick window a coarse-clock mkdir leaves behind), and asserts the turn appears; then asserts removal invalidates too.
  • providers::imap::read::tests::independent_mock_lifetimes_on_one_port_never_share_cached_mailboxes — two mock lifetimes deliberately given one port; the second must serve the whole discovery handshake.
  • providers::imap::read::tests::one_synthetic_account_reuses_its_cached_mailboxes_on_a_fresh_connection — the production-intended reuse, kept as a guard so the fix cannot silently disable account-scoped caching.

Fail-before on the un-fixed tree (only the behavioural hunk reverted, so the tests still compile) / pass-after:

thread 'agent::jobs::tests::head_index_sees_turns_created_within_one_root_timestamp_tick' panicked at src/agent/jobs.rs:
Store::revision must carry the root link count, not only timestamps
thread 'providers::imap::read::tests::independent_mock_lifetimes_on_one_port_never_share_cached_mailboxes' panicked at src/providers/imap/read/tests.rs:
assertion `left == right` failed
  left: "O1 SELECT \"INBOX\"\r\n"
 right: "O1 CAPABILITY\r\n"

That second signature is byte-identical to the CI failure, and the un-fixed run then reports Err("mail_connection_closed") for the call, exactly as CI did.

Determinism after the change: 20/20 targeted runs and 5/5 full library suites green, no sleeps, no retries.

Verification performed (local, synthetic data and loopback only)

  • cargo test --locked --no-default-features --features standalone,integration-test-credentials (all targets): lib 641 passed / 0 failed / 11 ignored, every integration target green.
  • cargo check --locked --all-targets --no-default-features --features standalone.
  • make test-rust and make test-js green.
  • Backend API 7 contract: python3 tests/test_backend_api.py --binary target/debug/omamail --expected-version 0.10.8 --standalone (three runs) and against the default-feature backend, both PASS: 38 methods, 182 advertised methods; binary 0.10.8.
  • python3 scripts/package-backend.py check-api reports 7; python3 tests/test_agent_native_bridge.py runs 43 tests OK.
  • git diff --check clean.

Exact-head CI for this branch, including the Published backend merge gate, is the required gate for this change.

Residual limitations

  • A same-tick remove that is cancelled by a same-tick create restores both the link count and the coarse timestamps, so the fingerprint cannot see it. This is deliberately not described as a general invalidation guarantee.
  • The signal relies on the filesystem counting subdirectories in a directory's link count, which is what this host does (ext2/ext3; ext4, APFS and tmpfs behave the same). A filesystem that reports a constant link count would need a durable amendment; the smallest one is to invalidate the in-process index from the turn create/remove path itself and keep the fingerprint as the cross-process fallback. Not implemented here, to avoid overbuilding.
  • The IMAP collision is a race; the isolation is structural (distinct keys), not a reproduction of the exact port pair, which the CI logs do not record.
  • Windows and macOS standalone jobs use finer inode timestamps and are structurally immune to the first defect; that immunity is structural, not measured.

Security

Security: PASS for the affected boundaries (private agent state store; IMAP loopback test mocks). Everything here is synthetic (synthetic:secret, 127.0.0.1, ephemeral ports): no credential is read, written or logged by either path, and no real account, mailbox or network egress is involved. testSession is a test-only settings field that changes only the in-process cache key and never reaches the wire, so the LOGIN bytes are unchanged. No secret, real mail or personal data is in this PR.

Release Notes

  • The AI conversation list now shows a turn created immediately after a poll straight away, instead of possibly waiting for the next poll.

…aches

Two pre-existing test-determinism defects, both reproduced in isolation before
this change:

- `Store::revision()` keyed the in-process conversation index on the store
  root's dev/ino and coarse mtime/ctime only. A turn directory created inside
  the same kernel tick as the previous poll leaves those unchanged, so
  `agent.jobsList` kept serving the index it built before the turn existed.
  The fingerprint now also carries the root's link count, which moves on every
  subdirectory create and remove even when the timestamps do not.

- Every IMAP test shares one process, and both the connection pool and the
  mailbox cache are keyed by account (settings + credential + oauth, which
  includes the ephemeral loopback port). Two independent mock lifetimes handed
  the same port inside the caches' 45-60 second lifetimes reused one another's
  discovered mailbox list, so the later mock saw `SELECT` where it expected
  `CAPABILITY`. Every fixture build now claims a distinct synthetic account,
  while fixtures that deliberately reuse a connection keep the parameter value
  they built once.

Verification (synthetic credentials, loopback only). Both new regression tests
fail on the un-fixed tree - the IMAP one with the same `O1 SELECT "INBOX"` vs
`O1 CAPABILITY` signature as CI - and pass after; 20x targeted runs and 5x the
full library suite are green; `cargo test --locked --no-default-features
--features standalone,integration-test-credentials` (all targets), `cargo
check --locked --all-targets --no-default-features --features standalone`,
`make test-rust`, `make test-js`, the Backend API 7 contract against the
standalone and the default-feature backends, and
`tests/test_agent_native_bridge.py` all pass locally.

Residual: a same-tick remove cancelled by a same-tick create restores both the
link count and the coarse timestamps, so this fingerprint cannot see it, and
the signal depends on the filesystem tracking subdirectories in a directory's
link count. Internal Rust and test changes only; no API revision or release.

## Release Notes

- The AI conversation list shows a turn created right after a poll immediately
  instead of possibly waiting for the next poll.
@latentoperator
latentoperator merged commit d281a64 into main Oct 3, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant