Skip to content

Preserve failed bulk reads with bounded per-chat retries - #167

Merged
nixfred merged 1 commit into
nixfred:mainfrom
curtisc:fix/durable-mark-all-read
Nov 16, 2026
Merged

nixfred merged 1 commit into
nixfred:mainfrom
curtisc:fix/durable-mark-all-read

Conversation

@curtisc

@curtisc curtisc commented Nov 13, 2026

Copy link
Copy Markdown
Contributor

What

Keep Mark All as Read effective locally when the Mac cannot complete the read action. A failed or interrupted bulk attempt now retains its original click timestamp and becomes per-conversation --through retries after a complete read-state snapshot is available. Newer incoming activity stays unread, and later explicit read/unread gestures take precedence.

Persist an attempt before invoking the bridge so a collector interrupted during --all never repeats that unbounded action. Refuse a known locked screen before acquiring the Messages UI lock or starting AppleScript.

Closes #166.

Why

Under push_read=thread, the next confirmed Mac snapshot supersedes historical local read marks. The old failure path deleted the bulk intent, so an unsuccessful Mark All as Read temporarily cleared the badge and then restored the same old unread conversations. This was observed as a recurring count of 15 on a locked gateway.

#133 already made individual read intents durable; it deliberately excluded --all because a later unbounded retry could consume new arrivals. This extends durability using the existing timestamp guard instead of repeating --all. It also handles service identifiers that cannot be opened individually: a scoped local marker survives without blocking other actions.

How it was verified

  • bun test: 885 passed, 0 failed, including the new bulk-read regressions.
  • All 33 Python/QML scripts invoked by the current CI workflow passed locally; bridge compilation, Bash syntax, and ShellCheck passed.
  • Mac read suites on the gateway's Python 3.9: 120 tests, 9 skipped, no failures.
  • Before-fix proof: four new integration cases fail on unmodified 69ca3ab (lost bulk intent, unavailable snapshot, missing pre-action persistence, and interrupted bulk attempt repeating --all). They pass with this change.
  • Coverage also checks retry recovery, newer inbound activity, later explicit unread actions, merged aliases, scoped service markers, malformed marker metadata, and the early locked-screen guard.
  • Live metadata-only verification on the affected installation: the badge stayed at zero across refreshes beyond the UI's temporary suppression timeout while the gateway remained locked. No messages were sent and no conversation content was inspected or included in this PR.
  • No QML layout change; screenshots are not applicable.
  • Updated the Read sync is an acknowledged, durable action queue invariant in CLAUDE.md, and documented recovery behavior in README.md.

Limits

The Mac must still be unlocked for Messages UI actions to succeed. This does not change that platform limitation or make the UI action asynchronous (the broader blocking-call work remains in #125). A failed bulk intent waits for a complete snapshot; it never retries an unbounded --all. Service/business identifiers that cannot be opened individually retain their local marker and need a later successful mark-all in Messages to synchronize. No message bodies are persisted; the additional state contains only bounded conversation identifiers.

Retain the mark-all click boundary across failure and interruption, then recover through timestamp-bounded conversation reads. Keep service identifiers scoped locally and preserve newer arrivals and explicit gestures.

Persist each attempt before invoking Messages and reject a locked Mac before acquiring its UI lock. Cover durable read-sync invariants with regression tests. Fixes nixfred#166.
@curtisc
curtisc requested a review from nixfred as a code owner November 13, 2026 19:05
@nixfred

nixfred commented Nov 13, 2026

Copy link
Copy Markdown
Owner

Hi Curtis, thank you, and welcome. This is a strong first PR: a real bug found on a real locked gateway, filed as its own issue (#166), four regression cases that fail on unmodified main before the fix, and a fix that reuses the existing timestamp guard instead of ever repeating the unbounded --all. An independent reviewer went through it read-only (nothing sent anywhere, no Mac touched), I reproduced the one blocker myself, and it is a one-line gate.

Checked and fine: merges cleanly on main 69ca3ab; the Mac read tests pass on the gateway's own Python 3.9.6 (test_read.py, test_read_sync.py, test_read_verification.py); bun test 885 pass; of 24 deliberate breakages of your guards only 2 survived (below); gitleaks and pyflakes clean. Under push_read=thread the design holds: the attempt is saved before the bridge runs, --all never repeats after a failure or an interruption, retries carry --through, and a message newer than the click stays unread. It also fits next to the unreleased #164 work (the diff applies cleanly on top of it; your lock check stops #164's reopen from running behind a lock screen).

Blocker: under the default push_read=all, a failed mark-all now turns into one imsg-read --chat <id> --through <ts> per unread conversation, and each one brings Messages forward on the Mac. expandFailedMarkAll is called whenever there is a snapshot (collector.ts:2426), with no check on the policy, and the action loop then runs the expanded intents under any policy except off. The README and CLAUDE.md promise that all never does this ("a per-thread read pushes NOTHING by design"; --chat must open the conversation, which is why all is the default). I ran the reviewer's trace (fixture with push_read=all, bridge fails, --mark-read, bridge recovers, let the retry backoff expire):

  • PR head: the poll runs --all, then --chat +15551234567 --through 2026-09-01T10:00:00Z.
  • Main, same steps: only --all.

Under all the expansion is not needed anyway: the global read mark already keeps the badge from coming back, and #166 is specific to thread. Fix: only expand when push_read=thread, and add a test for all that asserts the actions are exactly [["--all"]].

Should-fix (the first three the reviewer ran, the rest read; I did not re-run these)

  1. A leftover marker for business threads keeps the old error on screen. expandFailedMarkAll re-creates * with chats: unaddressable and carries the failed --all error forward, so once the Mac is healthy again the UI still shows "imsg-read: Accessibility is not granted" until that business thread is read elsewhere, and nothing retries it. Strip error from the scoped marker (or keep it out of the warning), and consider an expiry.
  2. Surviving mutant at read-sync.ts:68: changing if (next["*"]?.attempts === 0) continue; back to if (next["*"]) continue; leaves every test green, and with a long-lived scoped marker no per-chat intent could be acknowledged or cancelled by a snapshot. Add a test: a scoped * marker, a per-chat read intent, and a snapshot where that chat has latest > seen; the intent must be deleted.
  3. A message stamped ahead of the Linux clock comes back unread after a failed mark-all. The bound is nowTs() at the click, so a row with latest ten minutes ahead is skipped by row.latest > all.seen. Not a regression (every row came back before), but the local mark-all path already uses ts > now ? ts : now per chat (collector.ts:2503-2505). Recording {chat: latest} from the snapshot fetched on the click run would fix this and the next point.
  4. The bound is the click's wall-clock time, not what the user had seen. A message that reaches chat.db after the click with an earlier date (delayed delivery, iCloud catch-up) gets marked read without being seen. --all did the same, so no regression, but weaker than "only what the click covered".
  5. N failed marks become N separate focus-stealing --chat runs even under thread (15 to 300 over several minutes, all playing out once the Mac is unlocked). Matches the per-thread policy but is new amplification: say so in the README paragraph, or try one more --all when the snapshot shows no chat has latest > seen.
  6. screen_locked() is now a hard gate but reads every console session (imsg-read:293-308), not this user's. With fast user switching another user's locked session might block every push. I have not verified that on a real Mac; failing open when the console user is not the gateway user would be safest.

Thank you again. Push when ready and I will re-run the all trace first.


Generated by Claude Code

nixfred added a commit that referenced this pull request Nov 14, 2026
…phic, X-POSTS row)

WHAT: docs/x-post-2026-11-14.txt (5,465 characters, 14 handles each tagged once,
  no em dashes), the 1600x900 graphic docs/img/x-post-2026-11-14.png with its
  HTML source, and the X-POSTS.md row.
WHY: Fred asked for a detailed thank-you post covering every contributor since
  the last posted update (Post 4, 26 Sep). Posts 5, 6, 7 and W45 were never
  posted, so this one replaces them: 40 merged PRs, 38 from 18 outside people,
  2 ours, one cherry-pick (David Benedic), one reporter (edmooredop), and two
  PRs in review (Jason Alexander #165, Curtis Chambers #167).
HANDLES: from CONTRIBUTORS-X.md (all 13 match); @jasona verified on his GitHub
  profile and x user; no X for Marcus (credit by GitHub handle, haplab.com),
  Erik, Gerald, Colton, Dima, David, edmooredop, Curtis: named without @.
PLACED on vic's clipboard (image first so it sits in history, text last) and the
  PNG at ~/Pictures/blip-thank-you-2026-11-14.png on vic.

LR-T: blip, x-post, contributors, thank-you, graphic
LR-D: blip, docs, comms
LR-K: blip-post-8-all-contributors-since-26-sep

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BXN1v67b9f9VHGe3FhaQuD
@nixfred
nixfred merged commit 0e41f55 into nixfred:main Nov 16, 2026
1 check passed
nixfred added a commit that referenced this pull request Nov 16, 2026
WHAT: CHANGE.log entries for #167 (Curtis Chambers, no X known) and #165 (Jason
Alexander, @jasona verified via x user), roster rows for both, weekly note for
Mon 16 Nov.
WHY: standing rule, every merged PR gets its change line and roster row the day it
merges, with the verified X handle or the honest 'none known'.

LR-T: blip, docs, change-log, roster, pr-165, pr-167
LR-D: blip
LR-K: change-log-16-nov

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BXN1v68N3DWV2UhtuJggZz
@nixfred

nixfred commented Nov 16, 2026

Copy link
Copy Markdown
Owner

Merged in 0e41f55. Thank you, Curtis: a real bug from a real locked gateway, filed as #166, with regression cases that fail on main before the fix.

One thing I changed in the merge, so you are not surprised when you pull: the per-chat expansion now runs only under push_read=thread. Under the default all it would have opened Messages on the Mac once per unread conversation, which the README says all never does. There is a test for it, and it fails with the gate removed. Checked: bun test 903 pass, the Mac read suites pass, CI green.

I left the review's six should-fix items for follow-up, none blocking: the leftover marker for business threads keeps the old error on screen, a message stamped ahead of the Linux clock can come back unread after a failed mark-all, and screen_locked() reads every console session. If you want a second PR, they are yours.

One small question: do you have an X account you would like credited? Your GitHub profile lists none, so for now I credit you as github.com/curtisc.


Generated by Claude Code

nixfred added a commit that referenced this pull request Nov 17, 2026
… 14 Nov correction

WHAT: WEEKLY-NOTES.md gets its own 2026-W47 section (Mon 16 to Sun 22 Nov). The
"Mon 16 Nov: #167 and #165 merged" block moves into it from W46, where it was
mis-filed (16 Nov 2026 is a Monday, ISO week 47). W47 also records the audit of
all 128 merged PRs: 21 of them (#37 #38 #40 #41 #42 #46 #50 #51 #52 #53 #55 #56
#57 #58 #59 #60 #63 #64 #65 #66 #67, merged 6 to 10 Sep) had no comment and no
review from nixfred, so each now carries a specific thank-you on the PR itself
(Erik Fillipsveen @Fileri 13, Jon Kinney @jondkinney 7, Danny Cecil @jefehoser 1).
W46 gains Jason Alexander's 14 Nov correction to his own #165 evidence, quoted
verbatim, which is why #165 stays "not yet proven on a real RCS contact".
WHY: standing rule (Fred, 2026-10-31): audit every merged PR at the start of a run
and thank any author who got no visible comment; notes are written the day the
work lands.
VERIFIED: bun test on main, 927 pass, 0 fail, before the comments went out; all 21
comments posted without a failure.
OPEN: 2.6.2 (26 Sep) is still the latest release and manifest.json says 2.6.2 while
CHANGELOG.md Unreleased holds every PR since; a release is Fred's call. Erik, the
largest contributor (22 PRs), is not named in Post 4 or the 2.6.2 release notes.

LR-T: blip, weekly-notes, contributors, thank-you, audit, w47
LR-D: blip, docs, github, community
LR-K: blip-catchup-thanks-21-prs-2026-11-17, blip-w47-section, jasona-165-correction

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BXN1v3tUKC1p1NZKBgXXvk
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Failed Mark All as Read discards read intent under thread sync

2 participants