feat(history): author filter, extract around id, wholeWord, newest-first search

[antra-tess]

Changes to the history tools, driven by a resident's (Fable's) feedback from diary work:

no author filter … no extract-around-id … short words + a wide range → truncated at maxScan

What's added

• author / excludeAuthor on search and extract. Every incoming MCPL channel message is stored as participant user; the real author is only in metadata.author. The filter matches the author's name or id (case-insensitive, exact, never substring; @name and <@id> also accepted), or the stored participant for the agent's own turns. Results now include author. extract with an author filter scans in-process up to maxScan. It reports an exact matchedCount only when it read the whole window; otherwise it returns matchedCountAtLeast, truncated, and scannedThrough.
• extract({aroundId, before, after, allChannels}): the conversation around a message id returned by search, in the anchor's own channel by default. The anchor is marked anchor: true.
• search:
  • wholeWord (Unicode-aware: "mission" no longer matches inside "uncommissioned").
  • order: "newest".
  • When the scan stops early (at limit or maxScan): scannedThrough, plus a hint naming the from/to value to continue with.

What's fixed (found on a real store; the old stub hid it)

• A time-only native query (queryMessagesByTime / queryMessagesByTimeAndChannel without a channel) reports the page size as its matchedCount, not a total. Channel-scoped queries return messages in append order, and Discord catch-up appends old-timestamped messages late. The new edgeWindow() returns the n messages nearest a range edge in true timestamp order: the native reverse option when there's no channel, and a widening time window when there is one. Newest-first search and aroundId both use it.
• Plain extract without a channel was reporting that page-sized matchedCount as a total. It now reports hasMore, and keeps matchedCount only when a channel is given (a response-shape change).
• Both test stubs now mimic these real behaviours.

Known limits

The author filter depends on what each connector puts in author. In Eidoverse, id = name = the in-world name (no stable id), and narration is credited to the world, not to the person who acted. The same person can also appear under several names across platforms (antra_tessera / Antra). Callers can pass a list for now.

Testing

• test/history-module-ux.test.ts: 26 new tests, including catch-up backfill ordering and page-size counts. The existing history tests were updated for the new stub method and the limit+1 probe.
• npm test: 950 tests, 946 pass, 0 fail (4 not run).
• Ran against a copy of a real resident store (Linn): newest-first returns the latest messages, aroundId returns the true neighbours in the channel, and wholeWord / excludeAuthor behave as intended.

🤖 Generated with Claude Code

[slimepriestess]

Reviewed head 6ca10f0, and as a trial merge onto main@67991cf (CM 0.11.0). Second read by Opus 5.5 in a separate session; finding 2 and the sharper form of notes 3 and 4 are its catches, each re-run here on a real store.

Verdict: CHANGES REQUESTED, two findings of one class. The three claims about the native queries are true (checked in message-store.js, not the stub), and the ordinary cases on a real Chronicle store with backfilled timestamps come back right. The new contract the PR adds, "stop early, hand back scannedThrough, continue with from/to", is honest only when the scanned pool is in timestamp order, and there are two paths where it isn't.

Blocking

1. extract with author and a channelId resumes by timestamp over an append-ordered scan. The filtered scan pages the channel window with queryMessagesByTimeAndChannel({ offset: scanned }), which the store returns in ordinal order. scannedThrough is lastScanned.timestamp and the hint says continue with from:scannedThrough. Under catch-up backfill, messages past position scanned can carry older timestamps, and the continuation never reaches them.

   Real store, #diary, all by antra: a1..a5 appended at 60..20 min ago, then b1 (55 min) and b2 (25 min) appended late.

   extract({author:"antra", channelId:"diary", maxScan:5})
     → a1 a2 a3 a4 a5, truncated, scannedThrough = t(a5)
   extract({author:"antra", channelId:"diary", from: scannedThrough})
     → a5                       (b1, b2 unreachable)
   extract({author:"antra", maxScan:5})            // no channel: time-ordered
     → a1 b1 a2 a3 a4, scannedThrough = t(a4)
   extract({author:"antra", from: scannedThrough})
     → a4 b2 a5                 (complete)
   

   The no-channel path is fine because a time-only query is timestamp-ordered. The PR's test for this path (extract author: maxScan bound reports truncated + scannedThrough) has no channel and no backfill, so it can't see it; a test with both would go red on the current code (the probe above shows the failure; I didn't write the test). Fix: resume by raw window position (return scanned as something like windowOffset and accept it back), which keeps the cheap paging and doesn't depend on (2). Paging through time slices instead would inherit (2) until that's fixed. The pageFull hint has the same problem ("continue with from:scannedThrough and offset:0").

2. edgeWindow's overshoot fallback is wrong for any late-appended old message, not only "same-burst backfill". When the grown window holds more than fetchCap (max(4n, n+1000)), or the covered break takes the whole range, the code takes the window's ordinal tail. That is the timestamp tail only if nothing in the tail was appended late. A density jump across one ×4 step gets there without any burst: a channel that was busy last week and quiet since.

   Real store, channel d: dense0..dense1999 stamped 3 days ago one second apart, then q1..q5 at 55..15 min ago, then one message appended last and stamped 4 days ago (OLD-backfill). One 30-day-old message in another channel, so the first step isn't covered.

   search({channelId:"d", order:"newest", maxScan:10})
     → window = q5 q4 q3 q2 q1 dense1999 dense1998 dense1997 dense1996 OLD-backfill
       truncated:true, scannedThrough = 4.00 days ago
   extract({aroundId: q1, before:5, after:2})
     → OLD-backfill dense1996 dense1997 dense1998 dense1999 q1 q2 q3
   

   The window contains a four-day-old message and drops dense1995; the to: continuation then skips dense0..dense1995, 1996 messages never scanned. aroundId presents the four-day-old message as part of the conversation around q1. aroundId is the likeliest place to hit this: with the default before=10, fetchCap is about 1010, so all it takes is under 11 messages in one window step and over 1010 in the next. Fix: on overshoot, bisect the width between the previous step and the current one. Channel counts are exact, so the bisection lands in [n, fetchCap]. Keep the ordinal fallback only for a single millisecond holding more than fetchCap. The doc comment's "correct for everything but same-burst backfill" goes with it.

   Two routes to idx === -1 in handleExtractAround follow from this: the fallback plus backfill can push the anchor out of both sides' ordinal slices, and more than AROUND_TIE_CAP same-millisecond ties with a low-sequence anchor can too. Both throw "Message X is not in channel …", which is the wrong message. The comment "only reachable when an explicit channelId excludes the anchor" is overconfident. Reasoned from the code, not probed.

Notes

3. Author id keys never match if the id has an uppercase letter. normalizeAuthorSpec lowercases the spec; a.id is compared as stored. Discord ids are numeric, so nothing there; any platform with letter ids fails on the id half of "name or id". Fold a.id too.
4. wholeWord treats combining marks as boundaries. WORD_CHAR_RE is [\p{L}\p{N}_]; U+0301 and U+093E both test false. So cafe matches inside NFD café, and Indic-script words match mid-word because every vowel sign reads as a boundary. Add \p{M}. Cyrillic works, which is what the test covers.
5. Newest-side channel window is capped at Date.now(); the no-channel path isn't. When growth breaks on count, wTo = hi = Date.now(). Twenty messages in the last five minutes plus one stamped now+5s: channel search returns c19..c15, no-channel returns future c19..c16. A to: continuation walks backward, so it never reaches the skewed one. Small, but Eidoverse has skew (ew#201). Keep toMs open on the newest side.
6. author: "user" matches every MCPL-ingested message, since the stored participant is a key. Harmless; the description says the participant key is for "your own turns".
7. Cost: each channel count is a full time×channel intersection in the store (queryByTimeAndChannel rebuilds it per call). edgeWindow does up to ~13 of them, aroundId roughly 25 per call, and the author-filtered channel scan one per 1000-row page (50 for a full maxScan). Not correctness; antra's call whether CM should grow a cursor.
8. The no-channel extract drops matchedCount. That's a removed response field, filed under "fixed"; resident scripts parse these. I found no consumer in connectome-host or AF src, so minor, but a changed/breaking fragment would be more honest than fixed.

Verified

• Store contract, read in @animalabs/context-manager 0.11.0 message-store.js: time-only queryByTime returns matchedCount = page size and supports reverse; time+channel returns an exact matchedCount regardless of limit (so the limit: 0 counting in edgeWindow is sound) and pages in ordinal order; channel-only is exact too. Both test stubs mirror this.
• Real-store runs (JsStore + MessageStore, timestamps edited after append the way CM's own tests do), below fetchCap: order:"newest" in a channel returns the timestamp tail, not the append tail; aroundId returns timestamp neighbours across a backfill (r2 r90 r3 r91 r4) and interleaves channels with allChannels; author/excludeAuthor exact, @/<@id> forms, participant match for own turns; plain extract reports hasMore and drops the page-size matchedCount without a channel, keeps the exact one with; anchor-outside-channel and mixed-mode inputs error cleanly. A 3000-message burst plus five spaced messages and one backfill inside the burst comes back right for both search and aroundId, because the first ten-minute window step covers it without overshoot.
• edgeWindow terminates: lo/hi are finite (store-wide earliest or 0, and now), width grows ×4 from ten minutes, covered is checked before count.
• tsc clean on the trial merge; history-module + history-module-ux 75/75 on head and on the merge. No overlap with #173 (git merge-tree clean).
• package.json on the branch still says ^0.10.0 for context-manager; main is ^0.11.0. The merge takes main's, which is what the code was verified against.