Skip to content

fix(mcpl): store a server's metadata without the keys only the host writes - #622

Merged
nissa-seru merged 4 commits into
anima-research:mainfrom
nissa-seru:petra/ingress-strip
Oct 20, 2026
Merged

nissa-seru merged 4 commits into
anima-research:mainfrom
nissa-seru:petra/ingress-strip

Conversation

@nissa-seru

@nissa-seru nissa-seru commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Problem

A channel server's metadata is spread into the metadata its message is stored with, and the host writes its own fields over it, such as channelId, messageId, author and the inbound source envelope. Some keys on a stored message are the host's alone, and a reader takes each one on its own as what the message is. The host writes some of them only when it has something to say, and others only on its own notices, so where it wrote nothing, a server's copy was stored, and a server's message could pass for one the host wrote. @slimepriestess found this in review of #598, and @Anarchid's review of #598 found it too (R1-3). It's #611. The same already holds on main for the failed-reply notice. This fixes it on main, ahead of #598.

Closes #611.

Changes

  • One strip at both ingresses. withoutHostKeys drops the keys in HOST_METADATA_KEYS from a channels/incoming message's metadata and from a push event's origin, at the one place each lane builds the metadata it stores. The host writes its own marks after it, as before. Every other key a server sends is stored as before, beside the fields the host writes.
  • A log line. The first time a server sends one of these keys, the host logs the server and the key, without the value, once for each server and key until it restarts. A server may use one of these names for something of its own, so the drop shouldn't be silent.
  • The tool boundary. A message that arrives while an agent's tools run now reaches membrane there as its participant and content alone. A stored message's metadata is the host's record, not input for membrane.

The set, and the rule

A key is in the set when only the host writes it and some reader takes it on its own as what the message is:

  • system and kind: the host's own notices and their kind (isConversationalInjection, shedNewestTurn, and the failed-reply readers releaseWindow, unreadFailedReplies, flushDeferredWrites and failedReplyNoticeForPrimary).
  • draftOwner and draftIds: a notice naming a resident's drafts (draftsNamedInHistory, and the catch-up notice's read of the queue).
  • tuneOut: a message a tune-out holds from the residents (the residents' viewFilter, and the readers that treat a row as held).
  • coalescingSubject: the delivery of a coalesced occurrence, beside its eventId, which the host writes too (wasPublished).
  • firstView: the coalescer's first-view mark. It was already dropped, and its two drops are now this one.
  • deferredWriteId: the landed copy of a queued write (boot's dedup, landedCopiesById). addMessage drops it from the writes it stores, but a channel message for a conversation fork goes straight to the fork's store.
  • external: a module's own row for an outside message (the Discord module's storedRows and storedByDiscordId, matchesMessageQuery, findMessageByExternalId).
  • openaiResponsesItems: membrane's mark of an imported Responses history, which only a host's import writes.

What stays:

  • Keys a server writes by protocol, such as tags, mentioned, guildName and a push's raw channelId, since they're the server's claims to make.
  • Keys a reader takes only beside one of the set. A failed reply's reason and detail are read only on a message whose kind is discord-send-failed, and fix(channels): a channel notice reaches whoever can close the channel, and a released agent's unread ones reach the primary #598's passOn only on a channel-notice, so with kind dropped nothing reads a server's copy of them. The Discord module reads discordTextFirst only on a row it found by external.
  • Keys nothing reads where a server could set them. Nothing reads speaker. promptedBy and silentHeartbeat are read only on the agent's own rows, and an incoming message is stored as user.

So speaker and passOn, which #611 suggests, stay out by the same rule.

Both lanes build each content block afresh from the fields convertBlock knows (push-handler.ts, channel-registry.ts), so a server can't set a block's other fields either.

Messages stored or queued before this change keep what they were stored or queued with.

What servers were told

Nothing in 0.23.1 says the host keeps a server's metadata as sent:

  • The framework's event types, McplChannelIncomingEvent.metadata and McplPushEvent.origin (src/types/events.ts), carry no doc comment.
  • The wire types describe ChannelIncomingMessage.metadata as "Platform-specific metadata" and PushEventParams.origin as "Provenance metadata (server-defined)" (src/mcpl/types.ts).
  • The README mentions message metadata only for the recovery outbox, stores from before metadata.serverId, and the deprecated router's mentioned rule.

mcpl's SPEC §16.6 says tags are "untrusted claims authored by the producer, exactly like origin and metadata. A host MAY disbelieve them."

We read where discord-mcpl, slack-mcpl, telegram-mcpL and heartbeat-mcpl build their messages' metadata and push origins, at each one's main, and none of them carries a key in the set. None of those, and none of portal, spend-mcpl, terminal-sessions-mcp, anima-docs or eidoverse-worlds, sends external.

A context/beforeInference injection's metadata stays out of this. context-manager builds an injected message from its namespace and content alone, and no injection is stored, so nothing reads it.

What each part is for

  • HOST_METADATA_KEYS, one constant with the rule for joining it. Both lanes read one list, so a key the host starts to trust on its own is added in one place. Two inline lists, one per lane, would have to be kept in step.
  • withoutHostKeys. One copy-and-drop serves both lanes. It's a method rather than a module function only because the log line keeps per-framework state.
  • The log line and its Set. A server outside the house may use kind or system for something of its own, and would otherwise lose it without a word. The line names the server and the key, once each, so the log shows the drop and keeps the server's data out.
  • The tool boundary's hand-off. It's the one place stored rows reach membrane without a compile. Handing over participant and content only keeps the host's record the host's, whatever membrane reads later, and whoever wrote the row. The host's own readers of an injection's metadata run on the array before the hand-off.
  • Folding in firstView removes two else delete metadata.firstView lines.
  • The tests.
    • The table pins each key in both lanes.
    • The release test pins the reported case in a fork.
    • The log test pins the line for two servers and both lanes.
    • The boundary test pins the hand-off.
    • The table and log tests use inbound-source.test.ts's harness over the same fixture server.
    • The five coalescing assertions read the first-view mark from the stored row, storedFirstView, since membrane is no longer handed it, and the stored row is where the host reads it.
  • Not done: keeping the host's marks under a field of their own, apart from the server's metadata. That would change every reader and writer of a notice and every stored message, for the same guarantee this gives at the two places a server's metadata comes in.

Tests

  • test/ingress-host-keys.test.ts, over a real MCPL child:
    • a channel message whose metadata carries every key in the set is stored without them, and its own keys stay;
    • a push whose origin carries every key is stored the same way;
    • the log line comes once for each server and key, for two servers and in both lanes, and names no value.
  • test/route-failure-targeting.test.ts: a DM whose metadata carries every key and claims to be a failed-reply notice reaches the fork's window without them, and tells the primary nothing when the fork is released.
  • test/present-while-acting.test.ts: a message injected at a tool boundary reaches membrane as its participant and content alone, whatever its row carries.
  • test/mcpl-coalescing.test.ts: five assertions that read the coalescer's firstView from what membrane was handed now read it from the stored row.

Runs at this head, with main 06445063 merged in, on Node v24.22.0:

  • the full suite passes: 5,803 tests, 5,759 passing, 44 skipped, none failing;
  • with any one key left out of HOST_METADATA_KEYS, the release test fails, and so do the table tests for every key but deferredWriteId, which addMessage already drops from the trunk's writes;
  • with the log line silenced, or its once-only set keyed by the key alone, the log test fails;
  • with the tool boundary handing membrane each message whole, the boundary test fails.

At the branch's head before that merge, with main's src/framework.ts in place, the five new tests fail and the four files' other 298 tests pass.

Not verified

  • Live servers. The tests use the fixture MCPL server and in-process events. That discord-mcpl, slack-mcpl, telegram-mcpL and heartbeat-mcpl send none of the set's keys rests on reading their source.

Review

Reviewed at the Rookery: two runs of our review workflow, on the change and on its second round, with every finding judged; reads of the code, and of the merge with main, by two colleagues who didn't build it; and runs of the new tests against main's src/framework.ts.


  • Changelog fragment added: changelog.d/ingress-host-keys.fixed.md.

🤖 Written by Petra-1012 (Claude Opus 5.5) through the Rookery's tooling, contributing through nissa-seru. Ada-1017 (Claude Opus 5.5) made the design calls.

Description checked and posted by Fiona-2757 (Claude Opus 5.5).

Petra-1012 and others added 4 commits October 20, 2026 00:35
…rites

A channels/incoming message's `metadata` and a push event's `origin` were
spread into the metadata a message is stored with, and only `firstView`
was dropped (the inbound source envelope is written after). Several other
keys there are the host's alone, each one a reader takes on its own as
what the message is, so a server could send them:

- `system`: isConversationalInjection treats the message as not
  conversational, and shedNewestTurn (the refusal rewind) never sheds it;
- `kind`: as `discord-send-failed`, an unread message in a fork's window
  is passed to the primary at the fork's release (releaseWindow,
  unreadFailedReplies, flushDeferredWrites, failedReplyNoticeForPrimary),
  as a system notice carrying its `reason` and `detail`;
- `draftOwner` and `draftIds`: draftsNamedInHistory and the catch-up's
  queued read record the drafts named as noticed, with no `kind` check;
- `tuneOut`: residents' viewFilter hides the message for good, and the
  tool boundary, the coalescer's readers, noteArrival and asModuleRow
  treat it as held;
- `coalescingSubject`: wasPublished counts an occurrence delivered at
  restore when it finds the subject beside the occurrence's `eventId`.

HOST_METADATA_KEYS names them, with `firstView`, and withoutHostKeys drops
them from the server's metadata at the one place each lane builds it,
before the host writes its own marks; both `else delete
metadata.firstView` lines go. The first drop of each key from each server
is logged once per framework, naming the server and the key, never the
value, since a server may use one of these names for its own purposes.

Left out, as no reader trusts them alone: a notice's inputs (anima-research#598's
`passOn`, `reason`, `detail`), read only beside its `kind`; `speaker`
(no reader); `deferredWriteId` (addMessage drops it from every write but
the queue's); `promptedBy`/`silentHeartbeat` (read only on the agent's
own rows); `external` (context-manager documents a source's legacy
`external.channelId` on channel ingress). Context injections' metadata
stays out too: context-manager builds injected messages from namespace
and content alone, and nothing stores an injection.

Tests:
- test/ingress-host-keys.test.ts, over the real speech-route fixture
  server: a channel message's metadata and a push's origin carrying every
  key are stored without them, keeping the server's own keys; and the log
  line appears once per server and key, without the value.
- test/route-failure-targeting.test.ts: an unread DM whose metadata
  claims a failed-reply notice tells the primary nothing at the fork's
  release (on main it tells the primary once, in the server's words).

Found by slimepriestess's and Anarchid's reviews of anima-research#598 (its R1-3),
which this lands ahead of. Closes anima-research#611.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A second round of the ingress strip, after its review:

- HOST_METADATA_KEYS takes `deferredWriteId`, `external` and
  `openaiResponsesItems`, by the rule its comment now states in full: a
  key joins when only the host writes it and some reader takes it alone.
  - `deferredWriteId`: the fork writes go straight to the fork's store, so
    addMessage's drop never applied there, and boot's dedup and
    landedCopiesById match the key alone.
  - `external`: the Discord module's storedRows and storedByDiscordId,
    held-writes' matchesMessageQuery and findMessageByExternalId take
    external.source/id alone as a module's own row. `discordTextFirst`
    stays out: every reader of it runs on a row the module found by
    `external`. No house channel server sends `external` (scanned
    discord-mcpl, slack-mcpl, telegram-mcpL, heartbeat-mcpl, portal,
    spend-mcpl, terminal-sessions-mcp, anima-docs, eidoverse-worlds and
    mcpl-cc-bridge at their main heads).
  - `openaiResponsesItems`: membrane's mark of an imported Responses
    history; only a host's import writes it.
- The tool boundary hands membrane each injected message as its
  participant and content alone: a stored row's metadata is the host's
  record. The host's own readers of an injection's metadata run on the
  array before the hand-off.
- withoutHostKeys loses its `field` argument: a channel-scoped coalesced
  push reaches the channel lane with its origin as `metadata`, so the line
  named the wrong field. It now says "from a message".
- The constant's comment names main's notice inputs instead of anima-research#598's
  passOn, and says why `tags`, `mentioned` and `discordTextFirst` stay.

Tests:
- ingress-host-keys: the table takes the three keys; the log test adds a
  second server's drop of the same key.
- route-failure-targeting: the release test's forged metadata carries
  every key, and checks the row reached the fork's window unread without
  any of them.
- present-while-acting: an injected message reaches membrane as its
  participant and content only, host-written keys on its row included.
- mcpl-coalescing: five assertions that read "firstView":true from what
  membrane was handed now read the stored row, where the host reads it.

The changelog fragment names the new keys and the rule, qualifies the log
line (once per server and key until a restart) and rows stored before,
and adds the tool boundary's bullet.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…re read

A third round of the ingress strip, after the second round's review:

- ingress-host-keys' log test sends its messages one at a time, so their
  order is fixed, and the push now carries a key its server hasn't lost
  yet (`tuneOut`), so the push lane's first drop has a line of its own.
  Its comment no longer calls the push's `kind` the server's second drop
  of that key.
- mcpl-coalescing's five storedFirstView assertions say they read the
  stored row's mark.
- The hear-while-acting comment no longer says the stored row and the live
  injection share metadata: neither the live request nor compile's output
  carries a row's metadata.
- The changelog fragment says writes queued before the change keep what
  they were queued with, as stored rows do.

No source behaviour changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
main moved to 0644506 (anima-research#616). In test/route-failure-targeting.test.ts
both sides added tests after the same test, and both are kept, this
branch's first. src/framework.ts merged without a conflict.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru
nissa-seru merged commit c602d86 into anima-research:main Oct 20, 2026
6 checks passed
nissa-seru pushed a commit that referenced this pull request Oct 20, 2026
…rites

The 0.23 line's pick of #622. A channel server's metadata, a
channels/incoming message's `metadata` or a push event's `origin`, was
spread into the metadata its message is stored with, and the host wrote
its own fields over it. Where the host wrote nothing, a server's copy of a
key the host trusts on its own was stored, so a server's message could
pass for one the host wrote: with `kind: "discord-send-failed"`, a message
left unread in a conversation fork's window made the host tell the
primary, when the fork was released, that the fork's reply had failed
(#611).

- HOST_METADATA_KEYS lists the keys only the host writes that some reader
  in this line takes alone: system (isConversationalInjection,
  shedNewestTurn), kind (the failed-reply readers), draftOwner and
  draftIds (the drafts' catch-up readers), tuneOut (the residents'
  viewFilter and the held-row readers), coalescingSubject (the restore's
  published check, beside eventId), deferredWriteId (boot's dedup; a
  fork's channel write goes straight to the fork's store), external (the
  held-write match, the module registry's and the Discord module's
  lookups of their own rows) and openaiResponsesItems (membrane's mark of
  an imported Responses history). It's #622's set without `firstView`,
  which this line doesn't have.
- withoutHostKeys drops them from both ingresses, and logs the first drop
  of each key from each server, naming both and never the value.
- The tool boundary hands membrane each injected message as its
  participant and content alone: a stored row's metadata is the host's
  record, not input for membrane.

Tests: ingress-host-keys (both lanes over the fixture MCPL child, and the
log line for two servers in both lanes); route-failure-targeting (a DM
whose metadata carries every key and claims to be a failed-reply notice
reaches the fork's window without them, and tells the primary nothing
when the fork is released); present-while-acting (an injected message
reaches membrane as participant and content only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit that referenced this pull request Oct 20, 2026
…rites

The 0.22 line's pick of #622. A channel server's metadata, a
channels/incoming message's `metadata` or a push event's `origin`, was
spread into the metadata its message is stored with, and the host wrote
its own fields over it. Where the host wrote nothing, a server's copy of a
key the host trusts on its own was stored, so a server's message could
pass for one the host wrote.

- HOST_METADATA_KEYS lists the keys only the host writes that some reader
  in this line takes alone: system (isConversationalInjection,
  shedNewestTurn), draftOwner and draftIds (the drafts' catch-up readers),
  tuneOut (the residents' viewFilter and the held-row readers),
  coalescingSubject (the restore's published check, beside eventId),
  deferredWriteId (boot's dedup; a fork's channel write goes straight to
  the fork's store), external (the module registry's and the Discord
  module's lookups of their own rows), silentHeartbeat (indexSilentTickRows
  marks a stamped row's tick started before it checks whose row it is) and
  openaiResponsesItems (membrane's mark of an imported Responses history).
  - `kind` stays out: nothing in this line reads a stored message's kind.
  - `firstView` stays out: this line has no first-view mark.
- withoutHostKeys drops them from both ingresses, and logs the first drop
  of each key from each server, naming both and never the value.
- The tool boundary hands membrane each injected message as its
  participant and content alone: a stored row's metadata is the host's
  record, not input for membrane.

Tests: ingress-host-keys (both lanes over the fixture MCPL child, and the
log line for two servers in both lanes); present-while-acting (an injected
message reaches membrane as participant and content only).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@nissa-seru nissa-seru mentioned this pull request Oct 20, 2026
nissa-seru pushed a commit that referenced this pull request Oct 20, 2026
Brings 0.23.2's release commit back to main: the ## 0.23.2 section of CHANGELOG.md and package.json's version. Source and tests merge to main's own text, since the release's change is a port of #622, which main already holds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit that referenced this pull request Oct 20, 2026
0.23.2's section carries it. The merge leaves main's copy in place, since v0.23.1 never had it, and without this main's next release would list the fix a second time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 21, 2026
…ove)

main moved by 14 pull requests since this branch's last merge of it
(ea4297a, in 180b703). git stops at four places in src/framework.ts,
each resolved by keeping both sides:

- After EphemeralRun, anima-research#608 adds scriptResultBrief and this branch adds
  the re-initialization alerts' builders and ManagerReinitFailure. Both
  are kept, the builders first.
- Where a channel message or a push is written into a conversation
  fork, anima-research#252 stamps the stored copy's digest (withStoredDigest) and this
  branch writes through writeToFork, which defers the write while a move
  holds the store. writeToFork now gets the stamped metadata, so a write
  it defers lands with the digest of the blocks it stores.
- readdDrained: anima-research#616 passes a failed reply queued for a target that's no
  longer registered to the primary, and this branch adds the traceAs
  option. Both are kept: the trace covers what the drain stored, the
  passed replies included.

main's other hunks within 40 lines of this branch's were read in the
merged file: anima-research#616's rework of stop()'s comment and its pass for
postponed releases, beside this branch's status lines for writes kept
queued, and anima-research#622's withoutHostKeys, beside writeToFork's call.

anima-research#616 changes what stop() does to a fork a quiesced host disposed: the
pass for its postponed release now drains what was queued for it, so a
failed reply goes to the primary, deferred again, and the rest is
dropped, logged, at the stop() rather than at the restarted host's
resume. At this merge the test "a fork disposed on a quiesced host: a
stop before the resume leaves its held writes queued, ..." fails for
that (0 entries queued, not 3), and the disposal alert's sentence and
the description's Not covered point still say what main did before
anima-research#616. The commit after this one adapts them.

Checked at this merge's tree before committing: tsc and the build pass;
npm test 6,010 pass, 1 fail (that test), 44 skipped of 6,055; the three
files CI runs alone 130 of 130.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Channel ingress keeps server-supplied values for framework-owned metadata keys (kind, system, …), so a server can forge framework notices

1 participant