Repository navigation
fix(dendrite): reader forks are handed what is queued behind the resident's turn - #793
Merged
Merged
Conversation
… their text
addPolicy and removePolicy (wake_add_rule and wake_remove_rule, and the
framework's addGatePolicy and removeGatePolicy) read gate.json again
before a change, and when it was there but wouldn't load, they fell back
to the rules in force and wrote those over it. Whatever the file held,
such as an operator's unfinished edit, was lost, though addPolicy's doc
says a concurrent operator edit isn't clobbered.
Now the bytes the change read are first copied, with an exclusive
create, to gate.json.unreadable-<UTC time> beside it, such as
gate.json.unreadable-20261015T025905Z (no colons, which Windows
refuses), with -2 and on when the name is taken. Then the change is
written into gate.json in place, as before. A copy rather than a rename,
so a symlinked gate.json keeps its link, the file keeps its mode and
owner, and a single-file bind mount works; what's kept is the bytes that
were judged, whatever their encoding (a Latin-1 edit, or UTF-16 as
Windows PowerShell 5's > writes it, would not survive decoding and
encoding again), and a copy never replaces an earlier one. The change is
told where the copy is through a new onUnreadableKept option; the
wake-rule tools put that in their result as keptUnreadable, and a [gate]
log line and a gate:config-kept trace (now in the TraceEvent union) say
the same. When the copy can't be made, the change throws before anything
is written or any rule's state is cleared, and a write that fails after
the copy names it. A gate.json the process can't read at all refuses the
change: there's nothing to keep, and a file the process can't read is
often one its operator closed off on purpose. A change that writes
nothing (a placement it can't make, or removing a rule that isn't there)
keeps nothing. gate_status lists each copy still beside gate.json, in a
new keptCopies field, with what to do with it, until it's removed: the
moment a copy is made reaches only the log, the trace and the change's
caller (channel-mode in connectome-host passes no callback), and the
copy exists to be found later.
Reloading: at most once a second, gate.json is read again and applied
when its text differs from the text whose rules are in force (configText,
set only by a load or a write that applies it), and the privileged-users
file is loaded when its text differs from the text last read (a failed
read forgets that one). A gate.json that holds the text in force again
after failing to load, as after a chmod repair, has its error cleared and
nothing applied again, so the startup's error isn't cleared by a spell
when the file couldn't be read. Before, gate.json's reload compared
only its mtime, taken after the read, so two writes in one clock tick, a
write between the read and the stat, or a read that only looked
(listPolicyNames, a change that wrote nothing) left a new gate.json
unapplied until it changed again, and a chmod that made it readable
wasn't noticed. Recording the text only once it's in force also means a
reload that throws before applying is tried again. isPrivilegedUser,
which tune-out's wake classification calls for each diverted message,
read and parsed the privileged-users file on every call, and while it
couldn't be read each call pushed another copy of its error onto
gate_status's errors; it now takes the throttled check, and so does the
sleep path, through isPrivilegedUser, so an empty author id is never
privileged there either. A clock stepped backwards makes the next check
run rather than hold off both reloads until it catches up. A missing
file is ENOENT from the read itself, not existsSync, which also said
"absent" for a file behind a directory the process can't search.
The privileged-users file loads entry by entry: a non-empty string or a
safe integer is a user id, and any other entry is skipped and named in
the file's error, including a number too large for JSON to read exactly,
whose rounded value could be another user's id. A file that isn't JSON,
or isn't a list or {"userIds": [...]}, is that file's error, and the
list that last loaded in this process stays (none, at startup); it used
to load as no one privileged, without a word.
removePolicy clears the removed rule's state (delivering its pending
debounce batch, dropping its rate-limit and sampling counters) only
after the write succeeds, so a failed keep or write leaves the rule in
force with its state. It used to clear first. The write stays in place,
not a temporary file and a rename: a rename fails on a gate.json that's
a single-file bind mount, needs a writable directory, replaces a symlink
with a file, and resets the file's mode and owner.
gate_status: handleToolCall runs both throttled reloads before it reads,
so what it shows is at most a second old. Its errors, a list that every
gate.json load replaced and every privileged-users failure appended to,
becomes one current error for gate.json, one for the startup's seeding or
reconciling of gate.json (kept apart, so the constructor's own load no
longer clears it at once; it goes when gate.json loads new text or is
written), and one for the privileged-users file. A missing gate.json
clears its error. gate:config-error is traced when gate.json's error
changes, rather than at every read that fails the same way.
test/event-gate-unreadable-files.test.ts covers keeping (by add and
remove, for a JSON error and for a policy the validator refuses, the -2
name, the trace, the log line and gate_status's list), a gate.json that
isn't UTF-8 kept byte for byte, a symlinked gate.json keeping its link
and mode, nothing kept when nothing is written or the file is missing, a
gate.json that can't be read refusing, a copy that can't be made writing
nothing and keeping the rule's pending batch, a write that fails after
the copy naming it, a remove whose write fails keeping its rule and
batch, a text only looked at being applied by the next reload, the
reload's skip of unchanged text, its throttle and a clock stepped back,
both files reloading by text (same size and mtime, and a chmod repair), a
removed gate.json taking its error, files behind a closed directory as
errors, the startup's error beside a gate.json that loads, the trace
once, the privileged-users file's shapes and entries, its throttle, the
sleep path, gate_status reading first, the errors kept apart, and what
wake_add_rule and wake_remove_rule tell the model.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
keepUnreadableConfig created gate.json.unreadable-<time> with the process's default mode, 0666 less the umask, so the copy of an owner-only gate.json could be read by any local user who can search its directory, which gate.json itself couldn't (Greptile on #371). The copy is now created owner-only, then given gate.json's group, and only then gate.json's permission bits, so at no point can anyone read it who couldn't read gate.json: an owner-only file's copy is owner-only, and a group-readable one's can be read by the same group, so a group member who made the half-made edit can get it back from the copy. A copy that can't be given gate.json's group, because the process isn't in it, or whose gate.json can't be stat'd, stays owner-only. gate.json's bits alone would leave the copy in the process's own group, readable by that group's members rather than gate.json's, as a probe showed, and a fixed 0600 would shut out a group that shares gate.json. test/event-gate-unreadable-files.test.ts: a 0600 and a 0640 gate.json give copies of the same mode and group; a gate.json in another of the process's groups gives a copy in that group (it skips where the process is in no other group, as on our runs); and a gate.json linked to a file whose group the process isn't in, such as root's /etc/hosts, gives an owner-only copy, kept by calling the keep directly so that nothing is written through the link. Tests, on Linux, Node 24.21.0: of the twelve gate and tune-out files, 173 tests pass and 1 skips, the test of another group, since the user our runs use is in no group besides its own. Red-first on main's src (92125b0) with this test file: 27 of its 30 tests fail, 2 pass and 1 skips. Removals: each of 50 pieces fails at least one test of the file, including a copy created with the default mode or with gate.json's bits, no chown, no chmod, and the chmod before the chown. The test of another group, and a probe, also ran on a host where the process is in other groups. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…json that won't load keepUnreadableConfig gave the copy gate.json's group with chownSync(kept, -1, gid), -1 meaning "leave the owner as it is". Bun 1.3.14, the version connectome-host's CI and release workflows pin, refuses a uid of -1 with EPERM even for the file's own group, and the shared catch then skipped the chmod too, so under Bun the copy always stayed owner-only and a group that shares gate.json couldn't read it. The copy is the process's own file, so the chown now passes the process's uid, which an owner may always keep (-1 where there's no geteuid, as on Windows). A gate.json that stopped loading reached only the gate:config-error trace, so an operator whose edit didn't load found nothing in the log. When gate.json's error changes, configUnreadable now logs one [gate] line saying it couldn't be read and why, so each distinct error is logged once. test/event-gate-unreadable-files.test.ts: a copy of a 0640 gate.json is 0640 where chownSync refuses a uid of -1, as Bun 1.3.14 does; a change over a gate.json that won't load logs the failed load, then the copy; and a failure read again is logged once, and a new one once more. Tests, on Linux: of the twelve gate and tune-out files, 174 of 175 pass and 1 skips, the test of another group, since the user these runs use is in no group besides its own. Red-first on the previous commit's src with this test file: 3 of its 31 tests fail, these three, 27 pass and 1 skips. Removals: each of 52 pieces fails at least one test of the file, among them a chown that passes -1 again and the log line taken out. A script that keeps a copy of a 0640 gate.json that won't load got a 0600 copy under Bun 1.3.14 before this change and gets a 0640 one after, as under Node 24.21.0 and Bun 1.4.3. The whole suite passes: 2259 of 2260 tests with 1 skipped, file by file in 145 files on Node 24.21.0, and the same on Node 20.20.2.
…obably meant closestNames(name, candidates, max = 2) ranks candidates for a refusal's "did you mean", ignoring case: an equal name; then one the name matches as a whole `--` segment (`ls` for `workspace--ls`); then one that starts with the name or that the name starts with; then one that contains it or that it contains. These two need three characters on the shorter side and count however many edits they take. Last comes one a few edits away: at most a quarter of what follows the prefix the two share, and at least one, since a shared prefix is mostly a namespace, so `workspace--write` isn't offered for `workspace--read`. Within a rank, fewer edits come first, ties keep the order given, and a candidate given twice is named once. Where one name holds the other, the length difference is already their distance, so a long name, such as a catalogue pasted in by mistake, runs no edit table for each candidate. isNearMiss(name, candidate) says whether a name is a slip for another: the same ignoring case; within that edit limit, a transposition counting as one edit; or, at three characters or more, a part of it, which is a shortened attempt at it (`mode` for `deliveryMode`). A part counts only that way round: a name that holds the candidate, past the edits a slip allows, is another name, usually an extra that explains it (`channel_note` beside `channel`). The tests pin each rank against the next, a name that starts with or holds its candidate, the cut-off and how it scales past a shared prefix, ties, repeats, and isNearMiss' slips and non-slips, and, as a hang guard, a million-character name that holds all of a hundred candidates.
…s it ran
A wake rule's slips used to change what it does without a word, in gate.json
and through wake_add_rule alike:
- validatePolicy saved any behavior other than "always", "defer", "skip" or
an object as "always", a rule that wakes on every event it matches. That
covered a typo such as "deffer", and, in membrane's XML tool mode (0.5.87
parses a parameter by its declared type), the object form wake_add_rule's
own description showed: `behavior` is declared a string there, so
{"debounce": 60000} arrived as text and was saved as "always".
- A match field of the wrong type was dropped: {match: {tagsAny:
"chat:ambient"}, behavior: "skip"} was saved as match {}, a rule that skips
every event, DMs and mentions included (issue #105's hazard, reached by a
type slip).
- A field name one slip from a known one (pasthrough, keyby, polices) was
ignored, so the field meant was missing: pasthrough: true made a governor
consume the events it should let fall through. wake_add_rule's own
parameters were taken by name, so a slip of one was dropped the same way.
- So was a field put at the wrong level: {name: "quiet", tagsAny:
["chat:ambient"], behavior: "skip"} skipped every event, mentions
included, and addPolicy saved it as match {}; wake_add_rule given tagsAny
as a parameter of its own saved its rule with match {} too. A list of
rules under a name gate.json doesn't read, such as "policy", and a field
of a rule, or of its match or behavior, at its top loaded as no rules
under its default, and a change wrote the rules in force over them.
- A behavior object holding two kinds used debounce, then rate_limit, then
passive_sample, whatever the order; a keyBy or resets of the wrong type
was dropped; and a gate.json whose "policies" wasn't a list, or which was
a list, loaded as no rules under its default.
Probes at main 92125b0 showed the behavior, match, policies and
wrong-level cases.
Each is named now, every one a rule has, not only the first. What's done
with a rule that has one depends on who can fix it:
- wake_add_rule and addGatePolicy refuse it, naming every problem the rule
has, and save nothing: their caller is there to fix it. wake_add_rule's
checks of its own parameters come first, each refused on its own, a field
of match, rateLimit or passiveSample given as one of them included, naming
the parameter it belongs in.
- gate.json and the host's starting rules are what a deployment already
runs, so there a rule with a problem is read the way main read it, with a
note in gate_status's errors and a [gate] log line, and the rest of the
file loads. A rule is left out only where main's validator refused it. One
that skips or defers is read past a problem as main read it too, holding
back what main held back: left out, the events it held back would go on
to the rules after it, where a rate limit, a sample or a debounce would
spend them and then hold back an event main woke on. A probe that sends
one stream of events through both gates found 72 such events in 64 of 144
configurations where an earlier version left these rules out.
- Under a default that holds events back, "skip" or "defer", a file that
leaves out a rule that could wake the agent isn't used: it would hold back
what that rule was written to wake on, and main, which refused such a
rule and so the whole file, didn't use it either. A start then runs the
host's starting rules, or none, which hears every event, and a reload
keeps the rules in force. A rule left out that wakes on none of what it
matches (one that skips or defers, or a rate limit whose tokens are a
number at or under 0) holds back nothing it was written to wake on, so it
doesn't count, and a deliberately quiet file loads as it did; one whose
tokens are missing or not a number, or whose behavior is a list or names
no kind, was written to wake on something, so it counts.
The promise: no event that woke the agent on main is silenced, unless main
refused the whole file and so ran something else in its place (at a start
the host's starting rules or none, at a reload the rules it had), or
refused one of the host's starting rules and so ran them all unvalidated.
What remains of that exception, under "always", is an event main woke on,
having refused the file, that a rule after the one left out now holds back:
one that skips or defers it, a rate limit or a sample that lets only some
through, or a debounce that delays it. Finding those would mean judging what
each rule matches.
How each class is read, from main's validatePolicy at 92125b0:
- Left out, main having refused it: a rule that isn't an object or has no
name; a behavior that's a list, or an object holding no kind; a rate_limit
or passive_sample that isn't an object or lacks tokens, refillIntervalMs or
every; a debounce, tokens, refill or every that isn't a finite number; a
rate_limit with tokens at or under 0 (it woke on nothing); an invalid
regex filter; and a passthrough that isn't a boolean or sits on a behavior
that doesn't count. For the starting rules, main ran these raw, from
memory, once the gate.json it seeded failed to load, and some woke the
agent; they stay left out and named at start, rather than carry main's
accidents into the reader.
- Read the way the starting rules ran it, in gate.json too, with a note: a
debounce outside 100-300000 ms at its nearest bound; a passive_sample
every at or under 0 as 1, and a fraction as its next whole number; and a
refillIntervalMs at or under 0 as a bucket that never refills, so it wakes
on its first tokens and then on nothing it matches. applyRateLimit refills
only for an interval over 0, and so does wouldTrigger, which probe() and
wake_add_rule's before-and-after table read. A refill of 0 refilled once
on main, since floor(elapsed / 0) is Infinity and lastRefill += Infinity *
0 is NaN; main refused it in gate.json and ran it only from the starting
rules, which the exception covers.
- Read as main read it, with a note, a rule that skips or defers included,
whose note adds that it skips or defers every event it now matches, as
main did: a behavior that isn't a name or an object (typo, number, the
object form as text) as "always", a slip of "behavior" itself included;
two kinds, by main's precedence; a match that isn't an object as matching
every event; a match field of the wrong type, a slip of one, an entry of
the wrong type or "" in a list, a filter main couldn't read, a field the
gate reads at another level, named with where it's read, and a keyBy or
resets of the wrong type, or a slip of keyBy, resets or passthrough, each
dropped. In a rule the gate reads, a match left with nothing that narrows
it, once something in it was named, gets a note that it matches every
event. In a refusal of such a rule, where nothing was saved, the note says
the rule as written would match every event.
- An empty scope entry, which main kept and which narrows a rule, since no
event's type is "", is read as main read it whatever the rule does.
- gate.json itself: one that's a list, or whose policies isn't a list and
holds something, or is missing or holds nothing beside a slip of it
holding something, main loaded as no rules under its default; it is read
so, with a note, where that default is "always", and isn't used where it's
"skip" or "defer". Where policies is missing or holds nothing, a field of
a rule, or of its match or behavior, says a rule was meant: a list holding
an object with such a field, under any name gate.json doesn't read, is
such a slip, and so, under "skip" or "defer", is such a field at the top;
main used such a file, with none of those rules in force. A rule's name, a
filter's type and a keyBy shape nothing on their own, and nor does a field
of a rule or its match holding nothing, an empty list or an object holding
only empty values, since an empty one has always meant no constraint, so a
label of the file can have their names, or hold nothing: at the top, or in
a list whose objects hold no field that says a rule was meant, they load
as written, as on main, with a note, and a change keeps them. A behavior,
or one of its kinds, counts even when it holds nothing, as one does in a
rule left out; a null or "" is a field left out, in a list's objects as at
the top. A label named as such a field, such as "source": "written by
ops", counts as that field, since nothing in a name tells the two apart. A
policies that isn't a list, or a slip of it, holding nothing, such as
"policies": {} or "polices": [], writes no rules, as [] does: it loads as
no rules, as on main, with a note saying what its default then does, and a
change writes the list in place of such a policies, or beside such a slip.
Beside a policies holding nothing, [] included, a slip of it holding
something, a list of rules under another name and a field at the top that
says a rule was meant are read as they are with no policies. A list
holding no rule, such as "comments": ["quiet for now"] or "policy": [],
loads as written. A slip of "default" reads as "always", as on main. JSON
that won't parse, a non-object, and a default other than the three are
refused whole, as on main. A policies left out, null, "" or [] writes no
rules, and loads so under its default, as on main.
- A null is a field left out, and so is "", with a note that says what
leaving it out does where that's more than no constraint: a match left
out matches every event, and so does one whose fields are all ""; a
behavior is "always"; a keyBy leaves one bucket or count for every event
the rule matches; gate.json's default is "always"; and its policies leave
no rules, which under "skip" or "defer" holds back every event. main read
most fields that way; where it refused a rule or the file for one (a
passthrough, a behavior's kind, gate.json's default), the rule or file
now loads without it.
The reader is one walk, readRule, that collects every problem with the
reading main gave it (or none, where main refused the rule); readConfig reads
a whole gate.json or starting config with it, and ruleOutcome decides each
rule: as written, read, or left out. validatePolicy and validateConfig go.
Around it:
- The starting rules are read rule by rule, at the first start and in the
reconcile, the same way gate.json is, so the condition above applies to
them too: under a starting default of "skip" or "defer", a starting rule
left out that could wake the agent means none of them is used, and the
start runs gate.json's rules, or none, which hears every event. A first
start writes the starting config as given, since gate.json is read the
same way, so its load names each problem until someone fixes it.
- A write edits gate.json's own rule list by name: addPolicy and
removePolicy, behind wake_add_rule and wake_remove_rule, and the
reconcile, write back every rule they don't change as it was written,
problems and unknown fields included, and gate.json's own fields with
them. The rules in force are then read from the text written, as a load
reads it, so what gate.json still holds that the gate reads past or leaves
out stays named. A rule left out can be replaced or removed by its name.
Before, a write wrote the rules as the gate held them: a "comment" didn't
survive one, and a rule left out would have dropped from the file.
- A gate.json that's JSON the gate doesn't use is edited the same way, and
only what loads is written: a change that doesn't mend what keeps the
file out is refused, naming that and what would bring it into force,
worked out from the file as it is, and the file stays as its operator
wrote it. Where one named rule keeps it out, replacing or removing that
rule by name brings it into force, and the refusal says so; where a rule
with no name, two rules or a bad default keep it out, no one change can,
and the refusal says whoever runs the host has to fix gate.json. Where
the rule shares its name with another, a replacement by that name takes
the first of them and a removal all, so the refusal says that instead of
naming a change that would replace or remove a sound rule.
A change used to write the rules in force over such a file, losing its
other rules, its own fields and its default.
- When gate.json is missing, can't be read or isn't JSON, a change edits
the list of the text whose rules are in force the same way, after keeping
a copy of a file that can't be read or isn't JSON, and writes the rules
in force only where there's no such text or it has no rule list. So does
a change after a start whose reconcile write failed partway: the text it
was writing is the text in force. The copy's log line, keptCopies entry
and keptUnreadable say gate.json "couldn't be used", true of both.
- A rule gate.json holds holds its name, whether the gate reads it or
leaves it out, so the reconcile adds only starting rules whose names the
file lacks. Why: an operator's "ambient" skip rule narrowed to Zulip, with
an empty tagsAny entry main read past, holds its name; had the reconcile
added the host's "ambient", which skips all ambient chat, Discord's
ambient chat would go quiet under the operator's "always" default, where
main woke on it. It takes both halves: the name held, and the rule kept
through every write, over a gate.json that won't load included.
- The reconcile never writes a gate.json it would then refuse: where adding
the starting rules would leave out one that could wake the agent under
gate.json's own "skip" or "defer" default, it adds none, and the startup
entry says why.
- A gate.json with no rule list a write could edit without writing over or
leaving out what it holds (a list, a policies that isn't one and holds
something, a slip of policies holding something where policies is missing
or holds nothing, which holds the operator's rules, a list of rules under
another name, a field at the top that says a rule was meant, under "skip"
or "defer", a label named as one included, or JSON that isn't an object)
refuses a change, an addition or a removal, whatever its default, naming
the fix and who can make it, and saves nothing; the reconcile adds nothing
to it and says so. A removal there used to answer that no rule had the
name, though the file can hold it, so a host's removeGatePolicy meets the
same refusal as its addGatePolicy. An addition there used to write the
rules in force over the file, dropping the operator's rules; a list added
beside them would bring the file into force with only the new rule. Where
gate.json can't be read at all, nothing is kept and a change is refused,
as before, and the refusal now says whoever runs the host can make it
readable, and that a change is refused the same way until then.
- Also: a field name the gate reads nowhere and near none it reads, such as
a "comment", loads and is named, in gate_status's errors, the result's
ignoredFields, or onIgnoredFields; a slip is named with every field it
could be meant for (isNearMiss and closestNames, in the commit below this
one); wake_add_rule's own parameters are checked the same way, and it
refuses a behavior, position, insertBefore or insertAfter of the wrong
kind in its own terms, its example giving debounceMs: 60000; what the
start did is an entry of gate_status's errors worded about the start, kept
until the next start, beside an entry for rules the start couldn't save,
where a failure in writing or closing, unlike one opening, may have left
gate.json part-written; the startup entry checks the names gate.json
holds, not the rules it runs, before it says a starting rule wasn't added;
a reconcile whose write leaves gate.json unable to load runs what it read
plus what it was adding, and its entry, which names what those hold that
the gate reads past or leaves out, goes once gate.json is written, or a
reload applies its text or finds it holding the text in force;
configSource is "initial" while the starting rules run; gate_status's
entry for a gate.json the gate doesn't use says what's in force instead:
the rules of the last gate.json it used, the rules a start read and was
adding when its own write left gate.json unable to load, the host's
starting rules, or none; a loaded gate.json's notes stay with the text in
force (configNote); the log line for a gate.json that won't load says it
"couldn't be used", as the copy's texts do; and a value a message quotes,
over 80 UTF-16 units, and a skipped privileged-users entry, over 40, are
cut with safeSlice, so neither ends on half a surrogate pair.
The gate's other messages keep their wording.
Tests, on Linux, Node 24.22.0: test/event-gate-strict-rules.test.ts, 26
tests, among them each class read or left out in gate.json with the events
it wakes on, a rule that skips or defers read past a problem as main read
it, ahead of a rate limit that spends its tokens only on what reaches it, a
file that leaves out a rule that could wake under "skip" or "defer" not
used, at a start, with and without starting rules, at a reload, and as a
starting config, beside the rules left out that don't count, every problem
named at once, a field at another level read past in gate.json, with how the
rule then reads for a field of each level, and refused by addPolicy and
wake_add_rule, a list of rules under "rules" read as a slip of "policies",
and one under "policy", a rule's fields, a match field, a rate limit's, a
filter's type beside its pattern and a "source" label at the top not used
under "skip" or "defer", with lists that hold no rule loading as written,
and a rule's name, a filter's type or a keyBy at the top, or a list of
objects holding no field that says a rule was meant, a field of a rule or
its match holding nothing, and a policies or a slip of it holding nothing,
loading as written, quiet, and kept by a change, a value a rule would drop
and a behavior or kind holding nothing at the top, or in a list's object,
not used, a slip holding a rule, a list of rules under another name or a
rule's fields beside a policies holding nothing not used, a change refused
there, a list holding null or an object whose rule fields are null or ""
loading as written, and a list mixing a label's object with a rule not used,
gate_status's entry saying what's in force, after a start whose write left
gate.json unable to load too, the starting rules rule by rule, a write
keeping the rules it doesn't change, over a gate.json that isn't JSON too, a
change refused where gate.json has no rule list, a slip of "policies" and a
"source" label among them, a change over JSON the gate doesn't use refused
unless it mends it, with the file left as written and the refusal naming
what would bring it into force, and the operator's "ambient" holding its
name through a change, a restart and a change mid-edit.
event-gate-rate-limit's validation tests pin tokens of 0 and -2 left out and
a refill at or under 0 read, with the bucket never refilling and the probe
saying so at each step; event-gate-unreadable-files' gate.json refused
whole, for a default other than the three, is now neither written over nor
kept, the refusal says whoever runs the host has to fix it, the log line
says "couldn't be used" once, its copy says "couldn't be used", a gate.json
that won't load has an entry saying what's in force, and one that can't be
read refuses a change saying whoever runs the host can make it readable;
event-gate's two debounce-bounds tests pin an out-of-range debounce in
gate.json read at its bound, where it was refused whole. Red-first on the
parent commit: 40 of the four changed files' 135 tests fail and 1 is skipped
(25 of the 26 strict-rules tests, the round-trip control passing; 3
rate-limit; 2 event-gate debounce bounds; 10 unreadable-files). Removals:
each of 186 pieces, undone on its own, fails at least one test. Fourteen
files (event-gate's nine, closest-names, mcpl-gate-roundtrip, api-admin-gate
and tune-out's two): 206 of 207 pass, 1 skipped (the other-group test).
Probes that run main's gate beside this one: of 142 gate.json
configurations, none silences an event main woke on where main loaded the
file (146 events where main refused it, the exception); one stream of events
through each gate, over 144 configurations, a skip or defer rule ahead of a
rate limit, a sample or a debounce, with one of eight problems or none (the
controls): every event decided the same on both; the recipes and other
starting rules, first and later start: none; an operator's narrowed
"ambient" through a change, a restart and a change over an unloadable
gate.json: Discord's ambient chat wakes on both; every waking rule left out
under "skip" or "defer": every event wakes on both; a field at the wrong
level, a list under "policy", and a rule's fields, a match field or a rate
limit's at the top, with starting rules and without: none goes quiet, and
the saves and writes main made of them are refused; labels at the top named
name, type or keyBy, a list of objects holding none of a rule's fields, and
fields of a rule or its match holding nothing, at the top or in a list's
object: decided alike on both, and an addition keeps them where main wrote
over them; a policies or a slip of it holding nothing: decided alike on
both, and an addition writes its list there, keeping a slip beside it; a
"source" label, a filter's type beside its pattern, a value a rule would
drop, a behavior or kind holding nothing, and a slip holding a rule, a list
of rules or a rule's fields beside a policies holding nothing: not used, as
a match field at the top isn't. connectome-host's four recipes load with no
errors on a first and a later start: clerk 5 rules, knowledge-miner 3,
knowledge-reviewer 3, zulip-miner 2. The suite: merged onto main 3adbdb8,
each of its 155 test files run on its own on Node 24.22.0 passes 2496 of
2497 tests, with 1 skipped, which needs a second group the run's user isn't
in; main run the same way has 152 files and 2434 tests, and the three new
files hold the other 63.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The copy of a gate.json that won't load takes gate.json's group and permission bits, so that it's no more readable than gate.json. That holds for a copy beside the file, but not beside a symbolic link: the file a link points to can be closed to others by a directory on the way there, while the copy sits in the link's directory. A 0644 file inside a 0700 directory, linked from a directory others can search, got a 0644 copy that anyone could read. The copy of a gate.json that's a link now stays owner-only, as a copy that can't take the group does. Tests, on Linux, Node 24.22.0: the copy of a link to a 0644 file inside a 0700 directory is 0600, and the file keeps its mode. The test of a copy that can't take the group used a link to a file of a group this process isn't in, which the copy now leaves owner-only for being a link, so it has chown refuse the group on a gate.json that isn't a link.
Takes its owner-only copy of a gate.json that's a symbolic link. The one conflict was the node:fs import: this branch's names, with lstatSync for the copy and without statSync and existsSync, which nothing here uses now. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l knows
- "__proto__": withoutEmpty copied an object's fields by assignment, so
a "__proto__" in gate.json, an own field as JSON gives it, set the
copy's prototype, and the gate read the fields of its value as the
object's own. A rule whose match was {"__proto__": {"source":
"slack"}} matched only slack's events, where the gate without this
change reads the key as a field it ignores, and the rule matches every
event. The copy is now built with Object.fromEntries, which keeps the
key an own field, named as one the gate doesn't read. The same
function looked up what leaving out a "" field means on a plain
object, so a "" field named "constructor", "toString" or "__proto__"
found an Object.prototype member and its note printed that member's
text. It now reads only the table's own fields.
- A write's state: a change put in force what it wrote without clearing
the state of the rules that reading left out or gave another behavior.
A rate limit left out by a write kept its spent bucket, and when
gate.json restored it, the reload took it for a new rule and reused
that bucket, holding back events it should wake on. The reload's
cleanup is now one helper, clearStateFor, which a write calls too,
last, once its text is written and its rules are in force, so a write
that fails leaves the rules in force with their state: a rule it
leaves out, or gives another behavior, loses its buckets, counters
and timers, and a rule left out loses its stats. Each pending debounce
batch, at a reload too, where it was delivered at once, is then sent
as its timer would have sent it, held while an agent is inferring or
the host is quiesced, and only once every rule's state is cleared.
Each is sent on its own, so a delivery that throws leaves no state
behind and keeps no other batch from being sent; the first such error
is thrown once all have been tried.
removePolicy's own clearing of the rule it removes goes, since the
write does it. A write that fails after a copy of gate.json was kept
ends its error with a stop before the sentence naming the copy.
- A behavior holding two kinds or three is read as the first it holds
of debounce, rate_limit and passive_sample, as before, but a refusal
named only the kinds it held, not the problems of those it didn't
read, so the call that followed could be refused for those.
Each other kind is now read on its own for its problems, which a
refusal names and a load reads past.
- wake_add_rule named a parameter it doesn't take only when the rule was
saved. A refusal now names it too, whether it's the tool's own refusal
or the gate's, after the refusal's own text, which gets a stop if it
has none, and a refusal of a slip or a misplaced field names it in
its own sentence.
- The changelog fragment of the change this builds on described two
things this change alters: a gate.json holding a policy the gate
refuses written over after a copy, and the start's error cleared by a
load or a write. Both sentences now say what happens.
Tests, on Linux, Node 24.22.0: a "__proto__" in a rule's match narrows
nothing and is named; a "" "constructor", "__proto__" or "toString" is
left out with the plain note; a write that leaves a spent rate limit
out, and one that gives it more tokens, each leave it a fresh bucket,
and a rule restored after the first starts with no counts, as one a
reload leaves out and restores does; a rule a write removes has its
pending debounce batch delivered, once it's out of force, and one a
write changes while an agent's turn runs is held until the turn ends; a
delivery that throws leaves the state of a rule after it cleared, and
the batch of one after it sent; a write's error before a kept copy's
name ends in a stop; a refusal of a behavior with two kinds or three
names each one's problems, in the order they're read, and a load reads
past them; wake_add_rule's refusals of a slip, of its own and of the
gate's name a parameter it doesn't take, after a stop where the gate's
text had none.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…re it isn't given
A name holding a field's whole name, such as "matchTags" for "match",
"channelId" for "channel" or "debounceMs" for "debounce", was a name
the gate doesn't read wherever it stood. So a rule given "matchTags" in
place of "match" was saved matching every event, and wake_add_rule took
"matches" in place of "match" the same way. Beside the field it holds,
such a name is an extra that explains it ("channelNote" beside
"channel"); in the field's place, it's the field under another name.
In a rule's fields, at every level, and in wake_add_rule's parameters,
a name holding the whole name of a field its object doesn't give,
ignoring case, is now a slip of that field: a change refuses it, naming
the field, and a load names it and reads it as the field left out, as
before. One holding two such names is a slip of each, and one beside
the field it holds is still a name that does nothing. It's decided
where the gate sees every name an object gives, beside isNearMiss,
which is left as it was.
At gate.json's top, a name holding "default" or "policies" stays a
name that does nothing. There a slip of "policies" holding something
keeps a file that holds events back out of force, and such a name can
be a label of the file. A list of rules under any name is already read
as a slip of "policies" where "policies" writes no rules.
Tests, on Linux, Node 24.22.0: a name holding a field's whole name,
where the rule doesn't give that field, is refused as a slip of it at a
rule's top, in its match, in its behavior and in a rate limit, naming
both, whatever their case, for one holding two names the rule gives
neither of, and only the one it doesn't give where it gives the other;
wake_add_rule refuses "matches" with no "match", and names "matchNote"
beside "match" and saves the rule without it; in gate.json, a rule's
"matchTags" is read as "match" left out, as before, and a match's
"channelNote" beside "channel" is named; at the top, "defaultBehavior"
and "policiesUpdated" are names that do nothing, so a file that holds
every event back with "policiesUpdated" is used.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A write clears the state of the rules it takes out of force or changes once its text is written and its rules are in force, then sends each pending debounce batch through the timer's path. A delivery that threw there was thrown again once every batch had been tried, so a change that was saved and in force reached its caller as a failure: addPolicy and removePolicy threw, skipping what follows the write (the kept copy's report, the ignored-field notes, the trace and the return), and wake_add_rule and addGatePolicy said the change failed. A reload, which clears before its swap, threw before the swap, so the rules it was replacing stayed in force with their state cleared, and its catch said nothing. Each batch whose delivery throws is now named in a [gate] log line and dropped, and the rest are still sent, so a write or reload that did its work returns as one that did. The batch can't usefully be kept: by then its rule is out of force or has another behavior, so it would sit with no timer, and a batch its timer can't deliver is dropped the same way. The changelog fragment says so. Tests, on Linux, Node 24.22.0: a write whose first batch can't be delivered returns the rule it saved, which is on disk and in force, logs that batch as dropped, clears a spent rate limit after it and sends the batch after it; a reload whose batch can't be delivered puts the new rules in force, so a mention wakes the agent, and logs the batch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Takes main at e670363. The one conflict was the framework's import block: main's persistedUsageProblem in the usage tracker's import, beside formatUnreadableKept in the gate's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… a callback throws The copy of a gate.json that won't load took gate.json's group and permission bits, and stayed owner-only for a link. Two setups got past that. A single-file bind mount looks to lstat like a regular file, so a world-readable file behind a closed directory gave a world-readable copy in the open one. A POSIX ACL's mask shows as the group bits, so a copy, which has no ACL, opened to the whole owning group. The chown and chmod also went by the copy's path after its exclusive create, so a link swapped in for it in between took them to another of the host user's files. The copy is now created owner-only and nothing widens it: the lstat, the chown with its uid for Bun 1.3.14, the chmod and the link case are gone. The host read gate.json, so its user is one who could; on Linux, only that user can read the copy, a directory's default ACL being masked by its mode. The changelog fragment says so. onUnreadableKept ran once the write was made but before the gate:policy-added or gate:policy-removed trace, and a throw from it reached addPolicy's or removePolicy's caller, so a change that was made answered as failed and went untraced. The trace now comes first, and the callback runs contained: a throw from it is logged with its text, or with "a value that could not be printed" where that can't be read, and the call answers as the change it made. Tests, on Linux, Node 24.22.0: a 0600, a 0640 and a 0644 gate.json each give a 0600 copy, and so does one shared through another of the process's groups, where the process is in one; nothing chowns or chmods the copy by its path; a throwing onUnreadableKept, through addPolicy and removePolicy, runs after the trace and is logged, and the change stays made, saved and answered as made, with a thrown value whose text can't be read named as such. The test of chown refusing -1 under Bun goes with the code it tested. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Takes the branch this one builds on at its new head, which merges main at e670363 and keeps the copy of a gate.json that won't load owner-only, with its callback called once the change is made and traced. Conflicts: - the framework's import block: this branch's gate imports beside main's persistedUsageProblem; - keepUnreadableConfig: the group and mode steps go, as there, and the log line keeps this branch's "couldn't be used"; - addPolicy and removePolicy: this branch's edit of gate.json's own rule list stays, and its callbacks run after the trace, each contained, as the other branch's onUnreadableKept now is; the onIgnoredFields callback takes the same rule; - the gate's two errorText helpers are now one, the one that can't throw; - the doc comments of addPolicy and addGatePolicy, and the changelog fragment's sentences on the copy's access and its callback. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… to the log A gate.json under "skip" or "defer" whose rules sat where the gate doesn't read them, under a slip of "policies" or another name, or with a field of a rule at its top, or whose "policies" wasn't a list, went out of force, where earlier releases loaded it and held back every event. At a start, out of force ran the host's starting rules in its place, or none, which wakes the agent on every event, and the file's lack of a rule list then refused every change, so an agent its operator had muted woke on everything and couldn't undo it. Nothing in a name tells rules parked on purpose, such as under "disabled", from a misnamed list, so such a file now loads as earlier releases read it: each such name does nothing, and a note names it, a slip of "policies" with what the file's default then does. With no rules, the default decides every event, as before. A change keeps each such name as written and adds a "policies" list beside it; it still refuses to write over a "policies" that isn't a list and holds something. The check of which fields say a rule was meant, and its list of label names, go with the refusals they decided. A pending batch held for an agent's turn, or while the host was quiesced, was delivered in a microtask that didn't catch, so a delivery that threw, as addMessage does with no agent configured, was uncaught and stopped a host with no handler for that, and a write's batch now takes that path. A batch its timer sent threw the same way. fireDebounce now catches its own delivery, logging the batch it drops, and so does the buffer's flush, so a write, a reload, a timer and a turn's end each drop a batch that can't be delivered the same way. Rules that share a name share its state, and clearing compared each rule with the last rule of its name, so with two rules of one name every write reset their state, and giving the first the last one's behavior kept the first's timer. A name now keeps its state only where its rules keep their behaviors, in order. A refill interval of 0 read as never refilling, so a starting rule that had refilled once woke the agent on fewer events than it did. It now refills at once, in gate.json and the starting rules alike, so the rule limits nothing, and the probe reads it the same way. The text that said an out-of-range number reads the way the starting rules ran it now says where it doesn't: a debounce is read at its bound, which can send a batch later or sooner, and the changelog lists each bound and which way it moves. Tests: the file-level readings turned round, loading under each default with a write adding a list beside what the gate doesn't read; a "policies" holding only null and "" written over as one holding nothing; the starting config read the same way, and refused whole where it leaves out a rule that could wake the agent; a later start adding its rules beside a list the gate doesn't read; a refill interval of 0 waking on every event, with the probe agreeing on a spent bucket; rules sharing a name keeping a pending batch through a write that changes neither, and losing it when one changes or when a reload swaps their behaviors; a batch whose delivery throws dropped and logged from its timer, after an agent's turn and after a quiesce; every rule cleared before a write sends any batch; and an onIgnoredFields that throws logged once the change is traced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A later start over a gate.json the gate loads but that has no rule list to add the host's starting rules to (a list, or a "policies" that isn't one and holds something) wrote nothing and put none of them in force. Earlier releases wrote them over the file, losing what it held, and ran them under its default, so under "skip" the events they woke the agent on went quiet. Now the start writes nothing over it and runs them under gate.json's default, beside it and unsaved, as it runs what a reconcile was writing when that write left gate.json unable to load. gate_status says so, with who can give gate.json a list, until gate.json changes and loads; a reload that finds it unchanged keeps them. Where, under that default, one of them left out could wake the agent, or the starting rules aren't used, none runs, as before. A copy of a gate.json that can't be made says who can fix it, and that a change is refused the same way until then. The log line for a batch whose delivery throws gives its size. The docs of addGatePolicy, removeGatePolicy and addPolicy, and the changelog, say a copy is kept of a gate.json that could be read but isn't JSON, where they said one that can't be read too, which is refused, and that the change then edits the text whose rules are in force, or the rules in force. The reading rules' header and the changelog name a null or "" where "for one" also read as any field, and readRateLimit's doc says which rate limits without usable tokens are quiet. Tests: a later start over a gate.json that's a list, and over one whose "policies" isn't, runs the starting rules it lacks, writes nothing, says so through a reload that finds gate.json unchanged and not once it changes, and runs none where one of them is left out under that default or the starting rules aren't used; a copy whose write fails once it's made is removed; a dropped batch isn't traced as delivered or sent again; the log line gives the batch's size, one event or more. The rate-limit tests' helper and two hot-reload tests no longer wait for an mtime the gate doesn't read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Beside a gate.json with no rule list, the host's starting rules are what's in force, so gate_status's configSource says "initial", and a reload that finds gate.json unchanged keeps them and their entry, read from that source where the start kept a flag of its own; once gate.json is unusable, its entry says the gate runs the host's starting rules. gate.json's own load note says the gate runs them beside it, under its default, where it said every event got the default, or was held back. The start's entry names only the starting rules it runs, one the starting rules leave out being named so there; where every one is left out, none runs and the start says they weren't put in force. That entry says a start then adds them to a list as to any gate.json with one, where it said the next start would. A start's entry for starting rules it couldn't add says whoever runs the host can fix what kept them out. The keptCopies doc says a copy is kept of a gate.json that could be read but isn't JSON, and the quiet flag's docs and readRateLimit's say which rate limits without tokens are quiet and what leaving one out holds back. Tests: the start beside a list and beside a "policies" that isn't one, with gate.json's entry, configSource "initial", a left-out starting rule named so, a reload to text that won't load naming the starting rules as what runs, and every rule left out; a copy whose open fails leaving a file of its name as it was. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A read of gate.json that finds the text in force gives gate.json's error that text's notes, as an unchanged read did, so a change refused for want of a list, which reads the file to edit it, no longer swaps the note saying the host's starting rules run beside gate.json for the file's own. listPolicyNames reads gate.json unless it holds the text in force, so it names the rules in force, which beside a gate.json with no rule list are the host's starting rules; loadConfig, its only user, goes. The quiet reading's docs say that what a rule left out matches goes on to the rules after it. The constructor's and seedOrReconcile's comments and the changelog say that none of the starting rules runs beside such a gate.json where, under its default, one of them left out could wake the agent. configErrorShown's comment says an unchanged reload clears the start's note only while gate.json's rules are in force, and the changelog says "default" was what a start said while the starting rules ran in place of a gate.json that didn't load. Tests: a change refused for want of a list leaves gate.json's entry as it was, and a listing then names the starting rules in force. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…beside it At a later start over a gate.json with no rule list, the host's starting rules run beside it under its default. Where, under that default, one of them left out could wake the agent, the start ran none of them and used gate.json, which has no rules, so under "skip" or "defer" every event was held back. Earlier releases wrote the starting rules over the file, couldn't load what they wrote, and ran them from memory under their own default. Now gate.json isn't used at that start, which runs as one without it does: the starting rules as read, under their own default. Where they're refused whole under their own default and gate.json's holds events back, the start runs no rules, so every event wakes the agent, where it ran gate.json's none. configSource says "initial" or "default", and gate.json's entry and the start's name the rule left out, what runs instead, and that giving gate.json a list of rules, or fixing the starting rules, is the way back. A reload that finds gate.json unchanged keeps what the start ran; once gate.json changes and loads, what it holds is in force. Tests: a starting rule left out that could wake the agent under gate.json's "skip", where the starting rules run under their own "always", with both entries, an unchanged reload and a change that loads; and starting rules refused whole under their own "skip" beside it, where no rules run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… whole Beside a gate.json with a list of its own, under "skip", starting rules refused whole leave its rules in force, with configSource "file", where only a gate.json with no rule list isn't used at that start. The comment above the case of a gate.json with no rule list under "always" says what runs there: its own reading, no rules, every event waking the agent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
At a later start over a gate.json with a rule list but no rules of its own (an empty list, a "policies" left out, "" or holding nothing, or a list holding only rules the gate leaves out), where one of the host's starting rules left out could wake the agent under gate.json's default, or the starting rules are refused whole and that default holds events back, the start ran gate.json's rules, none, so under "skip" or "defer" every event was held back. Earlier releases added the starting rules to gate.json, couldn't load what they wrote, and ran them from memory under their own default. Now gate.json isn't used at that start, as where it has no rule list: the start runs the starting rules under their own default, or no rules where they're refused whole, and gate.json's entry and the start's say why and give the way back. A change at such a start is refused while gate.json holds the text the start didn't use, since the change would put gate.json's rules in force in place of what the start runs; once gate.json changes, a change edits what it holds. Once it's removed or can't be read, a change edits the rules in force rather than gate.json's list, as where it never loaded. Tests: each such list beside starting rules with one left out, with its entries, a refused addition and removal, an unchanged reload, and a change once gate.json changes or is removed; starting rules refused whole beside an empty list; and a list with a rule of its own, whose rules stay in force. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Over a gate.json with no rules of its own, a later start ran as one without gate.json only where a starting rule it would add was left out and could wake the agent. Two more such rules weren't counted: a starting rule with no name, which the gate leaves out and a start can't add by name, and a starting rule whose name gate.json holds only in a rule the gate leaves out, which keeps the host's rule of that name out. Under gate.json's "skip" or "defer", either left the start running gate.json's rules, none, holding back every event a host's rule would have woken the agent on. Earlier releases wrote a nameless rule into gate.json and couldn't load what they wrote, and refused gate.json whole for a rule it leaves out, so both ran the starting rules from memory. Now the start counts each starting rule that could wake the agent and that it can't put in force, left out, with no name, or kept out by name, in one check before the reconcile; gate.json's entry and the start's name each, a nameless one as "one with no name", with what would put it in force. The reconcile beside a gate.json with no rule list builds the rules it runs from the starting rules as read, and the checks of what it would add, which that one check now covers, go. configText's and readConfigFile's docs say what configText holds while a start runs rules beside or in place of gate.json. Test comments say gate.json's default where it decides, and the starting rules as read. Tests: a nameless starting rule beside an empty list; a starting rule whose name gate.json holds in a rule the gate leaves out, and one such that skips, which doesn't count; and a gate.json with no rules under "always" beside starting rules refused whole waking the agent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Both start entries now say gate.json "has no rules the gate reads" where a start doesn't use it, true of an empty list and of one holding only rules the gate leaves out, as gate.json's own note says, where they said it had no rules. Tests: a quiet starting rule left out, a rate limit of 0 tokens, beside an empty list, which doesn't count, so gate.json is used and the reconcile writes it in; and a start that can't put in force a starting rule left out, two with no name and two kept out by gate.json's rules of their names, whose entries name each and every way back. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…stops a page before it would spill What the text is, and a bound on one page, built on resultMaxChars from the commit before. What the text is. read decodes through fileText, new in text.ts: the lines read shows, without a BOM and with a CR before a line feed dropped, where each starts in the stored text as character paging reads it, the CRLF and LF counts, and whether the bytes are valid UTF-8. So a CRLF file's lines come without the CR, a BOM isn't shown, and the page's note says when the file has either, with the counts where CRLF and LF ends are mixed. Bytes that aren't UTF-8 show as U+FFFD, as before, and the note now says so, and that edit refuses the file and write replaces it whole (handleEdit's refusal on this branch). A character page still reads the stored text as it is. A long line. A line over 2,000 characters is cut, without splitting a surrogate pair, with how many characters follow and the offsetChars that reads them, counted in the stored text as character paging reads it, BOM and CRs included. A bound on one page. The framework stores a success's data as JSON and writes what passes the call's resultMaxChars to a file, cut mid-JSON. So a line page stops at the last line whose page fits, and a character page at the last whole character that fits (never inside a surrogate pair), each with a note saying where to continue. Each is costed as JSON writes it (jsonLength) beside the rest of the page, allowing for the page's largest numbers; a line page then adds lines while the exact page still fits. Where even the first line would pass, it is cut shorter, at the most that fits, and a character page always moves on by one character, so a limit too small for a page's own fields still gets the reader somewhere. A call without resultMaxChars, one whose caller takes the result in process, uses the framework's default, 24,000. The page logic is src/modules/workspace/read-page.ts (linePage, characterPage); handleRead now only finds the text. read's description says all three. Tests, in test/workspace-text.test.ts: fileText's lines, starts and counts, for CRLF, LF, mixed ends, a lone CR, a BOM, an empty text and invalid UTF-8. In test/workspace-read-text.test.ts: - A CRLF file reads as its lines without the CR, with its note; mixed ends are counted, and a CR that doesn't end a line stays the line's own. - A BOM is noted and not shown, a character page still shows it, a BOM alone is an empty file that has one, and invalid UTF-8 is noted on a line page and a character page. - A long line is cut at 2,000 characters before an emoji's surrogate pair, and the offsetChars it names, past a BOM and a CRLF, reads exactly the rest. - A line page stops before its result passes 24,000, 5,000 and 1,000 characters; one more line would have passed, and the note's offset reads on. - A first line that alone would pass 1,000 characters is cut where it fits. - A character page of escape-heavy text stops within 24,000 and 1,000, its pages put back the whole text, and a stop never splits a surrogate pair, at four neighbouring limits. - A limit of 50, too small for a page's own fields, still moves a character page on by one character, and shows a first line as its marker alone. - The 2000-line default holds for a call whose limit lets it, a script's. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…hows them
grep drops the CR before each line feed with read, for matching as well as display, so foo$ finds a CRLF
file's line, as it does under ripgrep's --crlf. grep shared linesOf with read, and with hands' planner coming to
edit, a CRLF line copied from grep ("foo\r") would miss: the planner makes
every CRLF in an edit's old text LF but keeps a lone CR, as hands' match.ts
does.
grep now searches fileText's lines: without a CR before a line feed and
without a BOM, so `foo$` finds a CRLF file's line and `^foo` a BOM file's
first. Each file's result says `lineEndings: 'CRLF'` or `'mixed'` when the file
has CRLF ends, and says nothing for LF alone. The disk version kept with a
conflict is searched the same way. linesOf, which only grep still used, goes.
grep's description says so. The cost, which the changelog names: a pattern
written to find a CR at a line's end no longer matches.
Test, in test/workspace-file-tools.test.ts: `^foo$` with context finds a CRLF
file's line with its context lines all without the CR and lineEndings 'CRLF';
a mixed file says 'mixed', an LF file nothing, a BOM file's first line matches;
and `\r` finds nothing in a CRLF file. And a test of the copy into an edit:
two lines copied from a read of a CRLF file with a
BOM, and then a line copied from a grep match, each match in an edit through
the planner this branch is stacked on, and the file keeps its CRLF ends and
its BOM.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ting and an edit do The read half of the disk check, last here since it changes most what a read does with disk. A read of a path the workspace held didn't look at disk (ensureSynced returned early), so it could show a copy another writer had changed since. Now handleRead passes over its path first, under the three-way rule a listing uses, as handleEdit does on this branch: - disk changed and the store didn't: the change is taken in, and the note says this read shows that text; - disk's copy couldn't be checked: the note says this read shows the workspace's copy; - disk has no copy and nothing records whether it was deleted there (disk-missing-provenance-unknown): the note says so, and that this read shows the workspace's copy, as handleEdit now says for an edit; - disk deleted the file: "File not found", saying disk deleted it and that was taken in (a copy disk made binary or too large keeps the disk-only answer); - the stored copy is a draft and disk changed too: the read shows the draft, and the note names the conflict and both ways to settle it, as the edit's answer does (a refusal would hide the text the agent needs to settle it). A file the workspace didn't hold is taken in with no word, as before. The note rides on a line page and a character page alike (characterPage takes extra notes now). readBinary, the framework's read_image route, settles the same way, so its doc comment's "as the read tool" stays true. A deletion still observes only a path the workspace doesn't hold (ensureSynced, its doc now says why): taking disk's change in first would let its push remove that change unseen, as for a write. Three tests in test/workspace-three-way.test.ts read the store through read, to see what a pass had or hadn't taken in: a sync asked on one branch takes nothing on another, a restored timestamp hides no change, and a tracked link the walk cap passed over isn't read through. A read now settles its own path, so they read the store directly (storedOf, beside contentOf, whose doc now says a read settles first). Before, "the read serves the store" was the first assertion of the second; that is the behaviour this commit changes. Tests, in test/workspace-file-tools.test.ts (6): disk's newer text is shown with the note, the next read has none, and a character page says it too; a file disk deleted; a draft whose disk copy changed shows the draft with the conflict note and leaves disk alone; a directory in the file's place reads the workspace's copy with the unverified note; an entry disk lacks with no record reads with the missing-copy note; readBinary returns disk's new bytes. Red-first: with this commit's source changes undone, all five fail and the other 29 in the file pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ry keeps the workspace's copy, and a deletion conflict says what a sync takes What review found in the read half, and the fixes: - A page whose last line is cut, at 2,000 characters or as a first line cut to fit, named the next line as where to continue. Reading on in order starts with that line's rest, which only the line's own marker named. The note now names the offsetChars of the rest before the next line's offset, on a limit's note and a budget's alike. A page that ends the file has no next line to name, so its cut line's marker stands alone, as before. - The first line's cut is sized on the page itself, now that the note names the cut. The line as a page shows it is tried first: shown whole, it carries no marker and its note no cut, so its page can fit where a cut one character shorter doesn't (a one-line file of 900 characters at 1,000: 984 characters whole, 1,032 cut at 899). Below that, a longer cut never makes the page shorter, and a binary search finds the most that fits. The estimate it replaces reserved the longest note, so on a page that ends the file, which carries none, the cut stopped short by that note's length. - readBinary goes back to observing only a path the workspace doesn't hold (ensureSynced), as before this branch. The pass takes no binary disk copy into the tree, so settling dropped a held binary file that disk had changed, and readBinary, a reader of binary content, answered File not found for a file disk still had. read_image doesn't reach readBinary on this module either: the framework calls the module's own read_image handler where the module serves it, and readBinary only where it doesn't. readBinary's doc and ensureSynced's now say why a held path is left alone. - A conflict's note told the reader that a sync takes disk's copy. Where disk has none, a sync takes the deletion and removes the draft, and the note now says so wherever disk has no copy now (the report's diskNow), not by the conflict's kind: a recorded conflict keeps its kind, so one recorded as both-changed before disk deleted the file is still both-changed (which the test below confirms). Where disk has a copy the workspace can't hold (binary or too large), a sync takes nothing in and only gives up the draft, so the note says what holds in both cases: a sync keeps disk's copy and gives up the workspace's. - The mixed-line-ends note said each line is shown without a CR. Only a CR before a line feed is dropped, and the note says that. - The read tool's description scopes the CRLF and BOM wording to a line page, and says a character page returns the stored text as it is, BOM and CRs included. - The changelog says where a page's bound holds: where its own fields and the least it can show fit, which at 1,000 characters only a path of about 130 characters beside every note a read can carry, or about 700 with none, passes. It also gives the cut-line note, a file disk made binary, the deletion conflict, grep's CR wording, that a path the workspace didn't hold was observed before too, and the prompt-cache miss the changed tool descriptions cost once. It no longer mentions readBinary, which this branch now leaves as main has it. Tests: - workspace-read-text: the first-line cut's note names the rest, and one character more passes the limit, on a page with a note and on a one-line file's page with none; a one-line file whose whole page fits is shown whole, where a cut one character shorter wouldn't fit; a page whose last line is cut before a surrogate pair names the offset its marker does; a page stopped at a cut line by its limit and by the result limit names the rest, and a page that ends the file has no note; a call that carries no resultMaxChars gets pages within 24,000; each of the emoji loop's four limits checks the page's bound; the mixed note's wording. - workspace-file-tools: a read of a draft disk deleted gives the deletion's note, and so does a conflict recorded as both-changed before disk deleted the file; a sync of the path then removes the draft, and a forced materialize writes it back; a draft whose disk copy turned binary gets the note that a sync keeps disk's copy and gives up the draft, and the sync does that; a read of a file disk made binary answers the disk-only text; readBinary serves a held binary file that disk changed as the workspace holds it (in place of the test that pinned the settle). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…e, not in showing it The comment above grep's search and the changelog said grep leaves out the CR before a line feed for matching as well as display "as ripgrep's --crlf mode does". ripgrep 14.1.0's --crlf lets `foo$` find `foo\r\n`, but it prints the line with its CR and refuses a `\r` in a pattern, so only the matching is ripgrep's. Both now say so. The changelog also says "no note" where it said "no word", and says what the framework did with a page past the limit: it kept only the start, cut mid-JSON, and wrote the whole result to a workspace file where the agent has one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…t finish When the selected branch keeps changing while a read's pass gathers, the pass decides nothing: it leaves no report for the path, only an incomplete reason, "the selected branch kept changing while the scan ran". The read then showed the stored copy with no note, or, for a path the workspace doesn't hold, a plain "File not found", so the caller couldn't tell that its disk check hadn't finished. The read now takes such a pass as a copy disk couldn't be checked, with the pass's reason: the note says the read shows the workspace's copy and why, and a path the workspace doesn't hold answers that disk could not be checked. Tests, in test/workspace-three-way.test.ts beside the listing's: a read of a held file, and one of a file only disk has, each under a branch that switches at every disk read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review finding R1-3 on #465: on the page that ends the file, the note was null even when the page's last line was cut, so a page whose toLine equals totalLines read as complete while its last line held a cut marker. A program that checks toLine < totalLines took the file as read whole. The changelog said a cut last line's note names its rest, with no exception. That page's note now names the rest: "Line N is cut: read its rest with offsetChars=K." The first-line search measures the page with its note, so the clause stays within the result limit, and the overhead estimate already allows the longer note that holds the same clause. A page that ends the file on a line shown whole still carries no note of its own. Tests, in test/workspace-read-text.test.ts: a one-line file of 1,500 '"' read at a limit of 1,000, and a four-line file whose last line is 3,000 characters read at 24,000, each now naming the cut line's rest; and a page that ends the file on a short line after a cut one, with no note. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… and the changelog says how a script's line pages read Review finding R1-2 on #465: a read whose call carries no resultMaxChars was held to 24,000 characters. The framework sets resultMaxChars on a model's, a script's and the operator's puppet's call, so only a call made in process carries none, unless its caller sets one, and the framework stores no such result. The page's budget fits it to what its receiver keeps whole; a caller that states none takes the data itself. connectome-host's workspace file viewer and its headless file request read with limit 5000, no resultMaxChars, a byte cap of their own and no way to page, and a 3,780-line file came back only as far as line 543. Such a call now pages by its line limit alone, with every line whole, as before pages had a budget: linePage and characterPage take no maxChars for it. CRs and a BOM are left out of its line pages as of any other. ToolCall.resultMaxChars' doc says so for the workspace's read, while its edit and edit_many answers keep the 24,000 default. The changelog names in-process callers. Review finding R1-4: lines are cut at 2,000 characters whatever the call's limit, so a script's 5,000,000-character call gets cut lines too, and the changelog's sentence on scripts read as if their reads were unchanged. Every call that states a limit keeps one rule: its line pages are for reading, and a character page gives the stored text exactly, which each cut line's marker names. The changelog now says so for scripts. Tests, in test/workspace-read-text.test.ts: a call that carries no resultMaxChars reads 2,500 of 3,000 lines of a CRLF file, with its 3,000-character line whole and only the limit's and the CRLF notes, its default page is 2,000 lines, and its character page gives 100,000 characters whole. The tests of the cut line, which leaned on the old default, now carry a model's 24,000. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- grep's touching-window assertions compare the whole text, so a `--` before a section's first run fails them. - A new test greps three files with context: each file's run starts afresh, and a context-only line holding a lone CR is quoted. - shownValue's test asserts the exact quoted form of a value holding NEL, two U+2028s, a U+2029 and a VT, pinning each escape and its four digits. - editReceipt's doc says a receipt past its limit reaches its caller as any such result does; read-page.ts loses a double blank line. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ts, not the clock
Three tests in "a round whose run or wait call watches a script that can
change its routing" took their order from wall-clock margins, and a loaded
host could take the margin away:
- "a call made while a run watches, that waits for room in the host until
the run has yielded, counts toward the later wait's round" needed four:
its 16 slow calls timing out in the script 2 s after they were made, so
the skip_reply came then; the run yielding 3 s after
its child was spawned; the skip_reply's own 2 s outlasting the wait's
start; and the slow calls released 20 ms after the wait's tool:started. A
child started 1.5 s late fails it with { waiting: 0, hostCalls: 16 } at
the wait, as one of our full-suite runs did on a loaded host.
- "a send counts toward its round once the host has it, though its tool is
still running when the run yields" needed the send to reach its tool
within the run's 3 s yield; a child started 3.5 s late fails it.
- "a skip_reply a script makes while a later round's wait watches it keeps
that round's prose private" released its slow call 20 ms after the
wait's tool:started.
Three helpers in the describe give each test its order from events:
- yieldOnEvent makes a run's own yield come due once a condition holds.
For the one call to the run's wait, setTimeout passes through to the real
one and records the callback and handle of the timer the yield sets;
when the condition holds, the helper clears that timer and runs its
callback, so ScriptRun.wait and the manager's observe return through
their own code, as at a yield. Exactly one timer of the yield's length
must be set, or it throws, so the test fails loudly rather than falling
back to the clock if the runner ever yields another way. setTimeout is
restored in a finally, after that one synchronous call. Left to come
due, the runner's own timer still ends a run the code gets wrong, so
the test fails rather than hangs; the calls ask a 60 s yield, which no
event waits on, so the clock never decides an order. The helpers have no
bounds of their own, since a 15 s one would itself order the run on a
slow enough host.
- timeOutOnEvent does the same for the timeouts of a run's calls to one
tool: ScriptRun.call sets a call's timer before it first awaits, so the
wrap records the one timer of toolCallTimeoutMs each such call sets
(exactly one, or it throws), and firing it runs the runner's own
timeout. It fires exactly the count it's given, or throws: fewer would
leave the rest to the 59 s clock, which still comes before the 60 s
yield, so the test would pass on the clock's order.
- stepWatches wraps this framework's AgentScripts.observe, on the
instance, as the stop test wraps its stop: the nth call that watches a
script runs its step's onWatch once it watches (the manager sets the
watch just before it observes), and yields by its step's yieldWhen.
The three tests:
- The first test keeps its mechanism: a call waits for room in the host
only once the script's earlier calls have settled in it while their
tools still hold the host's places, since the script itself sends at most
16 calls at once. Its toolCallTimeoutMs is now 59 s, a length of its
own beside the 60 s yield, so each capture can take only its own timer,
and its 16 slow calls' timeouts come due once all 16 have reached their
tool. The run yields once the skip_reply waits for room (waiting 1,
hostCalls 16). The wait's onWatch records the counts, and that the
skip_reply hadn't reached its tool and no slow tool had returned, then
releases the slow calls; the wait returns once the script ends. The
skip_reply's own 59 s can't come due.
- The send test's run yields once the send has reached its tool.
- The skip_reply test's slow call goes once the wait watches, and it now
asserts that the skip_reply hadn't reached its tool when the wait began
to watch. Its run's 100 ms yield orders nothing: the slow call is held
until the wait watches.
- The release bounds these three tests had (15 s and 30 s timers letting
the held calls go) are gone; done() releases them once the turn is over.
At main 9a66f0a, with each script's child started late through
ScriptsConfig.execPath (a wrapper that sleeps, then execs node): the
describe passed 65 of 65 with no delay; 1.5 s late, only the first test
failed, with waiting 0; 3.5 s late, the send test failed too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A value the text shows as it is could itself be a JSON string: a line of a file reading "a\nb" (a quote, a, a backslash, n, b, a quote) read exactly as the quoted form of a, a line break and b. shownValue now quotes a value that starts with a quote as well as one that holds a line break, so a value shown as it is never starts with one, and one that does is always the quoted form, which gives back exactly what the value holds. The changelog says so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(code-execution): an MCPL peer's blocks too deep for JSON reach a script as an unconvertible result, not a failed dispatch
test(workspace): replace six absolute CPU-time bounds with deterministic checks
…oop, and check the send test's yield Answers the automated review of #779: - timeOutOnEvent's per-call guard threw from the wrapped ScriptRun.call, which ScriptRun calls as `void this.call(...)` inside its loop over a chunk's frames, so a throw left that loop and the frames after it were never handled. The guard now rejects the call's own promise, which the runner leaves unawaited: the test fails on the rejection, and the loop goes on. - The setTimeout capture both helpers wrote is now one helper, captureTimers(ms, fn), with fire() for clearing a captured timer and running its callback, so the pass-through, the match on the delay and the restore in a finally live in one place. - stepWatches' steps can take onReturn, which gets what the call returned. The send test's "the send was still running when the run yielded" check read the log after the turn, while the send was held until done(), so it couldn't fail; it now asserts that the run returned with its script still running. - The checks taken at the wait's watch (skipReplyReached, slowReturned, reachedBeforeWait) say in a comment that they're premise checks: they fail only if the slow tool's hold failed. - stepWatches' comment says when each kind of call starts watching: a run's watch is set when its script is made, a wait's just before it observes, and a stop observes only after its stop. yieldOnEvent's guard says the run had ended before its wait when it had, rather than blaming the yield. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
When a provider hold releases, the scheduler merges the wakes it kept beside the turn's retry into one start with it. That start can park behind an auxiliary call or be held again before it takes its turn token, and abortInference() or a quiesce abandon then dropped it whole, wakes included, so their messages went unanswered. A start that carries a turn on now records the batch it was merged from (MERGED_BATCH, a module-private symbol), each request as it was, until it takes its turn token: - a caller's stop or an abandon that drops it puts the batch's wakes back in the queue, where they run as a turn of their own (after the stop, or at resume); stop() and a disposal put nothing back. This reaches a budget restart's merged start too, whose abandon dropped its wakes; - a scheduler pass that takes it back puts the batch in its place, before its wait and quiesce checks, so each request is judged as itself and the host's shouldInfer decides on every wake; - under a quiesce the retry starts alone and the wakes park until resume, at the pass and at the auxiliary park's re-check, which sends a merged start back as its batch; - a hold's dedup keeps a batch on the request it keeps, and a withdrawal of a coalesced subject's wakes reaches batches. Also: a failed start's catch closes its check window before the error policy's retry starts, so a stop made once that retry is held ends the held turn rather than being taken by a window nothing collects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eError, and any other fault fails the dispatch #773 guarded the JSON round trip that gives an MCPL peer's structured blocks RFC-005's treatment at any depth on a script's call, so that blocks too deep for JSON.stringify reach the script as a result it can't convert, not as a failed dispatch. Its catch took anything the round trip threw. The blocks come from the wire as JSON, which has no BigInt and no cycles, and, by our reading, the treatment throws no RangeError of its own. So the round trip can legitimately fail in two ways, both RangeErrors (by run, the overflow is one on Node 24.22.1 and Bun 1.4.4): the stack overflow of a result too deep, and a result past the runtime's longest string. Both are results the script can't receive. A fault anywhere else, the treatment's included, was reported to the script as a result that "can't be converted to JSON", with the trace and the call's terminal reading the peer's success. Now the catch takes a RangeError only, and rethrows anything else, so such a fault fails the dispatch, as any fault did before #773. Either way the blocks don't reach the script, and the thrown error's own message is kept. Tests: a TypeError from the treatment (a getter that throws when a nested block's type is read) and one from JSON.stringify (a BigInt in a peer's block) each fail the dispatch, marked as one, traced tool:failed with the fault's own message, which the script's failure carries too. Wire JSON has neither, so they stand in for a fault. The depth test pins the host's log line, which unconvertibleScriptResult writes as the script's own failure text. Raised by the automated review of #780, #773's 0.26.x pick. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eaches a script as unconvertible
The round trip's catch takes any RangeError, the stack overflow and a
result past the runtime's longest string alike, and its comment names
both. Only the overflow had a test, so narrowing the check to the
overflow's message would have passed. A result past the longest string
can't be built in a test, so a toJSON that JSON.stringify calls and
that throws V8's RangeError ("Invalid string length") stands in for it.
The script gets that error's message as a result it can't convert, and
the trace reads the peer's success, with no failed dispatch marked.
Raised by the automated review of this pull request.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
feat(workspace)!: read, grep, edit and edit_many answer as the text the model reads
fix(scheduler): a turn a provider hold keeps for its retry is alive
…st-order test(code-execution): order the routing tests' runs and waits by events, not the clock
…ngeerror fix(code-execution): the script lane's round trip catches only a RangeError, and any other fault fails the dispatch
main gained #726 (F427, deferred writes found later), #723 (a write heard mid-turn starts no turn of its own) and more since this branch's merge of 5143b30. Two conflicts, both in the tool boundary's comments in src/framework.ts: - The flush's account of what it injects: this change's "One the render gives null goes in as stored", in place of main's left-out notice, with #726's account of a write that threw after it was stored: looked up by its id and carried where it landed, and the known limit, one neither read back nor looked up, carried at the boundary that finds it. - After the consumed mark: main's paragraph from #723 on a message the boundary stored and left out of the injection, which the notice told the agent of. This change leaves no message out of the injection (a null goes in as stored, and a tune-out message is skipped before the render, as before), so the paragraph has nothing to describe and goes. The null branch's comment still names a write found at an earlier flush as the exception, which is #726's known limit. No code conflicted.
…ts it released Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
fix(injection): a message the render gives null goes in as stored (#720) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
chore(changelog): add 0.26.2's section to main and remove the fragments it released
…nels meet the registry, tune-out and the reply anchor What main brings since e4d1e46, the sixth catch-up's base: 0.25.0 and 0.26.0, and #709, #710, #719, #722, #724, #728, #727, #743, #744, #755, #759, #761, #763, #767, #707, #740, #760, #726, #371, #452, #768, #465, #770, #771, #723, #777, #776, #773, #775, #778, #774, #779 and #781. Most of the conflicts are where #710 met the branch: in main a channel the framework acts on is a ChannelRef, { serverId, channelId }, since two MCPL servers can register one id. Conflicts in 14 files (61 hunks), resolved by keeping both sides' intent; where the two had different designs for one thing, the branch's stays, moved onto main's types: - src/framework.ts: main's conversationAgentHomes map stays gone. A fork's home is the registry record's homeChannel, which homeOf reads wherever main read the map (isForkBound, mayTakeGateRoute, canCloseChannels, holdsChannelTraffic, agentForPrimary, the drafts resend, the channel tools' guard, the registry's homeChannelResolver, stop()'s unbinding, releaseAgentRuntime's). The record still keeps a bare id, read as main reads a wake that names its channel without its server (channelNamed), or, where it names no one registered channel, as a home on no server ({ serverId: '' }), which a path reading the pair finds unregistered and one reading the name finds no channel or two for. main's dropContinuations goes into releaseAgentRuntime, so every way a live non-ephemeral agent ends drops a continuation of its turn, and beside the registry's end of an ephemeral run. Arrival carries main's channel ref and the branch's addressed; the addressed-elsewhere notice names channels with main's channelShown and keys its once-per-channel set by destinationKey. isAddressedMessage stays tag-only, with main's ambient marks. publishInputWithTurnLocus gives the locus with its server, and only to a call made inside the turn (callingTurnOf); outside it, a conversation fork's call takes its home, as its dispatched calls do, and anyone else's the registry's default. The tune_out handler keeps the branch's shape (serverId, scope, check, peek) and resolves through ChannelRegistry.resolveChannel, now resolveToolChannelEntry with the calling turn's own channels, check's peek included; its answers name the channel with nameShown. holdForProviderWait holds nothing for an agent a listener of the failure it reports ended (stopAgent), where main's report-before-hold (#774) meets the branch's stopAgent at a conversation agent's hold after its compile. handleSubconsciousTool gets main's own channels, then the branch's caller. - src/tune-out/coordinator.ts: main's shown() names beside the branch's reader helpers; deliver_summary, cancel_tuneout and speak_in_channel resolve as main's do, for the subconscious and a reader fork alike; cancel_tuneout keeps the branch's serverId and all-except fallback, and a name that picks no channel (one two servers share, or a turn's own channel ownChannelFor refuses) doesn't keep a call naming the scope from ending it; findByChannel goes, as on main. focusChannel() returns a ref. - src/mcpl/channel-registry.ts: markAccepted records no anchor (admission does), and handleIncoming's arrival count of removals, which main's guard there read, goes with it; noteDelivered takes the channel with its server and sets nothing for a channel its server no longer registers, read at admission where main's guard compared removal counts at arrival and acceptance (a message admitted after its server registers the channel again sets the anchor here). buildChannelContext is main's, per server, with the branch's sharedFallback; incomingFor reads the anchor itself, not the speech fallback. globalFallbackFor, sharedFallback, fallbackLocus and focusChannelResolver are on refs. - src/index.ts: main's DefaultErrorPolicy beside the branch's exports. - Tests: the branch's registry stubs take main's shapes; main's appendInTwoParts in batched-wake-locus's rig. Beyond the conflicts: the all-except announcement goes to the excepted channel on its server (publishForAgent refuses a shared bare id since #710), and tune_out's all-except answer names its channel with nameShown; the tune_out schema's serverId description says channelId takes serverId:channelId too; main's refusals for a tuned-out channel no name picks alone give the tune_out cancel by id and server, since this branch's tune_out takes a serverId; a stale comment naming conversationAgentHomes in channel-registry.ts names the registry; docs/dendrite.md says where a publish naming no channel goes outside the turn, and no longer lists ChannelRef as unbuilt. Test adaptations: main's tests that set conversationAgentHomes give the fork its home through the registry. main's tests written to the latest-inbound rule run under it (reply-server-routes: speech LEGACY; shared-channel-id's registry fixture: latestInboundLocus, and noteDelivered as admission records). Two reply-server-routes tests of an in-turn publish naming no channel now check the branch's rule, the turn's locus through its server. tune_out's shared-channel-id tests wait for the handler's answer, and its shared-id refusal offers serverId. all-except- unit's stub registry has nameShown; present-while-acting and deferred-window-only build routing state in main's shapes. main's marker and removed-server anchor tests record the anchor at admission (noteDelivered), where this branch records it. At this commit alone, the tests where a fork's home has to name its server fail (two servers' channels with one id, or a home the registry doesn't register at that moment); the pull request lists them. The next commit stores the home with its server. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The merge before this keeps a conversation fork's home as the registry's
bare channel id, which can't say which server's channel it is. Where two
MCPL servers register one id, or none does at the moment of use, the fork
is left with a home on no server, and where one other server registers
the id once the home's own withdraws it, the bare id names that server's
channel. main's tests of these cases fail there.
AgentSpec.homeChannel is now { serverId, channelId }, and
conversationForkSpec takes it as homeChannel in place of channelId;
createConversationAgent gives it the router's ref. AgentRecord.homeChannel
is that or a string: a record written before this commit, read as before
(channelNamed, else a home on no server). Such a record is an ended one,
since a conversation fork ends at a restart. The registry snapshot stays
version 1, so code from before this commit can't read the homes it
writes: it finds an object where it expects a string. Going back past
this commit would need those records rewritten, or a version bump that
refuses them.
The derivation notice names the home as the agent is shown it, and a
fork's injections are scoped by its home's id. Tests give homes their
server; a registry test restores a snapshot whose fork kept a bare id,
and channelOfHome's test reads a home with its server as that channel.
This changes the branch's durable format, so it's its own commit, to be
taken, changed or replaced apart from the merge.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
One conflict, in the tool boundary's drain loop (src/framework.ts): #720 dropped `parts` from what the drain hands back and from `heard`, with the left-out notice that used it; the branch's side keeps its two lines there, releaseLandedMail for each stored message and the skip of a message held from the residents (heldFromResidents), which #720's side has as the older tuneOut check. The resolution is the branch's lines without `parts`. Everything else merged without conflict. #720's new present-while-acting test of a re-pin to an addressed message the render gives null runs under the legacy speech rule (speech.locus 'latest-inbound'), as the file's other re-pin tests do on this branch, since the branch's default locus doesn't re-pin. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…dent's turn While the resident's turn runs, every held message waits in the deferred queue, not the store. Reader forks were handed stored messages only, by a sequence cursor, so a fork derived then saw nothing, not even the addressed message that woke it, though a reader is most useful exactly while the resident is busy (#546, probes R2 and R20b). - A reader invocation carries the epoch's queued messages (`queued`, each with its deferred write id) after the stored ones, on a wake, a cadence look and a cancel. The coordinator records the write ids it hands out; once such a write lands, its row (`metadata.deferredWriteId`) is not handed again, so each held message still goes to exactly one reader. - The cancel's fork is handed what is still queued, and its notice no longer says those messages may not be in what it sees. - A reader's own cancel, and its own release of the all-except hold, are followed by one more fork only for messages no reader was handed. - The fork branch resets the cadence count, and a cadence period with nothing new derives no fork (#546, probe R13). - `defaultReaderFraming` shows queued messages after the stored ones, under a note that they arrived while the resident's turn was running. - The `tune-out:reader-fork` trace counts `queued` within `held`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
There was a problem hiding this comment.
Automated review
Recommendation: merge and fast-follow. 4 medium severity findings can be fixed after the merge.
Reviewed commit 7b11e5c with claude-opus-5-5, by reading the code. It did not run the code or look at the checks.
Does this change belong in the project?
It fits the project. This fixes an unreleased Dendrite feature, tune-out reader forks: they are now also handed held messages that are still waiting in the deferred queue. The coordinator still holds the backlog and passes it to forks as framing, and each held message still goes to exactly one fork, as the design notes require.
5 reasons with sources
- It makes reader forks do what the reader-forks feature says they do: hand on the traffic held since the last look. The pending fragment says each cadence tick, wake and cancel 'hands it the traffic held since the last look'. While the resident's turn runs, held messages sit in the deferred queue rather than the store, so forks were handed nothing. The PR adds those queued messages to the fork's framing. (changelog.d/dendrite-reader-forks.added.md; docs/dendrite.md, 'What the fork is handed')
- It keeps the design rule that a reader fork gets held traffic as framing, one message to one fork, with the coordinator holding the backlog. Queued messages are added to the coordinator's framing context (
queued), and the coordinator records which write ids it has handed out (readerHandedWrites) so a message isn't handed twice. Nothing is moved into the context manager or into a per-fork view. (docs/dendrite.md, design decision 10) - The code stays in the parts that already own this work: the tune-out coordinator, the reader framing in tune-out/tools.ts, and the framework's existing getDeferredMessages hook.
ReaderFramingContext.queuedis optional, so custom framings written against the documentedframing(context)interface keep compiling. The changed part of the public surface is small. (src/tune-out/coordinator.ts, src/tune-out/tools.ts) - Stacking on #790 is declared, and amending the pending fragment instead of adding a new one follows the changelog rules. CONTRIBUTING allows stacked PRs as long as the body says so, and this one does. The huge diff is mostly #790's commits plus earlier work; the PR's own change is one commit. The CI changelog check accepts a PR that 'edits a pending one'. (CONTRIBUTING.md, 'How changes land' and 'Changelog')
- The design doc's 'What the fork is handed' paragraph still says forks get stored messages chosen by cursor; that wording should be updated with the change. This is already one of the code review's findings. It is a documentation gap, not a conflict with the design: the change keeps design decision 10. (docs/dendrite.md:238)
Findings
4 findings of medium or high severity, see the inline comments:
- Medium severity. A custom framing that ignores
queuedsilently loses queued held messages for good. - Medium severity. Queued writes are marked handed before the fork exists, so a failed derive loses them.
- Medium severity. All-except release keys the 'already handed' check by the first message's server only.
- Medium severity, test coverage. Framework side of the change (writeId pass-through, held handles, trace counts) has no CI test.
5 more findings, less severe
- Low severity, documentation,
docs/dendrite.md:238-246. Dendrite design doc still describes reader forks as handed stored messages by cursor only. The 'What the fork is handed' paragraph says the coordinator 'keeps a per-channel cursor over the diverted backlog'. It lists the framing context's fields withoutqueued, and says the framing row carries 'the held message ids'. Under this change it also carriesheldHandle(writeId)handles for queued messages. The Trace paragraph (line 246) doesn't mention the newqueuedcount ontune-out:reader-fork. Readers of the design doc will not learn about the queued hand-off or the write-id dedup. - Low severity, test coverage,
src/tune-out/coordinator.ts:1368-1371. No test for a queued message without a write id being left for later.queuedHelddeliberately drops deferred entries with no stringwriteId, so they reach a reader only after they are stored. No test covers this branch, or the non-reader all-except release with queued messages (only the reader's own release is tested). A regression in the filter, for example handing id-less entries and then re-handing their stored rows, would go unnoticed. - Low severity, maintainability,
src/tune-out/coordinator.ts:265-271. readerHandedWrites keeps every handed id until the epoch ends. Ids are added for every queued message handed to a fork, but are only removed when the epoch is cancelled or released. They stay even after the write has landed and moved behind the cursor, where the id can no longer matter. A long all-except hold over busy channels grows one set per channel without bound. Ids whose stored row sequence is at or below the cursor could be pruned inunhandedBacklog. - Low severity, maintainability,
src/index.ts:198-204. Public ReaderFramingContext refers to QueuedHeldMessage, which isn't exported.ReaderFramingContextis exported from the package root, and its newqueuedfield is typedQueuedHeldMessage[].QueuedHeldMessage(andReaderInvocation) are not re-exported from src/index.ts. Hosts writing a custom framing must writeNonNullable<ReaderFramingContext['queued']>[number]to name the element type. Export it next to the other tune-out types. - Low severity, documentation,
changelog.d/dendrite-reader-forks.added.md:18-22. Changelog omits that a reader's own all-except release now derives forks. The fragment describes the new fork after a reader's own per-channel cancel. It does not mention that a reader's own release of the all-except hold now derives a cancel fork for each channel with messages no reader was handed. Before this change such a release derived none (basecancelAllExceptrequiredsource !== 'subconscious'). This change in behavior, and its cost, belongs in the entry.
…s messages back, channels by server Review of #793. - Queued messages are rows of `ReaderFramingContext.backlog`, last, with held handles for ids and provisional sequences (`queuedCount` says how many), so a custom framing that shows `backlog` shows them too. Before, they came in a separate optional field, and a framing that ignored it lost them for good once they were marked handed. - A fork whose framing, derive or run throws before it reports gives its held messages back (`readerFailed`) while the epoch still holds the channel: the next invocation for the channel is handed them, whether stored, still queued, or landed behind the cursor. New trace `tune-out:reader-fork-failed` with `reported` and `returned`. The cursor now only moves forward. - A channel is its server and id for the readers' bookkeeping: a wake matches held messages by both, and the all-except release derives its forks per server and channel, not per channel id with the first message's server. - A handed write id is dropped once its row lands behind the cursor. - New test/tune-out-reader-fork-handoff.test.ts runs the framework side on a real deferred queue with only the fork stubbed (no Dendrite builds): held handles in `heldMessageIds`, the trace counts, landed rows carrying `deferredWriteId`, and a failed derive's message reaching the next fork. Unit tests for a failed fork, an ended epoch, a queued message without a write id, the resident's all-except release, shared channel ids and the pruning. - docs/dendrite.md and the changelog fragment describe the hand-off. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…l is held `deliver_summary` marked the fork as having reported before it called `deliverFromReader`. When holding the mail threw, the fork got an error result, answered in prose and ended, and the framework skipped its prose as a second report, so neither reached the resident (#546, probe R14). The fork is now marked after the mail is held. A summary whose mail fails leaves the fork's prose as its report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru
pushed a commit
to nissa-seru/agent-framework
that referenced
this pull request
Oct 24, 2026
…sident's turn runs (anima-research#793) Two ambient messages and an addressed one arrive with the resident idle (a) or mid-turn (b), where they wait in the deferred queue. Logs which of them each of the subconscious's requests holds, at the wake, as the resident's turn ends, and at its next wake. Run at agent-framework#793's heads 7b11e5c and f7a8577 with chronicle d553ce9 and context-manager fea7a940 linked in. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
|
@antra-tess, we reran our reader probes at this PR's heads on the same linked setup as before (chronicle
Also seen, and not looked into: in both arms of P1, each wake gave the subconscious three requests. The probes are on Written by Hilda-2681 (Claude Opus 5.5). — Lottie-2661 |
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
@tel-0s, this fixes the three reader-fork findings in the review comment on #546 (probes R13, R2, R20b and R14).
While the resident's turn runs, every held message waits in the deferred queue rather than in the store. Reader forks were handed stored messages only, chosen by a sequence cursor, so a fork derived while the resident was busy was handed nothing. In R20b, the wake fork derived for an addressed message wasn't handed that message. That is the moment a reader is most useful: the resident is busy and can't look, and the reader is the part that can notice someone asking it to stop. At the cancel, the queued messages then landed view-filtered, where no reader read them either.
Stacked on #790. This branch is #790's head (
219bda53) plus three commits:7b11e5c8, the fix;f7a85773, the automated review's findings;848b6893, R14.Until #790 merges, the diff below includes #790's commits too. After it merges, the diff is these three commits.
Changes
queued: the epoch's held messages still in the deferred queue, each with its deferred write's id, after the storedbacklog. That applies on a wake, a cadence look and a cancel. The framework'sgetDeferredMessageshook passes the write id through.readerHandedWrites, cleared with the cursor). When a handed write lands, its stored row carries the same id asmetadata.deferredWriteIdand isn't handed again, even though its sequence is past the cursor. An id is dropped once its row is behind the cursor.readerFailedreturns its held messages while the epoch still holds the channel. The next invocation is handed them, whether they are stored, still queued, or landed behind the cursor. The cursor only moves forward. New trace:tune-out:reader-fork-failed(reported,returned).deliver_summarymarks the fork as having reported only after its mail is held. When holding the mail throws, the fork gets the error as its tool result, and the prose it ends with is delivered as its report. Before, the fork was marked first, and neither the summary nor the prose reached the resident.invokeSubconsciousresets the diverted count, as the persistent path does, and a cadence period with nothing new derives no fork.ReaderFramingContext.backlog, after the stored ones. Their ids are held handles (heldHandle(writeId), as module rows name them), and their sequences are provisional.queuedCountsays how many there are. So a custom framing that showsbacklogshows them too.defaultReaderFramingputs them under a note that they arrived while the resident's turn was running and aren't stored yet.tune-out:reader-forkcountsqueuedwithinheld, and the framing row'sheldMessageIdsare the backlog's ids.changelog.d/dendrite-reader-forks.added.mdrather than adding a fragment.Not in this PR:
Testing
New
test/tune-out-reader-queued.test.ts, 16 tests. They run the coordinator against a fake registry and fake hooks, so they run in CI without the Dendrite builds of chronicle and context-manager. They cover:On
219bda53, the first commit's 8 fail on their assertions. On7b11e5c8, the 6 for the second commit's fixes fail.New
test/tune-out-reader-fork-handoff.test.ts. It runs the framework side on a real deferred queue, with the MCPL fixture and a resident turn waiting on a tool. Only the fork is stubbed (deriveAgent,runEphemeralToCompletion), so it runs in CI. It checks:heldMessageIdsare held handles of the real write ids;deferredWriteId;It fails on
7b11e5c8.New
test/tune-out-reader-report.test.ts, 2 tests, for R14. The failing-mail case fails without848b6893.Hilda's probes, run at this head with chronicle
d553ce9and context-managerfea7a940built and linked in:Tune-out and Dendrite files on the same linked setup, 24 files at
f7a85773: 392/392 pass, includingtune-out-reader-forks-e2e, which runs there instead of skipping.Full suite on that setup at
7b11e5c8, Node 22.23.4: 6960 of 7001 pass. The failures don't involve this change:--test-concurrency=2;request-compile-statsfails the same way at219bda53on this setup;🤖 Generated with Claude Code