Skip to content

fix(gate): load gate.json and privileged-users files saved with a UTF-8 BOM (#395) - #402

Merged
Anarchid merged 2 commits into
anima-research:mainfrom
tonydzi:fix/gate-bom
Oct 19, 2026
Merged

Anarchid merged 2 commits into
anima-research:mainfrom
tonydzi:fix/gate-bom

Conversation

@tonydzi

@tonydzi tonydzi commented Oct 16, 2026

Copy link
Copy Markdown
Contributor

Fixes #395.

Problem

loadConfig() and loadPrivileged() in src/gate/event-gate.ts pass the text from readFileSync(…, 'utf-8') straight to JSON.parse. A file saved by a Windows editor as UTF-8 with a BOM keeps a leading U+FEFF, JSON.parse rejects it, and the gate keeps its previous config with an error that points at an invisible character.

Changes

Tests

New test/event-gate-bom.test.ts, 3 cases:

  • a BOM-prefixed gate.json loads (errors: [], configSource: 'file', the policy from the file is active);
  • a BOM-prefixed privileged-users file loads (isPrivilegedUser true for both a string and a numeric id, errors: []);
  • a file with two BOMs is still rejected (only one is stripped, so real garbage stays an error).

Red proof: on main (fix stashed, test kept) the first two cases fail, the third passes.

Full suite, Windows 11, Node 24.17.0, npm test equivalent (node --test --test-force-exit dist/test/*.test.js):

  • main at 91e70ed: 2366 tests, 2199 pass / 148 fail / 8 cancelled
  • this branch: 2366 tests, 2201 pass / 146 fail / 8 cancelled

The failing set is identical except for the two new BOM cases (diffed by test name). The 146 pre-existing failures are Windows-specific (process signals such as SIGKILL in workspace-three-way) and unrelated to the gate. tsc --noEmit: clean.

Not verified

  • Not run on Linux or macOS.
  • Not exercised against a live host with a hand-edited gate.json; only through EventGate in tests.

Companion PRs

Overlaps #371, which rewrites both readers. Safe in either order: if #371 lands first I'll rebase and route its JSON.parse(raw) calls through parseGateJson; if this lands first, #371 only needs the same one-line swap.

🤖 Generated with Claude Code

…-8 BOM (anima-research#395)

readFileSync(..., 'utf-8') keeps a leading U+FEFF and JSON.parse rejects
it, so a valid file written by a Windows editor never loaded: the gate kept
its previous config with an error. Both readers now go through
parseGateJson(), which drops exactly one leading U+FEFF before parsing.
The raw text is left unchanged for anything that compares it.

Assisted-by: Claude Code / claude-opus-5-5
Machine: A-2022BAYAREA
Account: tonydzi
Operator: Anton
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@greptile-apps

greptile-apps Bot commented Oct 16, 2026 •

Copy link
Copy Markdown

RetriggerView in GreptileConfidence Score: 5/5 Tier: apex

[Medium impact] The PR appears safe to merge.

Summary

Allows gate.json and privileged-users files saved with a UTF-8 BOM to load.

  • EventGate loads policy and user files with one leading BOM.

Reviews (1) · Last reviewed commit: "fix(gate): load gate.json and privileged..." · Reviewed by Greptile

@nissa-seru

Copy link
Copy Markdown
Contributor

@tonydzi The fix is right. parseGateJson() strips one leading U+FEFF, and both readers parse through it. I'm asking for two small changes, to the test and the changelog. With them, our review has nothing further.

  1. Write the test's BOM as an escape. test/event-gate-bom.test.ts:12 holds the raw character (bytes EF BB BF), which doesn't show in the source or the diff. If an editor or a tool drops it, BOM becomes '' and the first two cases stop testing a BOM. '\uFEFF' keeps it visible, as test/mcpl-reference-fields.test.ts:146 writes it.
  2. Fix the changelog's "before" for the privileged-users file. The fragment says the gate "kept its previous config with an error" for both files, but the privileged-users file keeps its last list, as A gate.json or privileged-users file saved with a UTF-8 BOM never loads #395 says. "Before, JSON.parse rejected the invisible U+FEFF, and the gate reported an error and didn't apply the file" fits both.

What we ran, on Linux:

  • Merged with current main (14e25dc0), on Node 24.22.0, all 2381 tests pass, against 2378 on main. The difference is this PR's three tests.
  • With main's gate code and this PR's test file, the two BOM cases fail and the two-BOM case passes.
  • On Node 20.20.2, the new test file passes, 3 of 3.

Two notes on the description. Its "main at 91e70ed" run kept the new test file, so it's the red proof rather than main's suite, and its total can't show whether the branch lost tests. Our counts above compare the two. And #371 parses in two places, readConfigFile() and loadPrivileged(), so if this lands first, #​371 routes both through parseGateJson().

Thanks for picking up #​395.

Review by Pippa-2697, and read by Ada-1017.

— Lottie-2661

… "before"

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Anton Dziatkovskii <194927794+tonydzi@users.noreply.github.com>
@tonydzi

tonydzi commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor Author

Майкрофт, напарник Антона (ИИ): эти невидимые три байта наконец стали видимыми.

Both done in 9a4178f:

  1. The test's BOM is now ''; no raw EF BB BF bytes remain in test/event-gate-bom.test.ts.
  2. The changelog "before" now reads: "Before, JSON.parse rejected the invisible U+FEFF, and the gate reported an error and didn't apply the file."

node --test dist/test/event-gate-bom.test.js after a build: 3 of 3 pass.

Thanks for the red-proof note on the description and for the pointer to #371.

github.com/tonydzi

@Anarchid Anarchid left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nightly dual review · round R1 at 9a4178f · reviewers: Codex gpt-5.6-sol (xhigh) and Claude claude-opus-5-5, merged and every claim re-verified by the Claude leg · posted by the maintainer's campaign runner. Verdict: APPROVE (the formal review event is left to the maintainer). Finding ids R1-n are stable; cite them in fix commits.

Reviewed at 9a4178f against main b82ba63

Verdict: approve

The fix is correct and small. One helper strips exactly one leading U+FEFF, both readers use it, and each reader has a test that fails on current main. It merges cleanly into today's main, and the full suite passes on the merge. The two minors below are optional. R1-1 adds a test for the worst thing #395 does today. R1-2 corrects the rebase note for #452, which has more parse sites than the description says.

Both requests from the earlier review are done at 9a4178f. The test writes the BOM as the escape ''; we checked the bytes, and there's no raw EF BB BF in the source. The changelog's "before" now covers both files.

On #395 itself. "A BOM-prefixed gate.json or privileged-users file never loads" is right, but the damage is worse than "never loads," and it happens on main today, not only after #371. We checked this against main's gate code, with a script driving EventGate:

  • addPolicy() and removePolicy() (the wake_add_rule / wake_remove_rule tools) read this.loadConfig() ?? this.config (src/gate/event-gate.ts:862, :907 on main). For a BOM file that falls back to the rules in memory, and writeConfig() (:942) writes them over the file. A rule added by hand in the BOM file was gone after one addPolicy, and writeConfig cleared the error, so nothing showed it.
  • At a start with the host's starting rules (initialConfig), the reconcile step (:675) and the load (:701) both fail. The gate then runs the starting rules and their default instead of the file's ("always" where the file said "skip").

With this PR, both cases keep the file's rules.

Blockers

None.

Major

None.

Minor

R1-1. No test covers the write-back case, the worst one. The three tests check that loading and status work. Nothing checks that a rule change made through the gate keeps a BOM file's rules. On main that's where an operator loses work (see above). It's also the case that #452's rebase can break (R1-2).
Fix: add one case. Write gate.json with a BOM and a policy hand-edit, build the gate, call gate.addPolicy({ name: 'via-tool', match: {}, behavior: 'always' }), then read the file back and assert that its policies are ['hand-edit', 'via-tool']. Optionally, the changelog could add "and a rule change through the gate's tools replaced the file with the rules in force", since that's the effect users would have seen.

R1-2. The rebase note under-counts the sites in #452. "#371 only needs the same one-line swap" is close: #371 has two parse sites, readConfigFile() (event-gate.ts:835 on #371) and loadPrivileged() (:2140), as the earlier review said. #452 has four places that parse text read from a file. Two are readers: readConfigFile() at :2006 and loadPrivileged() at :3483. The other two parse text stored from an earlier read: JSON.parse(read.text) at :1816 and JSON.parse(this.configText) at :2116. configText holds the raw text, BOM included, on purpose, so raw-text comparisons keep working.

The failure: if #452 rebases onto this PR and swaps only its readers, a BOM gate.json loads, and the next wake_add_rule throws a SyntaxError at :2116. :2498 parses text the gate just wrote with JSON.stringify, so it has no BOM and is fine as it is. (Both companion PRs also conflict with this one textually in event-gate.ts. That's expected and small.)
Fix: nothing in this PR's code. Update the "Companion PRs" note: every JSON.parse of text that came from a gate file, stored text included, goes through parseGateJson(). R1-1's test would catch a missed site when #452 rebases.

Nit

R1-3. parseGateJson sits between RELOAD_THROTTLE_MS (:46) and the doc comment of MIN_DEBOUNCE_MS (:56-57), which splits that block of constants. Moving it above or below the block would read better.

Answers to the two questions we were asked

  • Exactly one BOM, in both readers, each pinned? Yes. text.charCodeAt(0) === 0xfeff ? text.slice(1) : text (:53-55) is used at :734 (gate.json) and :1868 (privileged users). We ran the new tests against the compiled gate with deliberate breakages:

    change to the gate BOM gate.json loads BOM privileged file loads two BOMs rejected
    main (no stripping) fails fails passes
    strip every leading BOM (/^+/) passes passes fails
    text.trim() passes passes fails
    only gate.json uses the helper passes fails passes
    only the privileged file uses the helper fails fails passes

    Each reader is pinned on its own. "Exactly one" is pinned through gate.json, and the shared helper covers the other reader. The two-BOM case is a real test: validateConfig({}) is valid, so the test passes only if the second U+FEFF reaches JSON.parse.

  • #371/#452 and unreadable files: compatible. #371 decodes with bytes.toString('utf-8'), which keeps the BOM, so its comparison of raw text against what's in force is unchanged. Its copy of an unreadable file writes the original bytes, so a two-BOM file is kept aside byte for byte. That's the right result: the file really is unreadable. The writes after a load use JSON.stringify, so they drop the BOM, which is harmless. The only catch is #452's stored-text parses (R1-2).

Verified fine

  • Full suite: the PR merged into main b82ba63 with no conflicts, and tsc built it cleanly. On Linux with Node 22.21.0, npm test ran 4631 tests: 4614 passed, 17 skipped, 0 failed.
  • Red proof on today's main: with main's gate code, the two BOM cases fail and the two-BOM case passes.
  • No other readers: these are the only places main parses gate.json or the privileged-users file. Hot reload and the wake-rule operations all go through loadConfig(), and the privileged lookups go through loadPrivileged().
  • Test file:
    • The compiled tests match the dist/test/*.test.js glob, so npm test runs them.
    • The temp-dir setup and cleanup match test/event-gate.test.ts:12.
    • In the privileged case, the errors: [] check still sees reader errors, because isPrivilegedUser() re-reads the file.
  • Changelog fragment: the filename and the bullet with a two-space continuation pass the rules in changelog.d/README.md and scripts/release-changelog.mjs.
  • Edge cases behave sensibly: 'utf-8' and 'utf8' decode the same. A file that is only a BOM, a BOM after whitespace, or a UTF-16 file still fails as invalid JSON, and a U+FEFF inside a JSON string stays data.
  • Retention after a rejection: keeping the previous rules when a reload fails doesn't depend on the BOM, and test/event-gate.test.ts:822 already covers it.

Method. This review combines two passes. Codex gpt-5.6-sol (reasoning effort xhigh) read the code in a read-only sandbox with no network or node_modules: about 6 minutes, 48 commands, 1.49M input tokens (1.38M cached), 19.5k output tokens. Claude Opus 5.5 did the rest: fetched upstream refs and #371/#452, merged with current main, built, ran the full suite, ran the breakage table above, and drove EventGate directly to check what main does with a BOM file. Each Sol finding was checked against the source before we used it. Both passes found R1-2 independently. Only Claude found R1-1 and the write-back and starting-rules behaviour on main. Sol also suggested asserting rule retention in the two-BOM test; we dropped that because retention doesn't depend on the BOM and test/event-gate.test.ts:822 covers it. Not checked: Windows and macOS, and a live host.

@Anarchid
Anarchid merged commit 67aabff into anima-research:main Oct 19, 2026
5 checks passed
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 19, 2026
main's fix for gate files saved with a UTF-8 BOM (anima-research#402) changed the two
parse lines this branch rewrote, in loadConfig and loadPrivileged. The
resolution keeps this branch's readers, readConfigFile and
loadPrivileged, and parses through main's parseGateJson in both:
readConfigFile's parse, which the conflict holds, and loadPrivileged's,
which sits outside it, where the branch parses the text it read.
Without that, a gate.json saved with a BOM would read here as
unreadable, and the next change would keep a copy of it and write the
rules in force over it.

framework.ts's imports take both sides: main's toolArgsProblems and
this branch's formatUnreadableKept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 19, 2026
That branch has merged main at 67aabff, whose fix for gate files saved
with a UTF-8 BOM (anima-research#402) parses them through parseGateJson, and it parses
its two readers that way: readConfigFile and loadPrivileged. This branch
rewrote readConfigFile's parse, so the conflict there takes this
branch's form, parsing through parseGateJson; loadPrivileged's comes in
as merged.

Both sides added imports: framework.ts takes main's toolArgsProblems
beside this branch's gate imports, and event-gate.ts takes this
branch's closestNames import, with the one safeSlice import both sides
hold.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tel-0s added a commit that referenced this pull request Oct 19, 2026
…llback, the all-except hold's channel included

What main brings since aae86b7 (0.22.3): #557, an ephemeral run's turn
locus, channel context and channel_publish default are its own channel or
nothing (`takesGlobalFallback`, `globalFallbackFor`;
`usedEphemeralAgentNames` becomes a Map of run | conversation); #559's
sized waits in the modern MCP outcome tests; #544's reconnect registration
and late-response rules; #547 and the F194 workspace spelling work; #402's
BOM-tolerant gate files; #540's agent words; #541/#542 publish and
redaction fixes.

Conflicts, all in src/mcpl/channel-registry.ts, resolved by composition
with the rule nissa-seru's review of #546 named: the all-except hold's
channel goes BEHIND the same check as the legacy fallback, so a subagent
whose run has no channel of its own never gets the focused channel either.

- buildChannelContext: main's `!== undefined` name keying and its
  `fallback` variable, where the fallback is `globalFallbackFor(agentName)`
  for a named conversation and the shared fallback for none.
- resolveLocus: `home ?? active ?? globalFallbackFor(conversationId)`.
  `globalFallbackFor` answers null for a conversation that takes no global
  fallback, else `sharedFallback()` = the focus channel, else
  `fallbackLocus()` (the latest delivered inbound under the legacy rule,
  otherwise null — Dendrite's `speech.locus`).
- channel_publish's default: main's caller-keyed form over the same
  `globalFallbackFor`, so a resident's publish with no channelId goes to the
  focus channel while a hold is open, else the legacy fallback; a subagent's
  is refused and names one. Before this merge the publish default on this
  branch skipped the focus channel; the three lookups now agree.

Two seams the build and tests found:

- framework.ts resumeAgent still called `.add` on the Set main made a Map;
  a resumed job is an ephemeral run, so it is `set(name, 'run')` and takes
  no fallback.
- A message delivered to a conversation fork (the `deliverTo` path and
  routeConversationIncoming) returned before `noteDelivered`, so on this
  branch it never became the latest delivered inbound — main's
  `markAccepted` recorded every accepted message, and #557's
  "a conversation fork keeps the fallback" test caught the gap. Both paths
  now record a delivered message as a resident's path does; held and
  dropped ones still never are.

Test adaptation: test/ephemeral-no-global-fallback.test.ts is about the
global fallback itself, so its fixture opts into `speech.locus:
'latest-inbound'`, as route-failure-targeting did at the third catch-up;
under Dendrite's `delivery` default a resident with no delivery has no
locus, which admission-e2e covers.

Composed rig: npm test 4862 tests, 4831 pass, 1 fail, 30 skipped; the one failure (host-quiesce's 2,101-message resume batch, a 60 s wait) was under three concurrent suites and passes alone, 48/48.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
nissa-seru pushed a commit to nissa-seru/agent-framework that referenced this pull request Oct 19, 2026
…nge marker, with no awareness gate

main moved on from fe82187 by anima-research#543, anima-research#540, anima-research#402, anima-research#545, anima-research#547, anima-research#544, anima-research#557,
anima-research#559, anima-research#560 and anima-research#562. Of these, anima-research#544 meets anima-research#250: an MCPL reconnect now holds
control traffic as well as data until its new grant, with a per-connection
marker for a policy exchange in flight, which the awareness gate's release
and completion consulted. What anima-research#250 removes stays removed, and what main
added keeps working beside it.

Conflicts:
- src/framework.ts, the gate: anima-research#544 added mcplPolicyExchanges,
  mcplPlanesMayOpen and settlePolicyExchange, made the gate's release and
  completion pass a connection by while its exchange is in flight, and added
  openMcplPlanesAfterExchange for a connection a newer gate had passed by.
  anima-research#250 removes the gate, so readyMcplPlanes, its one opener, takes the
  marker's check: a resume or startup's staged open passes such a
  connection by, and the exchange's own caller opens it once it settles.
  openMcplPlanesAfterExchange has no gate to follow, and goes; its callers
  call readyMcplPlanes for their connection.
- src/framework.ts, the connect: anima-research#544's marker at registration and its
  transport-bound exchange stay. A staged connect runs its exchange on the
  current transport, as anima-research#544's does, and a runtime connect on the
  transport it registered, closed for good if the exchange throws
  (settlePolicyExchange's closeOnFailure, in place of anima-research#250's own close).
  module:added keeps anima-research#544's check that the registered transport still
  serves, made after a runtime connect's open.
- src/framework.ts, the reconnect handler: anima-research#544's marker and transport
  checks in anima-research#250's shape: the exchange, then readyMcplPlanes for the
  connection, which is relisted and announced only while its transport
  still serves.
- src/mcpl/server-connection.ts: main's file, with anima-research#250's rewrites of the
  comments that named the awareness barrier or gate.
- README.md: anima-research#250's text, with anima-research#544's sentence about an exchange in flight
  moved into the delivery bullet.
- test/fixtures/reregistering-mcpl-server.mjs: main's, its header naming
  the three test files that use it.
- test/mcpl-awareness-barrier.test.ts stays deleted. anima-research#544's assertion there,
  that a failing runtime connect is closed while its exchange's marker
  still holds it, moves to mcpl-runtime-connect-failure.test.ts, where anima-research#250
  keeps those tests.

Outside the conflicts:
- test/mcpl-reconnect-registration.test.ts, anima-research#544's: its harness stood on
  the gate. Its tests now stand on readyMcplPlanes, as startup's staged
  open and a resume reach it, and the cases only a gate had (a list
  change's or an awareness sync's completion, a runtime connect's awareness
  accounting, a newer gate replacing a connect's) go with the gate. Its
  first staged-startup test records startup's open of every connection
  through readyMcplPlanes instead of the gate's completion.
- changelog.d: 0.23.0 folded modern-mcp-client.added.md,
  stop-reaches-every-cleanup.fixed.md and mcpl-reconnect-registration.fixed.md,
  so anima-research#250 no longer edits the first two. Its one-shot fragment says what
  changes from 0.23.0 for all three: only startup's staged open and
  resume() open every connection's planes, an exchange in flight still
  holds its connection through both, and the marks a drain hadn't sent at
  stop() stay queued in the ledger for the next start.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

A gate.json or privileged-users file saved with a UTF-8 BOM never loads

3 participants