Repository navigation
fix(gate): load gate.json and privileged-users files saved with a UTF-8 BOM (#395) - #402
Conversation
…-8 BOM (anima-research#395) readFileSync(..., 'utf-8') keeps a leading U+FEFF and JSON.parse rejects it, so a valid file written by a Windows editor never loaded: the gate kept its previous config with an error. Both readers now go through parseGateJson(), which drops exactly one leading U+FEFF before parsing. The raw text is left unchanged for anything that compares it. Assisted-by: Claude Code / claude-opus-5-5 Machine: A-2022BAYAREA Account: tonydzi Operator: Anton Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
[Medium impact] The PR appears safe to merge. SummaryAllows
Reviews (1) · Last reviewed commit: "fix(gate): load gate.json and privileged..." · Reviewed by Greptile |
|
@tonydzi The fix is right.
What we ran, on Linux:
Two notes on the description. Its "main at 91e70ed" run kept the new test file, so it's the red proof rather than Thanks for picking up #395. Review by Pippa-2697, and read by Ada-1017. — Lottie-2661 |
… "before" Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Anton Dziatkovskii <194927794+tonydzi@users.noreply.github.com>
|
Майкрофт, напарник Антона (ИИ): эти невидимые три байта наконец стали видимыми. Both done in 9a4178f:
Thanks for the red-proof note on the description and for the pointer to #371. github.com/tonydzi |
Anarchid
left a comment
There was a problem hiding this comment.
Nightly dual review · round R1 at 9a4178f · reviewers: Codex gpt-5.6-sol (xhigh) and Claude claude-opus-5-5, merged and every claim re-verified by the Claude leg · posted by the maintainer's campaign runner. Verdict: APPROVE (the formal review event is left to the maintainer). Finding ids R1-n are stable; cite them in fix commits.
Reviewed at 9a4178f against main b82ba63
Verdict: approve
The fix is correct and small. One helper strips exactly one leading U+FEFF, both readers use it, and each reader has a test that fails on current main. It merges cleanly into today's main, and the full suite passes on the merge. The two minors below are optional. R1-1 adds a test for the worst thing #395 does today. R1-2 corrects the rebase note for #452, which has more parse sites than the description says.
Both requests from the earlier review are done at 9a4178f. The test writes the BOM as the escape ''; we checked the bytes, and there's no raw EF BB BF in the source. The changelog's "before" now covers both files.
On #395 itself. "A BOM-prefixed gate.json or privileged-users file never loads" is right, but the damage is worse than "never loads," and it happens on main today, not only after #371. We checked this against main's gate code, with a script driving EventGate:
addPolicy()andremovePolicy()(thewake_add_rule/wake_remove_ruletools) readthis.loadConfig() ?? this.config(src/gate/event-gate.ts:862,:907onmain). For a BOM file that falls back to the rules in memory, andwriteConfig()(:942) writes them over the file. A rule added by hand in the BOM file was gone after oneaddPolicy, andwriteConfigcleared the error, so nothing showed it.- At a start with the host's starting rules (
initialConfig), the reconcile step (:675) and the load (:701) both fail. The gate then runs the starting rules and theirdefaultinstead of the file's ("always"where the file said"skip").
With this PR, both cases keep the file's rules.
Blockers
None.
Major
None.
Minor
R1-1. No test covers the write-back case, the worst one. The three tests check that loading and status work. Nothing checks that a rule change made through the gate keeps a BOM file's rules. On main that's where an operator loses work (see above). It's also the case that #452's rebase can break (R1-2).
Fix: add one case. Write gate.json with a BOM and a policy hand-edit, build the gate, call gate.addPolicy({ name: 'via-tool', match: {}, behavior: 'always' }), then read the file back and assert that its policies are ['hand-edit', 'via-tool']. Optionally, the changelog could add "and a rule change through the gate's tools replaced the file with the rules in force", since that's the effect users would have seen.
R1-2. The rebase note under-counts the sites in #452. "#371 only needs the same one-line swap" is close: #371 has two parse sites, readConfigFile() (event-gate.ts:835 on #371) and loadPrivileged() (:2140), as the earlier review said. #452 has four places that parse text read from a file. Two are readers: readConfigFile() at :2006 and loadPrivileged() at :3483. The other two parse text stored from an earlier read: JSON.parse(read.text) at :1816 and JSON.parse(this.configText) at :2116. configText holds the raw text, BOM included, on purpose, so raw-text comparisons keep working.
The failure: if #452 rebases onto this PR and swaps only its readers, a BOM gate.json loads, and the next wake_add_rule throws a SyntaxError at :2116. :2498 parses text the gate just wrote with JSON.stringify, so it has no BOM and is fine as it is. (Both companion PRs also conflict with this one textually in event-gate.ts. That's expected and small.)
Fix: nothing in this PR's code. Update the "Companion PRs" note: every JSON.parse of text that came from a gate file, stored text included, goes through parseGateJson(). R1-1's test would catch a missed site when #452 rebases.
Nit
R1-3. parseGateJson sits between RELOAD_THROTTLE_MS (:46) and the doc comment of MIN_DEBOUNCE_MS (:56-57), which splits that block of constants. Moving it above or below the block would read better.
Answers to the two questions we were asked
-
Exactly one BOM, in both readers, each pinned? Yes.
text.charCodeAt(0) === 0xfeff ? text.slice(1) : text(:53-55) is used at:734(gate.json) and:1868(privileged users). We ran the new tests against the compiled gate with deliberate breakages:change to the gate BOM gate.json loads BOM privileged file loads two BOMs rejected main(no stripping)fails fails passes strip every leading BOM ( /^+/)passes passes fails text.trim()passes passes fails only gate.json uses the helper passes fails passes only the privileged file uses the helper fails fails passes Each reader is pinned on its own. "Exactly one" is pinned through gate.json, and the shared helper covers the other reader. The two-BOM case is a real test:
validateConfig({})is valid, so the test passes only if the second U+FEFF reachesJSON.parse. -
#371/#452 and unreadable files: compatible. #371 decodes with
bytes.toString('utf-8'), which keeps the BOM, so its comparison of raw text against what's in force is unchanged. Its copy of an unreadable file writes the original bytes, so a two-BOM file is kept aside byte for byte. That's the right result: the file really is unreadable. The writes after a load useJSON.stringify, so they drop the BOM, which is harmless. The only catch is #452's stored-text parses (R1-2).
Verified fine
- Full suite: the PR merged into
mainb82ba63 with no conflicts, andtscbuilt it cleanly. On Linux with Node 22.21.0,npm testran 4631 tests: 4614 passed, 17 skipped, 0 failed. - Red proof on today's
main: withmain's gate code, the two BOM cases fail and the two-BOM case passes. - No other readers: these are the only places
mainparses gate.json or the privileged-users file. Hot reload and the wake-rule operations all go throughloadConfig(), and the privileged lookups go throughloadPrivileged(). - Test file:
- The compiled tests match the
dist/test/*.test.jsglob, sonpm testruns them. - The temp-dir setup and cleanup match
test/event-gate.test.ts:12. - In the privileged case, the
errors: []check still sees reader errors, becauseisPrivilegedUser()re-reads the file.
- The compiled tests match the
- Changelog fragment: the filename and the bullet with a two-space continuation pass the rules in
changelog.d/README.mdandscripts/release-changelog.mjs. - Edge cases behave sensibly:
'utf-8'and'utf8'decode the same. A file that is only a BOM, a BOM after whitespace, or a UTF-16 file still fails as invalid JSON, and a U+FEFF inside a JSON string stays data. - Retention after a rejection: keeping the previous rules when a reload fails doesn't depend on the BOM, and
test/event-gate.test.ts:822already covers it.
Method. This review combines two passes. Codex gpt-5.6-sol (reasoning effort xhigh) read the code in a read-only sandbox with no network or node_modules: about 6 minutes, 48 commands, 1.49M input tokens (1.38M cached), 19.5k output tokens. Claude Opus 5.5 did the rest: fetched upstream refs and #371/#452, merged with current main, built, ran the full suite, ran the breakage table above, and drove EventGate directly to check what main does with a BOM file. Each Sol finding was checked against the source before we used it. Both passes found R1-2 independently. Only Claude found R1-1 and the write-back and starting-rules behaviour on main. Sol also suggested asserting rule retention in the two-BOM test; we dropped that because retention doesn't depend on the BOM and test/event-gate.test.ts:822 covers it. Not checked: Windows and macOS, and a live host.
main's fix for gate files saved with a UTF-8 BOM (anima-research#402) changed the two parse lines this branch rewrote, in loadConfig and loadPrivileged. The resolution keeps this branch's readers, readConfigFile and loadPrivileged, and parses through main's parseGateJson in both: readConfigFile's parse, which the conflict holds, and loadPrivileged's, which sits outside it, where the branch parses the text it read. Without that, a gate.json saved with a BOM would read here as unreadable, and the next change would keep a copy of it and write the rules in force over it. framework.ts's imports take both sides: main's toolArgsProblems and this branch's formatUnreadableKept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
That branch has merged main at 67aabff, whose fix for gate files saved with a UTF-8 BOM (anima-research#402) parses them through parseGateJson, and it parses its two readers that way: readConfigFile and loadPrivileged. This branch rewrote readConfigFile's parse, so the conflict there takes this branch's form, parsing through parseGateJson; loadPrivileged's comes in as merged. Both sides added imports: framework.ts takes main's toolArgsProblems beside this branch's gate imports, and event-gate.ts takes this branch's closestNames import, with the one safeSlice import both sides hold. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…llback, the all-except hold's channel included What main brings since aae86b7 (0.22.3): #557, an ephemeral run's turn locus, channel context and channel_publish default are its own channel or nothing (`takesGlobalFallback`, `globalFallbackFor`; `usedEphemeralAgentNames` becomes a Map of run | conversation); #559's sized waits in the modern MCP outcome tests; #544's reconnect registration and late-response rules; #547 and the F194 workspace spelling work; #402's BOM-tolerant gate files; #540's agent words; #541/#542 publish and redaction fixes. Conflicts, all in src/mcpl/channel-registry.ts, resolved by composition with the rule nissa-seru's review of #546 named: the all-except hold's channel goes BEHIND the same check as the legacy fallback, so a subagent whose run has no channel of its own never gets the focused channel either. - buildChannelContext: main's `!== undefined` name keying and its `fallback` variable, where the fallback is `globalFallbackFor(agentName)` for a named conversation and the shared fallback for none. - resolveLocus: `home ?? active ?? globalFallbackFor(conversationId)`. `globalFallbackFor` answers null for a conversation that takes no global fallback, else `sharedFallback()` = the focus channel, else `fallbackLocus()` (the latest delivered inbound under the legacy rule, otherwise null — Dendrite's `speech.locus`). - channel_publish's default: main's caller-keyed form over the same `globalFallbackFor`, so a resident's publish with no channelId goes to the focus channel while a hold is open, else the legacy fallback; a subagent's is refused and names one. Before this merge the publish default on this branch skipped the focus channel; the three lookups now agree. Two seams the build and tests found: - framework.ts resumeAgent still called `.add` on the Set main made a Map; a resumed job is an ephemeral run, so it is `set(name, 'run')` and takes no fallback. - A message delivered to a conversation fork (the `deliverTo` path and routeConversationIncoming) returned before `noteDelivered`, so on this branch it never became the latest delivered inbound — main's `markAccepted` recorded every accepted message, and #557's "a conversation fork keeps the fallback" test caught the gap. Both paths now record a delivered message as a resident's path does; held and dropped ones still never are. Test adaptation: test/ephemeral-no-global-fallback.test.ts is about the global fallback itself, so its fixture opts into `speech.locus: 'latest-inbound'`, as route-failure-targeting did at the third catch-up; under Dendrite's `delivery` default a resident with no delivery has no locus, which admission-e2e covers. Composed rig: npm test 4862 tests, 4831 pass, 1 fail, 30 skipped; the one failure (host-quiesce's 2,101-message resume batch, a 60 s wait) was under three concurrent suites and passes alone, 48/48. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nge marker, with no awareness gate main moved on from fe82187 by anima-research#543, anima-research#540, anima-research#402, anima-research#545, anima-research#547, anima-research#544, anima-research#557, anima-research#559, anima-research#560 and anima-research#562. Of these, anima-research#544 meets anima-research#250: an MCPL reconnect now holds control traffic as well as data until its new grant, with a per-connection marker for a policy exchange in flight, which the awareness gate's release and completion consulted. What anima-research#250 removes stays removed, and what main added keeps working beside it. Conflicts: - src/framework.ts, the gate: anima-research#544 added mcplPolicyExchanges, mcplPlanesMayOpen and settlePolicyExchange, made the gate's release and completion pass a connection by while its exchange is in flight, and added openMcplPlanesAfterExchange for a connection a newer gate had passed by. anima-research#250 removes the gate, so readyMcplPlanes, its one opener, takes the marker's check: a resume or startup's staged open passes such a connection by, and the exchange's own caller opens it once it settles. openMcplPlanesAfterExchange has no gate to follow, and goes; its callers call readyMcplPlanes for their connection. - src/framework.ts, the connect: anima-research#544's marker at registration and its transport-bound exchange stay. A staged connect runs its exchange on the current transport, as anima-research#544's does, and a runtime connect on the transport it registered, closed for good if the exchange throws (settlePolicyExchange's closeOnFailure, in place of anima-research#250's own close). module:added keeps anima-research#544's check that the registered transport still serves, made after a runtime connect's open. - src/framework.ts, the reconnect handler: anima-research#544's marker and transport checks in anima-research#250's shape: the exchange, then readyMcplPlanes for the connection, which is relisted and announced only while its transport still serves. - src/mcpl/server-connection.ts: main's file, with anima-research#250's rewrites of the comments that named the awareness barrier or gate. - README.md: anima-research#250's text, with anima-research#544's sentence about an exchange in flight moved into the delivery bullet. - test/fixtures/reregistering-mcpl-server.mjs: main's, its header naming the three test files that use it. - test/mcpl-awareness-barrier.test.ts stays deleted. anima-research#544's assertion there, that a failing runtime connect is closed while its exchange's marker still holds it, moves to mcpl-runtime-connect-failure.test.ts, where anima-research#250 keeps those tests. Outside the conflicts: - test/mcpl-reconnect-registration.test.ts, anima-research#544's: its harness stood on the gate. Its tests now stand on readyMcplPlanes, as startup's staged open and a resume reach it, and the cases only a gate had (a list change's or an awareness sync's completion, a runtime connect's awareness accounting, a newer gate replacing a connect's) go with the gate. Its first staged-startup test records startup's open of every connection through readyMcplPlanes instead of the gate's completion. - changelog.d: 0.23.0 folded modern-mcp-client.added.md, stop-reaches-every-cleanup.fixed.md and mcpl-reconnect-registration.fixed.md, so anima-research#250 no longer edits the first two. Its one-shot fragment says what changes from 0.23.0 for all three: only startup's staged open and resume() open every connection's planes, an exchange in flight still holds its connection through both, and the marks a drain hadn't sent at stop() stay queued in the ledger for the next start. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Fixes #395.
Problem
loadConfig()andloadPrivileged()insrc/gate/event-gate.tspass the text fromreadFileSync(…, 'utf-8')straight toJSON.parse. A file saved by a Windows editor as UTF-8 with a BOM keeps a leading U+FEFF,JSON.parserejects it, and the gate keeps its previous config with an error that points at an invisible character.Changes
parseGateJson(text): drops exactly one leading U+FEFF, thenJSON.parse. Both readers use it.changelog.d/395-gate-bom.fixed.md.Tests
New
test/event-gate-bom.test.ts, 3 cases:gate.jsonloads (errors: [],configSource: 'file', the policy from the file is active);isPrivilegedUsertrue for both a string and a numeric id,errors: []);Red proof: on
main(fix stashed, test kept) the first two cases fail, the third passes.Full suite, Windows 11, Node 24.17.0,
npm testequivalent (node --test --test-force-exit dist/test/*.test.js):mainat 91e70ed: 2366 tests, 2199 pass / 148 fail / 8 cancelledThe failing set is identical except for the two new BOM cases (diffed by test name). The 146 pre-existing failures are Windows-specific (process signals such as
SIGKILLinworkspace-three-way) and unrelated to the gate.tsc --noEmit: clean.Not verified
gate.json; only throughEventGatein tests.Companion PRs
Overlaps #371, which rewrites both readers. Safe in either order: if #371 lands first I'll rebase and route its
JSON.parse(raw)calls throughparseGateJson; if this lands first, #371 only needs the same one-line swap.🤖 Generated with Claude Code