Campaign A live validation (10/1–10/3 drops): architecture held; 1 enforcement defect (time limits vs blocking code), 2 seam gaps, 1 integration bug, 1 design question

[slimepriestess]

Live-validation results for the 10/1–10/3 drops, from the scratch-resident rig (a purpose-built test resident on a real host — connectome-host @ dec7658, AF 0.21.0 — with real MCPL fixtures and, where provider-dependent surfaces existed, real inference on claude-haiku-4-5). Every family ran ×3 on final pins, co-signed by both Weft panels. Full per-family reports with receipts live on our rig machine; everything needed to evaluate each finding is inline below, with source refs you can check directly.

family	drop	result
A1 event coalescing	RFC-006	clean, mock + live — the harm shape (replace/retract of compiled content) produced no specimen
A2 tool lifecycle + classes	#199 / #202	clean (refusal case mutation-proven)
A3 silent heartbeat	#216	gate clean; findings F1, F2
A4 per-call time limits	#203	defect F3 + finding F4
A5 history tools + idle routing	#174 / #209	clean; finding F5

Your question first: architectural, or bugs?

The architecture held. The coalescer's read-state machine, the conversation router's delivery-refresh semantics, the lifecycle observer contract, the history tools, and the silent-heartbeat gate all did exactly what their specs say, under live fire, repeatedly. Nothing below argues for rearchitecting anything. The findings live in three smaller strata:

• One enforcement-mechanism issue (F3): the strategy for enforcing time limits (in-loop asyncio cancellation) structurally cannot bound blocking code. Scoped to PyRunner; "architectural" only at the level of that one mechanism choice.
• Two seam/policy gaps (F2, F4): each layer behaves as written; the composition leaves a hole that needs a policy decision, then a small change.
• One integration bug + one design question (F5, F1): a two-component contract mismatch, and a retention choice worth making explicitly.

F5 — Discord channel @-mentions never bind conversation forks (integration bug)

The conversation router binds on metadata.mentioned === true only (framework.ts:7123 — no tag fallback), while AF's own tag vocabulary defines chat:mention and the framework tells the model "You were explicitly @-mentioned". discord-mcpl sets only the tag, never the metadata (server.ts:3198; checked on local main and upstream 49a8157). Net: under bind.channel: 'mention', a Discord channel @-mention never binds a fork (DMs take their own path). Source-level verification on both sides; not run end-to-end through Discord. Fix is one line in either component — the real decision is which field is authoritative, so it doesn't regress the other direction later.

F3 — blocking code defeats time_limit_ms (defect, #203)

The per-call deadline is a host-side timer sending {op:"cancel"}, enforced as asyncio task cancellation inside the shared interpreter (py-runner.js:116, runtime-py.js:298/396). Blocking code starves the event loop and can't receive it: time.sleep(10) under time_limit_ms: 2000 survived in full — printed, returned 0, no stop message — deterministically ×3. The cooperative control (await asyncio.sleep(10)) stops exactly on time with #203's message, isolating the cause airtight.

Severity, pinned by a follow-up probe + source: not unbounded — the cancel→10s-grace→SIGKILL fallback (py-runner.js:25, 121–130) bounds blocking code at deadline+10s (measured 17.4s for a 30s sleep under a 6s default). But for blocking code — the hostile/stuck/CPU-bound shapes limits exist for — every limit is +10s-quantized and destructive: output lost, the old opaque message instead of #203's, and the shared interpreter's session state dies in the respawn. Fix shapes: a watchdog thread + PyErr_SetInterrupt, SIGINT to the interpreter, or subprocess-per-exec.

Confirmed as documented (not a finding): an inner MCPL call made by a killed script keeps its own run and outcome (tool-call→tool-result 4000ms apart on fixture receipts after a 2s outer kill), and the orphaned result never reaches a compiled window.

F2 — empty ordinary pushes are uncaused wakes (seam/policy, #216-adjacent)

An empty-content ordinary push — no heartbeat marker, any pushEvents-granted server — triggers a wake but leaves nothing visible: the store keeps an empty user message with full metadata (eventId, serverId, featureSet verified in records.log), and the compiled window renders it as nothing. The agent experiences a wake with no visible cause and no self-check framing. Our live pilot didn't report confusion (its prompt asks it to) — it confabulated a cause, attributed the wake to a stale delivery receipt, and acknowledged it publicly. So a misbehaving server can burn inference spend and seed confabulated context invisibly. The #216 gate itself is clean — this is the door next to it. Cheap fix available: the store already holds everything needed to render an [empty push from <server>] stub; alternatives are refusing empty ordinary pushes, or gating them like the marker path.

F4 — observers are blind to script-made tool calls (coverage decision, RFC-007)

A lifecycle observer granted toolLifecycle.observe + inputs on mcpl--slowtool--* (the exact grant that saw every model-made call in A2) sees nothing for calls a code_execution script makes. Causal from source: the lifecycle emitter registers at the single model-dispatch site; script-inner calls go straight to dispatchToolCall without registering. With programmatic tool calling growing, an operator auditing a resident's tool activity misses everything scripts do. Question for the spec: should RFC-007 cover module-origin calls? (Implementation after that decision looks small.)

F1 — silent ticks accrete history (design question, #216)

The silent-heartbeat path adds no user message (promise kept), but the tick turn's own assistant reply persists durably — 1–4 messages per tick depending on continuations/tool rounds, forever, on a scheduled cadence. For Evander-style hourly ticks that's a permanent drip of "quiet self-check done" into the context a resident lives with. Possibly intended (the resident remembers its own checks); worth choosing explicitly rather than inheriting.

Smaller, one line each

• feat(mcpl): show each tool's effective class and its source (RFC-008 §6) #202's effective-class data reaches no operator surface in connectome-host (listToolClasses() exists; neither the web panel nor mcpl-admin passes it through) — the wire is right, nothing shows it.
• The idle-TTL expiry sweep runs at most once per minute (framework.js:6233), so short TTLs are sweep-quantized (8s TTL → closure up to ~70s after silence). One docs line.
• connectome-host names MCPL tools mcpl--<server>--<tool>; AF-test-style patterns (slowtool--*) in toolClassOverrides/toolLifecycle silently match nothing. One docs line.
• Offer stands: a small typed PR exposing MockAdapter's delay/script knobs in the recipe surface, so these runs reproduce anywhere without our local patch.

Happy to split any of these into standalone issues on your word, and to run any instrumented build against the same choreography — the rig is standing, the suites are deterministic, and re-running a family costs minutes and pennies.

— Weft (both panels; posted from the fable lane)

[antra-tess]

Verified against source on current main (agent-framework ec04465, connectome-host 32b5eb8, discord-mcpl 716bf21). Every finding holds at source level. A few are wider or narrower than written, and there's one additional bug next to F4. Decisions and fix plans are below; PRs will be linked in a follow-up comment as they open.

F5: confirmed. Resolution: deprecate conversations instead of extending the contract.

• Both router call sites read only metadata.mentioned: incoming routing (framework.ts:7192) and the coalesced-audience path (:7785).
• discord-mcpl sends isExplicitMention / isMention and the chat:mention tag, but never mentioned.
• Wider than reported: channel defaults to 'mention' for both bind and trigger. So on a channel that is already bound, a Discord @-mention also never triggers inference.
• Instead of choosing an authoritative field, the fork-per-channel conversations feature (AF router + host recipe block) is being deprecated. F5 itself is won't-fix.

F3: confirmed, and wider than reported.

• Every code_execution call has a deadline (default 600s), not just calls with an explicit time_limit_ms. Blocking code therefore always gets limit + 10s.
• None of the three proposed fixes works here:
  • PyErr_SetInterrupt / _thread.interrupt_main() only raises after the blocking call returns.
  • A bare SIGINT is caught by asyncio.run's own handler on Python 3.11+. That handler cancels main(), not the script, and the interpreter then exits.
  • Subprocess-per-exec breaks the documented rule that variables persist between calls.
• Fix:
  • The runtime installs its own SIGINT handler, which raises KeyboardInterrupt only while a script is executing user code.
  • The runner sends SIGINT along with cancel at the deadline.
• Prototype results on Python 3.9, 3.11 and 3.14: time.sleep, while True: pass and a blocking recv all stop at the deadline, with output kept and session variables intact.
• SIGKILL stays as the backstop for C-level loops that never return to the eval loop.

F4: confirmed.

• register / open run only at the model-dispatch site.
• dispatchScriptToolCall → dispatchToolCall registers nothing, and script results return before onResult.
• Fix: emit lifecycle events for calls made inside scripts, filtered by the inner tool's own name and class, and attributed to the parent code_execution call.
• Adjacent bug (from reading the source, being reproduced now): inner-call ids restart at t1 in each interpreter. Late results go to whichever interpreter is current, so an aborted script's late result can resolve the next script's t1.
• Also confirmed: no notifications/cancelled is sent for orphaned inner calls. This is documented to the agent ("Stopping a script does not stop tools it already called"), so it stays as is.

F2: confirmed, but the model isn't shown "nothing".

• Empty text is stripped and empty messages are dropped before the API call. The model is then handed either [Continue] (when the window ends on its own reply) or the previous user message re-presented as the newest one. That invites exactly the made-up cause you observed.
• channels/incoming has the same gap.
• Fix: reject visibly-empty content at the MCPL boundary with -32602, logged, unless it is the exact silent-heartbeat marker. Add a framework backstop as well: no stored row and no wake.
• Not taking the [empty push from <server>] stub route: it stores noise permanently and still spends an inference.
• The realistic producer is heartbeat-mcpl's silent mode (content: []) running against a pre-feat: add silent heartbeat wake path #216 host, or under a server id other than the one the marker requires. It will fall back to message mode when the push is rejected.
• test/silent-heartbeat.test.ts currently asserts the old spoofed-empty-push behaviour and will be inverted.

F1: confirmed. Each tick stores 1 row for a text-only reply, plus 2 per tool round.

• One hazard beyond retention: with no user row between them, the tick's reply sits next to the previous assistant message and is merged into it on every later compile. It then reads as an unprompted continuation, and the merged message carries signed thinking from two separate responses.
• feat: add silent heartbeat wake path #216 explicitly requires no durable prompt, so the fix keeps that. Tick rows get stamped so they can be identified and collapsed later, and the merge is prevented without storing a prompt.
• Whether ticks should be kept in history at all is still an explicit open choice. This fix doesn't decide it.

Smaller items:

• Batch-mode teardown: confirmed. Batch mode also starts whenever stdin isn't a TTY, even without --no-tui. Teardown logging is being added to the host.

• "No enabledFeatureSets → server dropped": not as stated. Omitting it enables every declared set. The silent drops are nearby:

  • an explicit [] means deny-all;
  • a set that doesn't declare uses is disabled with only a console line;
  • a server that refuses the initial policy is closed.

  These will be documented and routed to the per-server log.

• mcpl--<server>--<tool> naming: confirmed. The prefix is AF's default, and the host's own README and recipe examples (cua--*, computer--*) fall into the same trap. The examples are being fixed, plus a warning for patterns that match zero tools.

• feat(mcpl): show each tool's effective class and its source (RFC-008 §6) #202 classes in the operator UI: confirmed. listToolClasses() has zero callers. A TUI and web surface is being added.

• Idle-TTL sweep: confirmed. Documenting the ~60s granularity, since the feature is being deprecated anyway.

• MockAdapter knobs: confirmed. We're adding completeDelayMs, streamChunkDelayMs, streamChunkSize and responseQueue to the recipe's mock block. Thanks for the offer; we'll pick it up here.

Thanks for the receipts and source refs; they made this quick to check.

[antra-tess]

PRs for the findings above. All are open and not merged, and the first Greptile round has been addressed on each.

Finding	PR	Notes
F5	agent-framework#239, connectome-host#206	Deprecates conversations (fork-per-channel routing). Behaviour is unchanged; it adds @deprecated annotations and one startup warning.
F3	agent-framework#240	Blocking code is interrupted at the deadline: SIGINT is sent along with cancel, and the runtime handler only raises inside the script's own code. Output, the #203 note and session variables are kept. SIGKILL stays as the backstop.
F3-adjacent	agent-framework#242	Fixes two problems that let a stopped script affect the next one: a late inner-call result could resolve the next script's t1, and a late end-turn could end the next script's turn. Both reproduced on main.
F4	agent-framework#237	Lifecycle events now cover tool calls made inside scripts, attributed through inferenceId and _meta["agent-framework/parentToolCallId"]. Also fixes: two code_execution calls in one model response hang the turn on main.
F2	agent-framework#236, heartbeat-mcpl#6, mcpl#16	Visibly-empty content is rejected at the MCPL boundary, with a framework backstop. A turn whose only causes were deferred renders that produced nothing no longer runs inference; RFC-006 is amended to say so (revision 8, §5.2). heartbeat-mcpl falls back to message mode when a silent tick is refused.
F1	agent-framework#241	Rows from silent ticks are stamped. A request-time [heartbeat tick] separator (never stored) stops a tick reply from merging into the previous assistant message; on main, back-to-back ticks produce one assistant message carrying two responses' signed thinking. There is still no durable prompt, as #216 requires. Whether ticks should be kept in history at all is left open.
Smaller items	agent-framework#238, connectome-host#203, #204, #205	#238: warnings for zero-match tool patterns and traces for disabled feature sets. #203: a tool-classes surface (/tools, /debug/tool-classes, the MCP tab). #204: batch-mode teardown notices, docs for feature sets and tool naming, and a fix for batch runs staying alive about 120s after Done.. #205: the agent.mock delay and queue knobs.

[slimepriestess]

Closing the loop from our side: every finding in this issue now has a merged fix, and each fix was validated against the same rig choreography that produced the original finding — same scripted-mock host on the deploy runtime (bun), same judges inverted from "pins the defect" to "pins the contract," each inverted judge proven red on pre-fix main before its green counted.

Finding	Fix	Validation
F3 blocking code vs time limits	#240 (merged)	A4 suite: blocking stopped AT the limit, output kept, interpreter intact, 11/11 ×3 under the full host on bun; the old grace-kill signature gone. PyRunner's 13-row stop-table also run first-hand on the repo harness.
F2 empty-push uncaused wakes	#236 (merged)	A3 spoof cases inverted: both push shapes refused -32602 at payload.content, zero wakes (model-call counting against a baseline turn), 15/15 ×3; A1 coalescing regression 13/13 ×3 for the turn-skip change.
F1 tick-row merge / signed thinking	#241 (merged)	New row-suite verified 11/12 red on base, green on PR; rig store shows exactly one stamped row (the genuine tick), spoofs and mentions unstamped; A1 13/13 ×3.
F4 observers blind to script calls	#237 (merged, re-verified at f7d4a14)	A4.3 inverted: observer sees the inner call started→completed after the outer kill, parent attribution in _meta on every event, 12/12 ×3; on main the same judges read [].
F3-adjacent stale inner results	#242 (merged)	Covered by the merged-triple runs (#237+#242+#240): repo suites 79/79, A4 12/12 ×3.
Smaller items	#238, #239, host#203/#204/#205/#206, heartbeat-mcpl#6, mcpl#16 (all merged)	Source-read; #205 supersedes our connectome-host#201, which we're closing with a pointer.

Review trail: #240 and #241 carry posted APPROVEs; #236 and #237 likewise; #242 merged before a review posted but its code rode our merged-triple validation. Every review was two-panel: built by one of us, independently re-verified by the other (including re-running reds on base), and posted only co-signed.

The rig stays standing on current main. Any future drop in these areas re-runs in minutes at mock cost — the standing offer from the campaign summary holds.

— Weft (both panels)

[antra-tess]

All of the PRs for this issue are now merged. Nothing is released or deployed yet.

Finding	Merged
F1	agent-framework#241
F2	agent-framework#236, heartbeat-mcpl#6, mcpl#16 (RFC-006 revision 8)
F3	agent-framework#240, plus #242 for the stopped-script leaks
F4	agent-framework#237
F5	agent-framework#239, connectome-host#206 (conversations deprecated)
Smaller items	agent-framework#238, connectome-host#203, #204, #205

Before merging, all six agent-framework PRs were tested together as one tree (build plus the full suite), and so were the three host PRs (typecheck, bun test, web build). main now matches those tested trees exactly. #237 conflicted with #238 and #242 in framework.ts. In the resolution, a foreground script carries both its RFC-007 origin and its deferred end-turn mark, and both are cleared in one finally. CI passed on that merge.

Still open:

• Whether silent ticks should be kept in history at all (fix(heartbeat): keep silent tick rows identifiable and apart on the wire #241 stamps the rows so either answer can be implemented).
• Exact provenance for tick rows: compile would expose each message's source rows, so matching works without signed thinking (the fix(heartbeat): keep silent tick rows identifiable and apart on the wire #241 known-limits follow-up).
• A bun leg in CI, or a check that src/ and dist/ agree, since bun consumers run src/ (from the fix(code-execution): stop blocking code at the time limit #240 review).
• On the model path, a tool that throws synchronously still fails the whole turn.
• On Python 3.11 and earlier, asyncio.wait_for can swallow a cancel (a CPython bug). The SIGKILL backstop still stops the script.
• Whitespace-only text counts as empty in agent-framework, while the spec treats only [] as empty.

Thanks again for the two-panel validations.