Skip to content

fix: Slack bridge self-heals channels resolved after startup (#677)#142

Draft
drewdrewthis wants to merge 3 commits into
mainfrom
fix/677-bridge-channel-reresolve
Draft

fix: Slack bridge self-heals channels resolved after startup (#677)#142
drewdrewthis wants to merge 3 commits into
mainfrom
fix/677-bridge-channel-reresolve

Conversation

@drewdrewthis

@drewdrewthis drewdrewthis commented Jul 10, 2026

Copy link
Copy Markdown
  • Scariest thing — startup channel resolution changed from a per-agent resolveChannelId (resolve on presence) to a single conversations.list snapshot gated on is_member. If is_member were wrong for a currently-healthy agent, that agent would drop to pending and go silent on the next bridge restart. Mitigated: existing agents' bots are members (they work today) → is_member=true → they resolve at startup exactly as before; the full suite (286/286) stays green; the change only adds a retry path for the case that was previously dropped on the floor.
  • Start herecli/src/slack/reresolve.ts (matchPendingChannels, the membership gate) and the two edited blocks in cli/src/slack/bridge.ts: the startup resolve (partition into mirrored vs pending) and the self-rescheduling resolver just before socket.start().

Why

Part of #677 (this is the bridge fix; the closing PR is the companion langwatch-saas deploy change that lands it on the box — see Anything surprising). The Slack bridge resolved each agent's channel once at startup and skipped (never retried) any that didn't resolve. config-sync restarts the bridge when the roster changes, but the human creates the agent's Slack channel and invites the bot after that restart — so the channel isn't resolvable when the bridge reads it, the agent is dropped, and it stays silent until a manual systemctl restart agents-slack-bridge. This recurred for every newly-added agent.

What changed

  • Gate promotion on bot membership (is_member), not channel resolutionalternative: resolve-on-presence (today) → consequence: conversations.list returns a public channel the moment it exists, before the bot is invited; mirroring it then posts into a channel the bot can't write to (not_in_channel) and receives no events from — a silent half-mirrored agent. Membership makes "mirrored" mean "can actually converse."
  • Self-rescheduling setTimeout retry over a pending setalternative: setIntervalconsequence: overlapping ticks under Slack rate-limiting could both see an agent pending and double-append its tail (every turn posts twice). The chain re-arms only while agents remain pending, so it costs nothing once all are mirrored.
  • One conversations.list snapshot per pass, matched against all pending agentsalternative: a paginated lookup per agent → consequence: N-agents × pages every interval on the same token bucket as posts/pills would 429-starve the healthy agents.
  • Picker loop reads tails livealternative: the existing once-built tails.filter(...) snapshot → consequence: a late-resolved Claude agent would get text mirroring but no interactive pickers, and zero Claude agents at boot would never schedule the loop at all.

How it works

cli/src/slack/
  reresolve.ts   NEW — matchPendingChannels(pending, snapshot): pure, membership-gated,
                 removes resolved from stillPending (promote-once). The tested core.
  client.ts      + listChannels(): one paginated conversations.list snapshot
                 -> [{id, name, isMember, isPrivate}]
  bridge.ts      runSlackBridge: startup partitions agents into mirrored (buildTail) vs
                 pending via ONE snapshot; a self-rescheduling resolver promotes pending
                 agents as their channel + bot-membership appear, appending tail + mapping
                 LIVE (read by reference by the poll loop and inbound router -> no restart);
                 picker loop reads tails live so late Claude agents get pickers.
cli/src/
  slack-reresolve.test.ts  NEW — 9 tests incl. the is_member falsifiability case.

Human verification (for if you really don't trust me)

  1. git checkout fix/677-bridge-channel-reresolve && cd cli && pnpm install
  2. pnpm run build → tsc exits 0. pnpm test286 pass.
  3. Falsifiability: in src/slack/reresolve.ts change if (hit && hit.isMember)if (hit); run pnpm exec tsx --test src/slack-reresolve.test.ts → the "not a member" and "mixed roster" tests FAIL. Revert → 9/9 pass.

How I can prove I was successful

  • [proven] tsc typecheck clean; full suite 286/286; and the is_member gate is load-bearing — removing it fails exactly the two membership tests (not a member, mixed roster) and restoring it returns 9/9. Terminal evidence:

    #677 Track A proof

  • [claimed — not exercisable from the dev environment] the live end-to-end behavior (a running bridge mirroring a real Slack channel created after startup, with no restart) requires the agents-box and the Slack app tokens, which live only in the box's secrets.env; kanban slack bridge exits at startup without SLACK_BOT_TOKEN/SLACK_APP_TOKEN, so there is no user-observable runtime surface reachable from here. The wiring around the tested core is typechecked and reads tails/mapping by reference by construction. The box E2E is the deploy-time gate — runbook attached to #677: add a test agent → apply → create its channel + invite the bot → send a message → get a reply with no manual restart; confirm journalctl -u agents-slack-bridge shows now mirroring <slug> and ExecMainStartTimestamp unchanged across create→reply.

Anything surprising?

  • This fix does not deploy itself. /opt/kanban-code is built only at provision time on the agents box; nothing updates it post-boot. A companion langwatch-saas change (pin kanban_code_ref in tfvars + a config-sync converge — "Track B3" in the plan) is required to land this on the running box without re-provisioning (the box has prevent_destroy). Until that ships, this reaches the box only on its next provision.
  • The membership gate slightly changes startup for a misconfigured public channel the bot was never invited to: it previously "resolved" (uselessly — the bot couldn't post), now it stays pending until invited. Not a regression; it never worked before.

Ubuntu and others added 2 commits July 10, 2026 17:42
The bridge resolved each agent's Slack channel once at startup and skipped
(never retried) any that didn't resolve. A channel created — or the bot
invited — AFTER the bridge last restarted was therefore never mirrored until
a manual `systemctl restart agents-slack-bridge`. config-sync restarts the
bridge on a roster add, but that restart fires before the channel exists (the
human creates the channel + invites the bot afterward), so every newly-added
agent hit this.

Fix: keep unresolved agents in a `pending` set and retry them against a fresh
conversations.list snapshot on a self-rescheduling timer (not setInterval —
overlapping ticks could double-append a tail), gating promotion on bot
MEMBERSHIP (is_member): a public channel is listed before the bot is invited,
so resolving on mere presence would mirror a channel the bot can't post to
(not_in_channel) — the exact silent-agent symptom. New tails/mapping are
appended live (read by reference by the poll loop + inbound router), so a late
resolve needs no restart. The picker loop now reads `tails` live so a
late-resolved Claude agent gets pickers too.

- src/slack/reresolve.ts: pure matchPendingChannels (membership-gated, promote-once)
- src/slack/client.ts: listChannels() one-shot snapshot (id/name/is_member)
- src/slack/bridge.ts: pending set + self-rescheduling resolver + picker heal
- src/slack-reresolve.test.ts: 9 tests incl. the is_member falsifiability case

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ifiability)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@drewdrewthis drewdrewthis self-assigned this Jul 10, 2026
…(#677)

A throw in resolvePendingOnce (e.g. statSync racing a just-deleted transcript)
would reject the awaited pass and skip the re-schedule, silently killing the
self-heal retry chain. Wrap in .catch().finally(schedule) so a bad pass logs
and the chain always re-arms while agents remain pending.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant