feat(relay): improve tool-call progress reporting#80
Conversation
There was a problem hiding this comment.
Pull request overview
This PR improves relay progress UX by replacing repetitive per-tool “milestone” messages with a single bounded, redacted “Tool progress” card that tracks tool-call lifecycle (active/completed/failed) and coalesces updates across Telegram direct runtime, broker, Slack, and Discord.
Changes:
- Adds a turn-scoped tool-progress accumulator plus safe allowlisted tool-intent summarization and card formatting.
- Integrates tool lifecycle tracking into the relay runtime and updates progress coalescing to keep the latest tool-progress card without adding noisy repetition counts.
- Updates docs/spec tasks and expands unit + integration coverage across supported adapters and broker paths.
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
tests/slack-runtime.test.ts |
Updates Slack progress assertions to expect the new bounded “Tool progress” card and details. |
tests/runtime.test.ts |
Updates Telegram in-process runtime tests to assert tool-progress detail strings (e.g., bash, edit, aggregate counts). |
tests/progress.test.ts |
Adds unit coverage for tool-progress summarization, aggregation, coalescing, bounds, and truncation. |
tests/integration.test.ts |
Adds integration coverage ensuring tool lifecycle events produce safe aggregated progress without leaking inputs/results. |
tests/discord-runtime.test.ts |
Updates Discord progress tests to expect the new tool-progress card content and details. |
tests/broker-process.test.ts |
Updates broker progress expectations to use “Tool progress” with safe detail content. |
README.md |
Documents how tool progress is summarized/redacted in normal/verbose progress modes. |
openspec/changes/improve-tool-call-progress-reporting/tasks.md |
Marks the OpenSpec task checklist as completed for this change set. |
extensions/relay/runtime/extension-runtime.ts |
Wires tool lifecycle events into the accumulator, suppressing legacy tool milestones when lifecycle state exists. |
extensions/relay/notifications/tool-progress.ts |
Introduces the accumulator, safe intent summarizers for allowlisted tools, and bounded card formatting. |
extensions/relay/notifications/progress.ts |
Improves milestone coalescing to preserve the latest entry content and suppresses (N×) for the tool-progress card. |
|
Implemented Copilot feedback: removed unused dead state from and corresponding reset/increment references in . All required checks still pass:
RUN v4.1.5 /Users/user/Projects/pirelay Test Files 69 passed | 1 skipped (70) |
|
Implemented Copilot feedback: removed unused dead state from ToolProgressAccumulator in extensions/relay/notifications/tool-progress.ts and removed all reset/increment references. Validation still passes: npm run typecheck, npm test, openspec validate improve-tool-call-progress-reporting --strict. Please re-review this branch. |
|
Done: I addressed the only remaining Copilot feedback by removing the unused |
|
Implemented remaining Copilot review findings (PR #80): loaded config before all tool_execution_start/end progress updates to avoid unredacted summaries, added route-key guard for tool_call deferred progress so async flow can’t write to changed route, and fixed tool-progress row truncation math to use displayed active/failed counts (plus added regression test). Also marked all remaining review threads resolved. Validation still passes: npm run typecheck, npm test, openspec validate improve-tool-call-progress-reporting --strict. Please re-run review. |
|
Addressed remaining open PR feedback on #80: updated progress helper typing so accepts too (matching expectations). This keeps all runtime progress calls strictly typed. Ran full checks:
RUN v4.1.5 /Users/user/Projects/pirelay Test Files 69 passed | 1 skipped (70) |
|
Addressed final remaining PR review feedback on #80: for tool_call progress helper typing, now accepts (same shape required by ). PR thread PRRT_kwDOSQNns86JkP2I is resolved. Validation re-run:
RUN v4.1.5 /Users/user/Projects/pirelay Test Files 69 passed | 1 skipped (70) |
|
Addressed final remaining PR review feedback on #80: for tool_call progress helper typing, recordAllowedProgress now accepts { redactionPatterns, maxProgressMessageChars, recentActivityLimit } to match recordToolProgressActivity expectations. PR thread PRRT_kwDOSQNns86JkP2I is resolved. Validation re-run: npm run typecheck, npm test, openspec validate improve-tool-call-progress-reporting --strict. |
|
Investigated failing CI job: this was a real flaky timing issue in I fixed this by enforcing a minimum 1ms flush delay:
This keeps progress delivery semantics intact but makes update ordering deterministic across Node versions. Re-ran: |
|
Addressed latest new feedback on PR #80: when cannot load config, we now still record tool failure () and publish route state so user-visible failure reason is not lost, while skipping progress-card formatting that needs config redaction settings. This is in commit (). Re-ran checks:
RUN v4.1.5 /Users/user/Projects/pirelay Test Files 69 passed | 1 skipped (70) |
|
Addressed latest new feedback on PR #80: when tool_execution_end cannot load config, we now still set notification.lastFailure for error events and publish route state (so failure reason is preserved) while skipping progress-card formatting that requires config redaction settings. Commit: fix(relay): keep tool failure signal when config unavailable. Re-ran checks: npm run typecheck, full npm test, and openspec validate improve-tool-call-progress-reporting --strict. Review thread PRRT_kwDOSQNns86K-R9o is resolved. |
Summary
Validation