Skip to content

feat(relay): improve tool-call progress reporting#80

Merged
zikolach merged 17 commits into
mainfrom
feature/improve-tool-call-progress-reporting
Jul 11, 2026
Merged

feat(relay): improve tool-call progress reporting#80
zikolach merged 17 commits into
mainfrom
feature/improve-tool-call-progress-reporting

Conversation

@zikolach

Copy link
Copy Markdown
Owner

Summary

  • Replaces generic, noisy tool milestones with safe, bounded aggregated tool-progress cards.
  • Adds tool-call lifecycle tracking by toolCallId with active/completed/failed state.
  • Summarizes built-in tools via allowlisted args only (bash/read/edit/write/grep/rg/find/ls) with redaction and truncation.
  • Suppresses noisy legacy tool milestones when lifecycle state exists.
  • Preserves completion-only/quiet behavior and existing authorization gating.
  • Adds unit and runtime/integration coverage across Telegram direct, broker, Slack, and Discord progress paths.

Validation

  • npm run typecheck
  • npm test
  • openspec validate improve-tool-call-progress-reporting --strict

Copilot AI review requested due to automatic review settings June 14, 2026 16:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves relay progress UX by replacing repetitive per-tool “milestone” messages with a single bounded, redacted “Tool progress” card that tracks tool-call lifecycle (active/completed/failed) and coalesces updates across Telegram direct runtime, broker, Slack, and Discord.

Changes:

  • Adds a turn-scoped tool-progress accumulator plus safe allowlisted tool-intent summarization and card formatting.
  • Integrates tool lifecycle tracking into the relay runtime and updates progress coalescing to keep the latest tool-progress card without adding noisy repetition counts.
  • Updates docs/spec tasks and expands unit + integration coverage across supported adapters and broker paths.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
tests/slack-runtime.test.ts Updates Slack progress assertions to expect the new bounded “Tool progress” card and details.
tests/runtime.test.ts Updates Telegram in-process runtime tests to assert tool-progress detail strings (e.g., bash, edit, aggregate counts).
tests/progress.test.ts Adds unit coverage for tool-progress summarization, aggregation, coalescing, bounds, and truncation.
tests/integration.test.ts Adds integration coverage ensuring tool lifecycle events produce safe aggregated progress without leaking inputs/results.
tests/discord-runtime.test.ts Updates Discord progress tests to expect the new tool-progress card content and details.
tests/broker-process.test.ts Updates broker progress expectations to use “Tool progress” with safe detail content.
README.md Documents how tool progress is summarized/redacted in normal/verbose progress modes.
openspec/changes/improve-tool-call-progress-reporting/tasks.md Marks the OpenSpec task checklist as completed for this change set.
extensions/relay/runtime/extension-runtime.ts Wires tool lifecycle events into the accumulator, suppressing legacy tool milestones when lifecycle state exists.
extensions/relay/notifications/tool-progress.ts Introduces the accumulator, safe intent summarizers for allowlisted tools, and bounded card formatting.
extensions/relay/notifications/progress.ts Improves milestone coalescing to preserve the latest entry content and suppresses (N×) for the tool-progress card.

Comment thread extensions/relay/notifications/tool-progress.ts
Comment thread extensions/relay/notifications/tool-progress.ts
Comment thread extensions/relay/notifications/tool-progress.ts
Comment thread extensions/relay/notifications/tool-progress.ts
@zikolach

Copy link
Copy Markdown
Owner Author

Implemented Copilot feedback: removed unused dead state from and corresponding reset/increment references in . All required checks still pass:

@zylab/pirelay@0.8.0 typecheck
tsc --noEmit,
@zylab/pirelay@0.8.0 test
vitest run

RUN v4.1.5 /Users/user/Projects/pirelay

Test Files 69 passed | 1 skipped (70)
Tests 737 passed | 2 skipped (739)
Start at 21:53:21
Duration 70.16s (transform 3.42s, setup 0ms, import 15.26s, tests 33.29s, environment 22ms), Change 'improve-tool-call-progress-reporting' is valid. Please re-review this branch.

@zikolach

Copy link
Copy Markdown
Owner Author

Implemented Copilot feedback: removed unused dead state from ToolProgressAccumulator in extensions/relay/notifications/tool-progress.ts and removed all reset/increment references. Validation still passes: npm run typecheck, npm test, openspec validate improve-tool-call-progress-reporting --strict. Please re-review this branch.

@zikolach

Copy link
Copy Markdown
Owner Author

Done: I addressed the only remaining Copilot feedback by removing the unused revision dead field and related reset/increment usage from extensions/relay/notifications/tool-progress.ts (commit 90e2639). All checks still pass (npm run typecheck, npm test, openspec validate improve-tool-call-progress-reporting --strict). I also marked the feedback thread resolved and marked it as re-reviewed.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 4 comments.

Comment thread extensions/relay/runtime/extension-runtime.ts Outdated
Comment thread extensions/relay/runtime/extension-runtime.ts Outdated
Comment thread extensions/relay/runtime/extension-runtime.ts
Comment thread extensions/relay/notifications/tool-progress.ts Outdated
@zikolach

Copy link
Copy Markdown
Owner Author

Implemented remaining Copilot review findings (PR #80): loaded config before all tool_execution_start/end progress updates to avoid unredacted summaries, added route-key guard for tool_call deferred progress so async flow can’t write to changed route, and fixed tool-progress row truncation math to use displayed active/failed counts (plus added regression test). Also marked all remaining review threads resolved. Validation still passes: npm run typecheck, npm test, openspec validate improve-tool-call-progress-reporting --strict. Please re-run review.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Comment thread extensions/relay/runtime/extension-runtime.ts Outdated
@zikolach

Copy link
Copy Markdown
Owner Author

Addressed remaining open PR feedback on #80: updated progress helper typing so accepts too (matching expectations). This keeps all runtime progress calls strictly typed. Ran full checks:

@zylab/pirelay@0.8.0 typecheck
tsc --noEmit,
@zylab/pirelay@0.8.0 test
vitest run

RUN v4.1.5 /Users/user/Projects/pirelay

Test Files 69 passed | 1 skipped (70)
Tests 738 passed | 2 skipped (740)
Start at 23:38:50
Duration 41.80s (transform 1.99s, setup 0ms, import 7.66s, tests 21.27s, environment 10ms), and Change 'improve-tool-call-progress-reporting' is valid. PR thread PRRT_kwDOSQNns86JkP2I resolved.

@zikolach

Copy link
Copy Markdown
Owner Author

Addressed final remaining PR review feedback on #80: for tool_call progress helper typing, now accepts (same shape required by ). PR thread PRRT_kwDOSQNns86JkP2I is resolved. Validation re-run:

@zylab/pirelay@0.8.0 typecheck
tsc --noEmit,
@zylab/pirelay@0.8.0 test
vitest run

RUN v4.1.5 /Users/user/Projects/pirelay

Test Files 69 passed | 1 skipped (70)
Tests 738 passed | 2 skipped (740)
Start at 23:39:49
Duration 42.60s (transform 2.05s, setup 0ms, import 7.79s, tests 21.78s, environment 10ms), Change 'improve-tool-call-progress-reporting' is valid.

@zikolach

Copy link
Copy Markdown
Owner Author

Addressed final remaining PR review feedback on #80: for tool_call progress helper typing, recordAllowedProgress now accepts { redactionPatterns, maxProgressMessageChars, recentActivityLimit } to match recordToolProgressActivity expectations. PR thread PRRT_kwDOSQNns86JkP2I is resolved. Validation re-run: npm run typecheck, npm test, openspec validate improve-tool-call-progress-reporting --strict.

Copilot AI review requested due to automatic review settings June 20, 2026 07:43
@zikolach

Copy link
Copy Markdown
Owner Author

Investigated failing CI job: this was a real flaky timing issue in extensions/relay/broker/process.js for progress delivery scheduling on Node20. syncProgressDelivery could schedule immediate (0ms) flushes, creating a small race where consecutive registerRoute progress updates in the same turn can coalesce and skip the expected edit path.

I fixed this by enforcing a minimum 1ms flush delay:

  • changed Math.max(0, interval - elapsed) -> Math.max(1, interval - elapsed)

This keeps progress delivery semantics intact but makes update ordering deterministic across Node versions. Re-ran: npm test tests/broker-process.test.ts, npm test, npm run typecheck, and openspec validate improve-tool-call-progress-reporting --strict all pass locally.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.

Comment thread extensions/relay/runtime/extension-runtime.ts
@zikolach

Copy link
Copy Markdown
Owner Author

Addressed latest new feedback on PR #80: when cannot load config, we now still record tool failure () and publish route state so user-visible failure reason is not lost, while skipping progress-card formatting that needs config redaction settings. This is in commit (). Re-ran checks:

@zylab/pirelay@0.8.0 typecheck
tsc --noEmit, full
@zylab/pirelay@0.8.0 test
vitest run

RUN v4.1.5 /Users/user/Projects/pirelay

Test Files 69 passed | 1 skipped (70)
Tests 738 passed | 2 skipped (740)
Start at 09:59:23
Duration 43.35s (transform 2.04s, setup 0ms, import 8.27s, tests 21.77s, environment 10ms), and Change 'improve-tool-call-progress-reporting' is valid. Review thread PRRT_kwDOSQNns86K-R9o resolved.

@zikolach

Copy link
Copy Markdown
Owner Author

Addressed latest new feedback on PR #80: when tool_execution_end cannot load config, we now still set notification.lastFailure for error events and publish route state (so failure reason is preserved) while skipping progress-card formatting that requires config redaction settings. Commit: fix(relay): keep tool failure signal when config unavailable. Re-ran checks: npm run typecheck, full npm test, and openspec validate improve-tool-call-progress-reporting --strict. Review thread PRRT_kwDOSQNns86K-R9o is resolved.

Copilot AI review requested due to automatic review settings June 26, 2026 05:34

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated 1 comment.

Comment thread extensions/relay/notifications/tool-progress.ts Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings July 11, 2026 13:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Comment thread extensions/relay/runtime/extension-runtime.ts
Copilot AI review requested due to automatic review settings July 11, 2026 14:07

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Comment thread extensions/relay/notifications/tool-progress.ts Outdated
Comment thread extensions/relay/notifications/tool-progress.ts Outdated
Copilot AI review requested due to automatic review settings July 11, 2026 14:12

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Comment thread extensions/relay/notifications/tool-progress.ts
Copilot AI review requested due to automatic review settings July 11, 2026 14:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated 1 comment.

Comment thread extensions/relay/notifications/tool-progress.ts
Copilot AI review requested due to automatic review settings July 11, 2026 14:19

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings July 11, 2026 14:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

Copilot AI review requested due to automatic review settings July 11, 2026 16:40

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 14 out of 14 changed files in this pull request and generated no new comments.

@zikolach zikolach merged commit ceb5ada into main Jul 11, 2026
3 checks passed
@zikolach zikolach deleted the feature/improve-tool-call-progress-reporting branch July 11, 2026 16:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants