Skip to content

test: stabilize output polling coverage (Fixes #564) - #565

Merged
Karthik Nadig (karthiknadig) merged 1 commit into
mainfrom
test/issue-564-output-coverage
Sep 30, 2026
Merged

Karthik Nadig (karthiknadig) merged 1 commit into
mainfrom
test/issue-564-output-coverage

Conversation

@karthiknadig

Copy link
Copy Markdown
Member

Make the output tests exercise their real polling helpers deterministically, so coverage no longer depends on whether the writer finishes before the first poll.

  • Coordinate the first observed pending state with bounded, one-shot channels.
  • Cover both pending and initially-ready success/error paths.
  • Keep production code, coverage exclusions, and regression budgets unchanged.

Validation: 53 pet-jsonrpc all-target/all-feature tests; workspace all-target/all-feature Clippy; mandatory Rust precommit checks; independent Reviewer LGTM. Signed commit 840e70a.

Fixes #564

Exercise pending and ready states deterministically in the real output-test polling helpers, without changing production behavior or coverage budgets.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Performance Report (Linux)

Result: ✅ Within regression budgets

Metric PR Baseline Delta Change Blocking budget Status
Server startup P50 1ms 1ms +0ms +0.0% >5ms and >100% ➖
Server startup P95 1ms 1ms +0ms +0.0% >50ms and >200% ➖
Discovery duration P50 68ms 48ms +20ms +41.7% >25ms and >30% 🔺
Discovery duration P95 73ms 52ms +21ms +40.4% >50ms and >50% 🔺
Startup-to-first environment P50 14ms 10ms +4ms +40.0% >20ms and >100% 🔺
Startup-to-first environment P95 17ms 12ms +5ms +41.7% >25ms and >100% 🔺
Cold discovery duration P50 148ms 108ms +40ms +37.0% >100ms and >50% 🔺
Refresh round-trip P50 68ms 48ms +20ms +41.7% >25ms and >30% 🔺
Refresh round-trip P95 73ms 52ms +21ms +40.4% >50ms and >50% 🔺
Request-to-first environment P50 13ms 9ms +4ms +44.4% >20ms and >100% 🔺
Request-to-first environment P95 15ms 11ms +4ms +36.4% >25ms and >100% 🔺
Cold refresh round-trip P50 148ms 108ms +40ms +37.0% >100ms and >50% 🔺
Workload PR Baseline
Environments 5 5
Managers 1 1

A regression must exceed both the documented absolute and relative budget. Environment and manager inventories must match exactly within the same inventory schema.

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Test Coverage Report (Linux)

Result: ✅ Within regression budget

Metric PR Baseline Delta
Lines 85.580% 85.548% +0.033pp
Functions 88.400% 88.368% +0.032pp

Allowed numerical tolerance: 0.01 percentage points.

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Performance Report (macOS)

Result: ✅ Within regression budgets

Metric PR Baseline Delta Change Blocking budget Status
Server startup P50 86ms 78ms +8ms +10.3% >100ms and >50% 🔺
Server startup P95 789ms 868ms -79ms -9.1% >750ms and >100% ✅
Discovery duration P50 129ms 146ms -17ms -11.6% >100ms and >50% ✅
Discovery duration P95 210ms 223ms -13ms -5.8% >300ms and >100% ✅
Startup-to-first environment P50 125ms 109ms +16ms +14.7% >150ms and >50% 🔺
Startup-to-first environment P95 181ms 175ms +6ms +3.4% >250ms and >100% 🔺
Cold discovery duration P50 319ms 269ms +50ms +18.6% >250ms and >50% 🔺
Refresh round-trip P50 130ms 147ms -17ms -11.6% >250ms and >50% ✅
Refresh round-trip P95 213ms 231ms -18ms -7.8% >300ms and >100% ✅
Request-to-first environment P50 35ms 41ms -6ms -14.6% >50ms and >50% ✅
Request-to-first environment P95 79ms 74ms +5ms +6.8% >100ms and >100% 🔺
Cold refresh round-trip P50 319ms 270ms +49ms +18.1% >600ms and >50% 🔺
Workload PR Baseline
Environments 10 10
Managers 1 1

A regression must exceed both the documented absolute and relative budget. Environment and manager inventories must match exactly within the same inventory schema.

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Performance Report (Windows)

Result: ✅ Within regression budgets

Metric PR Baseline Delta Change Blocking budget Status
Server startup P50 7ms 11ms -4ms -36.4% >10ms and >50% ✅
Server startup P95 11ms 14ms -3ms -21.4% >50ms and >100% ✅
Discovery duration P50 107ms 162ms -55ms -34.0% >150ms and >50% ✅
Discovery duration P95 118ms 173ms -55ms -31.8% >250ms and >100% ✅
Startup-to-first environment P50 18ms 29ms -11ms -37.9% >25ms and >50% ✅
Startup-to-first environment P95 22ms 45ms -23ms -51.1% >100ms and >100% ✅
Cold discovery duration P50 106ms 164ms -58ms -35.4% >150ms and >50% ✅
Refresh round-trip P50 108ms 163ms -55ms -33.7% >150ms and >50% ✅
Refresh round-trip P95 118ms 174ms -56ms -32.2% >250ms and >100% ✅
Request-to-first environment P50 11ms 18ms -7ms -38.9% >25ms and >50% ✅
Request-to-first environment P95 14ms 34ms -20ms -58.8% >100ms and >100% ✅
Cold refresh round-trip P50 106ms 165ms -59ms -35.8% >150ms and >50% ✅
Workload PR Baseline
Environments 8 8
Managers 1 1

A regression must exceed both the documented absolute and relative budget. Environment and manager inventories must match exactly within the same inventory schema.

@github-actions

github-actions Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Test Coverage Report (Windows)

Result: ✅ Within regression budget

Metric PR Baseline Delta
Lines 83.130% 83.075% +0.055pp
Functions 85.791% 85.753% +0.038pp

Allowed numerical tolerance: 0.01 percentage points.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The test-only changes deterministically cover the required polling paths and preserve existing runtime behavior.

Review effort: Balanced
Findings: None

What changed in this PR

Stabilizes pet-jsonrpc output polling coverage through deterministic test synchronization.

Changes:

  • Adds bounded-channel coordination for pending polling states.
  • Tests pending and initially-ready length/error paths.
  • Updates existing helper call sites for optional coordination.
File Description
crates/​pet-jsonrpc/​src/​output.rs Adds deterministic polling-helper tests without changing production behavior.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@karthiknadig

Copy link
Copy Markdown
Member Author

Current-head Copilot review recommends approval with no findings. Native raw LCOV confirms both real polling helpers now execute their pending paths on every platform: length-wait assertion/yield hits are Linux 8, Windows 139, macOS 1; error-wait hits are 9, 491, and 3 respectively.

Coverage against the workflow baselines improves on all three platforms: Linux 85.580% (+0.033pp), Windows 83.130% (+0.055pp), native macOS 80.586% (+0.069pp). All three CodeQL analyses have zero results and no errors/warnings.

I inspected all performance snapshots and exact-inventory checks. Windows/macOS warm discovery improves; Linux is 68ms versus 48ms (+20ms), and cold discovery is 148ms versus 108ms (+40ms). Those differences remain within the unchanged budgets and are accepted as measurement variation for a change wholly inside the test module; no production code or benchmark setup changed. No reruns or relaxed thresholds were used.

The two ordinary macOS test jobs are still finishing, so this remains draft until those gates complete. Required human approval will not be bypassed.

@karthiknadig
Karthik Nadig (karthiknadig) marked this pull request as ready for review September 30, 2026 20:18
@karthiknadig
Karthik Nadig (karthiknadig) merged commit 7e8391d into main Sep 30, 2026
38 checks passed
@karthiknadig
Karthik Nadig (karthiknadig) deleted the test/issue-564-output-coverage branch September 30, 2026 20:52
Karthik Nadig (karthiknadig) added a commit that referenced this pull request Sep 30, 2026
Merge main after PR #565 so session benchmark coverage no longer depends on output-test scheduling. Preserve all published benchmark commits without rewriting history.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make JSONRPC output-test coverage independent of scheduling

3 participants