Skip to content

[SDK Sentinel] Make Nexus metrics test shutdown-aware - #1503

Draft
sdk-sentinel-bot wants to merge 1 commit into
temporalio:mainfrom
sdk-sentinel-forks:automation/sdk-sentinel/ci-flake/rust/31852541129-1-09f0cad6579f
Draft

[SDK Sentinel] Make Nexus metrics test shutdown-aware#1503
sdk-sentinel-bot wants to merge 1 commit into
temporalio:mainfrom
sdk-sentinel-forks:automation/sdk-sentinel/ci-flake/rust/31852541129-1-09f0cad6579f

Conversation

@sdk-sentinel-bot

@sdk-sentinel-bot sdk-sentinel-bot commented Aug 15, 2026

Copy link
Copy Markdown
Contributor

Caution

This draft contains untrusted AI-generated code. Do not approve or run CI until a maintainer has reviewed the diff. SDK Sentinel verified that GitHub Actions remained approval-gated with zero executable jobs when this PR was opened.

Summary

Fix an intermittent nexus_metrics panic seen in a [failing Windows integration attempt](https://github.com/temporalio/sdk-rust/actions/runs/31741132284/job/94584505811). The [preceding successful main CI](https://github.com/temporalio/sdk-rust/actions/runs/31741023941) and [same-SHA successful rerun](https://github.com/temporalio/sdk-rust/actions/runs/31741132284) passed.

Root cause

The test starts four Nexus operations but polls exactly five tasks, assuming the timeout cancellation is always delivered before worker shutdown. Once the workflow completes, shutdown can win that race; the fifth poll then returns ShutDown and the test unwraps it.

Fix

Poll until shutdown, acknowledge cancellation tasks whenever they arrive, and assert that all four required start operations were observed. Existing metric assertions remain unchanged.

Validation

The canonical formatting, lint, test-lint, and check script passed. Focused execution was attempted twice, but the isolated sandbox blocked zstd-sys compilation with EPERM before the test ran. Windows and Ubuntu integration CI are required.

Validation status: validation-incomplete

  • Flake confidence: high
  • Fix confidence: high
  • Value: medium
  • Patch scope: standard — 1 changed file; 1.6 KiB
  • Local reproduction: unavailable — The focused integration binary could not be built locally because the sandbox denied an operation in zstd-sys build.rs.
  • Regression coverage: not-run — The existing nexus_metrics test was corrected, but focused execution was blocked before the test ran.
  • Unchanged baseline (Cargo formatting, linting, and checks) — passed
  • Independent candidate (Cargo formatting, linting, and checks) — passed

Investigator-run checks

  • CARGO_BUILD_JOBS=2 timeout 180 cargo integ-test nexus_metrics — failed; 2 attempt(s). Both attempts stopped before test execution because zstd-sys build.rs received EPERM from the isolated sandbox.
  • .ci-flake-runtime/input/validate.sh — passed; 1 attempt(s). Nightly formatting, cargo lint, cargo test-lint, and cargo check passed.

Required target CI

  • Per merge CI — Integ tests (windows-latest)
  • Per merge CI — Integ tests (ubuntu-latest)

Residual risks

  • The candidate compiled under repository checks but was not executed locally.
  • The exact cancellation task sequence was not present in the failed log.
  • Other credible clusters, including routing eventual consistency and metric-recording latency, remain unfixed.

Automation provenance

If this finding should not be fixed, apply the sdk-sentinel:false-positive label and close the PR. Sentinel will suppress the finding until its affected source changes.

SDK Sentinel recurrence history

  • Unique failed CI run attempts: 3
  • First occurrence: 2026-08-13 20:52 UTC
  • Latest occurrence: 2026-08-18 23:03 UTC
  • Recent occurrences:

@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

1 similar comment
@CLAassistant

Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants