Skip to content

test(callbacks): make transient heartbeat-busy lease fixture deterministic #1144

Description

@dennisonbertram

Work type

Bug

Problem

TestCallbackManagerTransientHeartbeatBusyRetainsClaim races a 90 ms initial lease, a 22.5 ms heartbeat ticker, an injected first renewal failure, and a raw 150 ms sleep. Under race CI the initial lease can expire before the post-busy renewal; production safely releases/retries and starts attempt 2, but the fixture incorrectly expects attempt 1 to remain active.

Evidence: #1143 race CI logged a terminalized started callback at attempt 2 with callback admission unavailable. Current main 4436a590 and #1143 have byte-identical callback production code and the entire failing test. Local exact-main and exact-PR stress passed 100/100, confirming a latent baseline timing race.

Scope

Test/docs only. Instrument transientLeaseStore with deterministic channels for injected first failure and first successful delegated ExtendLease; remove the raw sleep. Use a one-second lease and bounded causal waits.

Out of scope

  • Callback product retry/lease semantics.

Acceptance criteria

  • Fixture waits for injected failure then first successful renewal, and proves durable renewed deadline exceeds the captured initial deadline while attempt remains 1.
  • Starter release/cleanup cannot leave blocked goroutines before manager shutdown.
  • No raw timing sleep is used to establish renewal.
  • Focused normal/race stress, full regression, and hosted fast/race pass.

Impact/rollback

Test/docs only; no runtime/API/persistence change.

Test-first plan

Add causal gates and prove the old timing-only sequence is eliminated; retain the intended durable ownership assertions.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions