Skip to content

Batch timer: make it interruptable by termination flag#635

Merged
assaf758 merged 1 commit intomlrun:developmentfrom
assaf758:batch.timer.fix
Mar 24, 2026
Merged

Batch timer: make it interruptable by termination flag#635
assaf758 merged 1 commit intomlrun:developmentfrom
assaf758:batch.timer.fix

Conversation

@assaf758
Copy link
Copy Markdown
Member

Issue: test_batch_by_user_key "hangs"

Root cause: Commit c34fda6 replaced cancel() on the timer task with a _terminating flag + await self._timeout_task. The problem: _sleep_and_emit calls asyncio.sleep(flush_after_seconds) which doesn't observe the _terminating flag mid-sleep. Since the test uses flush_after_seconds=100, termination would block for up to 100 seconds waiting for the sleep to finish.

Fix: Replace asyncio.sleep(remaining) with asyncio.wait_for(self._stop_timer_event.wait(), timeout=remaining)
An asyncio.Event (_stop_timer_event) is created when the timer task starts and set immediately when _terminating is flagged. This wakes the sleeping timer instantly while still allowing any in-progress _emit_batch call to complete cleanly (the cancel-safety concern from c34fda6 is preserved).

@assaf758 assaf758 requested review from alxtkr77 and gtopper March 23, 2026 23:48
@assaf758 assaf758 changed the title Batch timer: make it timer interuptable by therinmating flag Batch timer: make it interruptable by terminating flag Mar 23, 2026
@assaf758 assaf758 changed the title Batch timer: make it interruptable by terminating flag Batch timer: make it interruptable by termination flag Mar 24, 2026
@assaf758 assaf758 merged commit d55b973 into mlrun:development Mar 24, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants