Repository navigation
fix(verl): stop admitting batches at the epoch limit - #694
Open
Zhengyang Liu (liuzhengyang699) wants to merge 1 commit into
Open
Zhengyang Liu (liuzhengyang699) wants to merge 1 commit into
Zhengyang Liu (liuzhengyang699) wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Unconditional optional Torch and VERL imports in the new regression test cause collection failures when the verl-cpu dependency group is absent and should be guarded with pytest.importorskip.
1 open issue
What changed in this PR
Fixes #693 by stopping VERL training inputs at the epoch limit while allowing asynchronous groups already admitted to finish.
Changes:
- Bounds dataloader reads and async completion targets, preserving the explicit step limit.
- Runs final validation and checkpointing at the last successful update without duplicating scheduled runs.
- Documents training limits and adds CPU regressions for exhaustion, partial refills, and carry-over draining.
| File | Description |
|---|---|
tests/verl/test_training_epoch_limit.py |
Adds regression coverage for epoch limits and finalization. |
docs/20-trainer-configuration.md |
Explains training limits and final partial collections. |
agentlightning/verl/trainer.py |
Bounds input collection, drains admitted groups, and finalizes on exhaustion. |
Review effort: Balanced
Comment on lines
+10
to
+14
| import numpy as np | ||
| import pytest | ||
| import torch | ||
| from omegaconf import OmegaConf | ||
| from verl import DataProto |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Fixes #693.
With one epoch, two dataloader batches, and a larger explicit step budget,
fit()currently trains on three batches: the iterator starts the next epoch before the outer loop sees that the limit has been exceeded. Async refills can cross the same boundary.Bound input collection at the dataloader helper. Synchronous training stops before another rollout; asynchronous training allows a partial final refill and finishes groups already admitted. Its completion target is capped to the available groups so the last collection cannot wait for inputs that will never arrive. The explicit training-step limit still takes precedence.
When data exhaustion ends training, enabled final validation and checkpointing use the last successful update, without repeating a scheduled validation/save or counting another optimizer step. The configuration docs describe these limits. Existing checkpoint-resume bookkeeping is unchanged.
Twelve CPU regressions cover one/two epochs, sync and async collection, exact/partial refills, carry-over draining, final saves, skipped collections, and step-limit precedence. The carry-only case also exercises the real
_train_step()path through emptyDataProtoconstruction. Nine of the initial 11 cases fail on unchanged main.Validation
uv run --locked --no-sync pytest -v --durations=20 tests— 253 passed, 4 skipped.uv run --locked --no-sync pyright— 0 errors, 0 warnings.uv run --locked --no-sync pre-commit run --all-files --show-diff-on-failure— passed; the new test file was also checked explicitly.uv run --locked --no-sync ruff check .,uv run --locked --no-sync ruff format --check ., anduv run --locked --no-sync python scripts/check_headers.py— passed.UV_PROJECT_ENVIRONMENT=/tmp/agent-lightning-epoch-docs uv run --locked --no-sync mkdocs build --strict,uv build --no-sources, andgit diff --cached --check— passed.The development environment used the locked
devextra/group andverl-cpugroup; the separate docs environment used the lockeddocsgroup. Independent review also exercised a realStatefulDataLoaderacross two async epochs.This was found during a follow-up audit with AI assistance. AI assistance was also used for the fix, tests, and review. Validation used CPU tests with worker operations stubbed; no GPU rollout or end-to-end training was run.