arc-runners: let runner defs select a workflow scheduler#804
Draft
georgehong wants to merge 4 commits into
Draft
arc-runners: let runner defs select a workflow scheduler#804georgehong wants to merge 4 commits into
georgehong wants to merge 4 commits into
Conversation
Capacity reportcommit ✅ simulate-cluster✅ analyze-utilization |
tofu plan — arc-cbr-production✅ Plan succeeded · commit Plan output |
tofu plan — arc-cbr-production-uw1✅ Plan succeeded · commit Plan output |
tofu plan — meta-prod-aws-ue1✅ Plan succeeded · commit Plan output |
tofu plan — lf-prod-aws-ue1❌ Plan failed · commit Plan output |
tofu plan — lf-prod-aws-ue2✅ Plan succeeded · commit Plan output |
georgehong
added a commit
that referenced
this pull request
Jun 22, 2026
Add an optional per-def scheduler_name. generate_runners.py stamps it on both the real workflow pod (schedulerName) and the listener's CAPACITY_AWARE_WORKFLOW_SCHEDULER_NAME env, which the ARC fork applies to the ph-w-* workflow placeholder. Keeping both in sync ensures the placeholder reserves a slot the real pod can actually claim. Empty = default scheduler. This adds only the capability; opting specific defs in is done separately. ghstack-source-id: 9710623 Pull-Request: #804
jeanschmidt
approved these changes
Jun 24, 2026
Contributor
|
@claude can you explain this changes? How is the custom scheduler toggled on/off for particular clusters? |
|
Claude finished @jeanschmidt's task in 1m 27s —— View job Explanation: per-def workflow scheduler knobThis PR adds plumbing for a per-runner-def What the change does
Both are driven by the same value on purpose. The placeholder reserves a slot ahead of the real workflow pod; if the two used different schedulers, the placeholder could reserve a slot the real pod can't claim ( How it's toggled on/off — and the key caveatThe toggle is per def, not per cluster. In scheduler_name = runner.get("scheduler_name", "")
scheduler_name_line = f" schedulerName: {scheduler_name}" if scheduler_name else ""
The subtlety for your question: defs under Note on the "pilot" defsThe PR description says it opts |
Contributor
|
this change only allows you to deploy all at once, or none. Consider making it conditional on the existence of the custom scheduler module on cluster definitions |
georgehong
had a problem deploying
to
osdc-staging
GitHub Actions
Failure
huydhn
pushed a commit
to huydhn/pytorch-ci-infra
that referenced
this pull request
Jun 25, 2026
Stack from [ghstack](https://github.com/ezyang/ghstack/tree/0.14.0) (oldest at bottom): * __->__ pytorch#830 * pytorch#829 * pytorch#828 Add bin-pack-scheduler to the meta-staging-aws-ue1 module list so it deploys to staging. Split from the module (pytorch#803) and the scheduler_name knob (pytorch#804) so turning it on is its own revertible switch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
Add a per-def scheduler_name knob. generate_runners.py stamps it onto both the
real workflow pod (schedulerName) and the listener's
CAPACITY_AWARE_WORKFLOW_SCHEDULER_NAME env, which the ARC fork applies to the
ph-w-* workflow placeholder. Keeping both in sync ensures the placeholder
reserves a slot the real pod can actually claim. Empty = default scheduler.
Opt two CPU defs (l-x86iavx512-8-64, l-x86iavx2-8-32) into bin-pack-scheduler
as a pilot. Unit tests cover both the set and unset paths.