solx 1.0.2: fix job jump from inside another allocation#42
Merged
Conversation
A `solx job jump` run from within another allocation execs a nested
`srun` step that inherits the enclosing job's SLURM_* step-shaping
environment, which breaks on the target node: leaked SLURM_CPU_BIND
fails cpu binding ("Unable to satisfy cpu bind request"), SLURM_MEM_PER_*
fails step creation ("Memory required by task is not available"),
SLURM_NTASKS launches N tasks/shells, and SLURM_JOB_NUM_NODES warns
about spreading the shell across nodes.
Pin the pty step with `--nodes=1 --ntasks=1 --cpu-bind=none
--mem-bind=none --mem=0` — srun's command line overrides the inherited
env vars (each override verified on Sol's srun 25.11.6), the target
job's cgroup still confines the shell to its cpuset, and jumping from a
login node is unaffected. `job jump` is the only command that launches a
job step, so it is the only one exposed.
Also: derive the `job start` attach hint from the same argv builder so
it can't drift, and add a nested-jump integration test asserting the
env-independent argv plus the nesting notice.
Release: bump CLI + skill to 1.0.2, refresh Cargo.lock, move CHANGELOG
[Unreleased] -> [1.0.2], bump coverage.md.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR fixes solx job jump when invoked from inside another SLURM allocation by ensuring the srun --pty step is shaped by explicit CLI flags rather than inherited SLURM_* environment from the enclosing job. It also aligns the job start “attach” hint with the actual jump argv builder, adds an integration test to prevent regressions, and bumps release metadata to 1.0.2.
Changes:
- Pin
srun --ptywith--nodes=1 --ntasks=1 --cpu-bind=none --mem-bind=none --mem=0to neutralize leaked step-shapingSLURM_*vars for nested jumps. - Reuse the same
srun_pty_argvbuilder for thejob startattach hint to prevent drift. - Add CLI integration coverage for env-independence + nesting notice; bump versions/docs/changelog for 1.0.2.
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| solx/tests/cli.rs | Adds a shared expected srun string and an integration test covering nested-jump env leaks + nesting stderr notice. |
| solx/src/slurm.rs | Updates srun_pty_argv to include explicit anti-leak flags and documents the rationale; updates unit test accordingly. |
| solx/src/jobs.rs | Derives the printed attach hint from srun_pty_argv via shell_join so it matches jump behavior. |
| solx/Cargo.toml | Bumps crate version to 1.0.2. |
| solx/Cargo.lock | Updates lockfile’s solx package version to 1.0.2. |
| skills/sol-skill/SKILL.md | Bumps skill version to 1.0.2. |
| docs/solx.md | Documents the pinned srun flags and the nested-jump behavior/implications. |
| docs/coverage.md | Updates documented version to v1.0.2. |
| CHANGELOG.md | Adds 1.0.2 entry and updates compare/tag links. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Running
solx job jump <OTHER_JOBID>from inside an existing allocation fails. The reported symptom:job jumpexecssrun --ptyto open a shell on the target node. When thatsrunruns inside another allocation, it inherits the enclosing job'sSLURM_*step-shaping environment, which is wrong for the target job. Four independent leaks (all reproduced on Sol'ssrun25.11.6):SLURM_CPU_BINDUnable to satisfy cpu bind request(a cpuset from a different job)SLURM_MEM_PER_NODE/SLURM_MEM_PER_CPUMemory required by task is not available— hard step-creation failure (e.g. jumping from a--mem=64GGPU job)SLURM_NTASKS-n 16job starts 16; only task zero gets the pty)SLURM_JOB_NUM_NODESwarning: can't run 1 processes on 2 nodesFix
Pin the pty step so it derives its shape from the target job, not the enclosing one:
srun's precedence is command line > input env var, so each flag deterministically overrides its leak — each override verified on real Sol. The target job's cgroup still confines the shell to its cpuset, and jumping from a login node is unaffected (nothing to leak from).
job jumpis the only command that launches a job step, so it's the only one exposed —job start(new allocation viasalloc),stop,list,time, andkeep(direct filesystemtouch) are not.Residual, left as-is (no clean flag override):
SLURM_CPUS_PER_TASK(-chas no zero/none sentinel) and GRES vars (--gres=nonewould strip the shell's GPU cgroup access).Also in this PR
solx job startattach hint is now derived from the samesrun_pty_argvbuilder, so the printed command can't drift from whatjumpactually runs.SLURM_*vars and that the nesting notice fires.Cargo.lock, moved CHANGELOG[Unreleased]→[1.0.2], bumpeddocs/coverage.md.Verification
cargo test: 104 unit + 40 integration pass ·cargo clippy -D warningsclean ·cargo fmt --checkclean · release build clean.🤖 Generated with Claude Code