[earakely-scale]

What

agent_env.training.slime trains on a bundle's tasks with slime through slime's agent layer (slime.agent). The model plays the task, as with the Miles, verl and SkyRL helpers, on a bundle served as an OpenEnv env by agent_env.openenv.build_app.

• python -m agent_env.training.slime URL OUT writes the server's tasks as slime's prompt data, OUT/<split>.jsonl. The rows are the Miles helper's:

  • chat messages;
  • metadata with the episode, its tools and the server's settings.

• agent_env.training.slime.generate.generate is slime's generate function (--custom-generate-function-path). For each sample it:

  • opens a session on slime's OpenAIAdapter, which records the model's turns as training samples;
  • plays the row's episode through that session;
  • returns the session's turns as samples carrying the episode's reward, so slime runs no reward model.

  It shares the Miles helper's episode loop, so it also inherits:

  • non-object tool arguments answered as the model's error;
  • the model deadline;
  • flagging an episode the env or the session failed, or that ended without a reward.

• In an eval, generate returns one sample per episode carrying the reward, as slime's own agent examples do.

  • An episode otherwise comes back as a sample per turn when the adapter can't join a turn to the one before (Qwen3's thinking-off template).
  • slime's eval reward averages over samples, so without this it weighted episodes by their turns: 0.62 against the true per-episode 0.69 at step 40 of the run below.

• generate opts into slime's request-level abort (generate.abort_mode = "request").

  • When a step's batch is full, slime cancels the rollouts still playing.
  • Each drops its adapter session and closes its episode, rather than playing on against the aborted model and running its verifiers.

• generate.keep_scored_groups is the dynamic-sampling filter (--dynamic-sampling-filter-path). It leaves out a flagged episode's group, and an episode with no recorded turn comes back as one aborted sample with remove_sample.

• The adapter is one per rollout process, served in-process. It needs --sglang-tool-call-parser, so the helper refuses to start without one. Without a parser, every rollout would quietly score 0, as it did in Miles' first GPU run.

• The slime extra is the openenv extra under the trainer's name; install it into slime's own environment.

Shared with the Miles helper

The chat rows, the row-writer command and the episode played over a trainer's chat endpoint move from agent_env.training.miles into a private agent_env.training._chat, which both helpers use. The code is unchanged and the Miles helper's public names stay:

• miles.agent.run, reward and keep_scored_groups;
• miles.prepare.write_rows;
• python -m agent_env.training.miles.

Maintainability

• Fewer slime internals. The generate function's per-process adapter is cached on our own class, not through slime's SingletonMeta, and the tier resets it through that class rather than through slime's private state. The tokenizer still comes from slime's load_tokenizer, the helper slime's own examples build this adapter with. Importing transformers directly would need it declared in the slime extra, and that could move slime's environment off the transformers its SGLang pins.
• Shared test tasks. The tasks and model call the trainer tiers play with move into tst/unit/training/_support.py, beside the serve fixture: PLAYED, FAILS_AFTER_THE_TURN and ADDS_A_CRATE. Before, tst/miles and tst/slime each had a copy, and the unit tier and five trainer tiers imported them from nemo_gym_test. The unit test of the shared rows and episode is now chat_test.py.
• AGENTS.md. The training row of the architecture table, a single 6,661-character cell, becomes a "Training helpers" subsection with one entry per module. It adds harness.py, which had no entry.

slime versions

• Supported: slime main from 540ea91 (2026-10-27), the commit the GPU runs used.
• Not supported: v0.4.0, the latest release. Its adapter predates two things the helper passes: rollout_args, which main's adapter uses for score centering, and per-session apply_chat_template_kwargs, which thinking-off training needs. The CI tier caught this.

GPU runs

Setup:

• One g6e.12xlarge (4×L40S), with slimerl/slime:latest (main 540ea91) and agentenv-framework 0.9.1418 installed into it.
• GridSim served by build_app on the host's Docker.
• Qwen3-4B after the SFT warm start the other trainers' GridSim runs use, as a full fine-tune: slime trains only on Megatron, and its arguments have no LoRA. Megatron is colocated with SGLang, at lr 1e-6 constant.
• The Miles run's GRPO settings: 32 rollouts a step (4 days × 8), temperature 1.1, thinking off, at most 4 turns.
• The runs used an early version of this generate function: the released Miles agent function, given the slime session id as the request's user, inside a generate function of the same shape. This PR's code runs through slime's real adapter in tst/slime on CPU.

A 20-step run first:

Qwen3-4B, eval reward per episode on GridSim's 200 eval days	slime
Step 0 (after SFT)	0.578 (SE 0.025)
Step 10	0.595
Step 20	0.602 (SE 0.024)

Training updated (grad norm about 0.85 every step) but showed no measurable gain, as SkyRL, TRL and Unsloth didn't at 20 steps. A 40-step run, from a fresh SFT warm start:

Qwen3-4B, eval reward per episode on GridSim's 200 eval days	slime
Step 0 (after SFT)	0.514 (SE 0.027)
Step 20	0.615
Step 40	0.694 (SE 0.022)

• The gain is +0.18, about 5 SE. Schedules scoring 0 fell from 21% to 3%.
• By battery size: 1-hour 0.70→0.81, 2-hour 0.53→0.66, 4-hour 0.30→0.61.
• At 40 steps the other trainers' GridSim runs scored 0.57 to 0.67.
• A step took about 107 s: 31 s of rollouts and 58 s of training.
• There were no env failures and no malformed calls in 1,880 episodes.

What slime needed in those runs:

• --sglang-tool-call-parser qwen25.
• On L40S, --attention-backend unfused. The image has no FlashAttention-2:
  • flash uses FlashAttention 4's CuTe kernels, which don't compile on sm_89;
  • fused (cuDNN) doesn't take packed sequences there.
• --max-tokens-per-gpu 4096 and no --accumulate-allreduce-grads-in-fp32. At 8192 tokens with fp32 gradient accumulation the backward ran out of memory.
• Qwen3's template with thinking off renders the empty think block in the generation prompt but drops it from history. slime's adapter therefore can't join a turn to the one before, and returns a sample per turn (about 2.5 per episode). Training is unaffected: --global-batch-size counts rollouts, and a rollout's samples stay in one step. slime's own eval/eval averages per sample, so it weights episodes by their turn count; the tables above are per episode.

Tests

• tst/unit/training/miles_test.py covers the shared rows, episode, reward hook and filter. Its command test now runs both python -m agent_env.training.miles and python -m agent_env.training.slime.
• tst/slime (make slime-test) plays rows through the generate function on slime's own OpenAIAdapter. It uses slime's own parse_args, and a stand-in for SGLang's /generate with Qwen3's tokenizer. It checks:
  • a played row comes back as samples with the episode's reward, the model's tool call and reply trained and the tool's answer masked;
  • an episode the env failed is flagged, and slime's call_dynamic_filter leaves its group out;
  • an episode with no recorded turn comes back as one aborted sample that the filter leaves out;
  • an eval gives one sample per episode, with nothing to train and the session dropped;
  • a rollout cancelled mid-episode drops its session and closes its episode (a teardown);
  • the adapter refuses to start without --sglang-tool-call-parser.
• .github/workflows/slime.yml runs the tier on slime 540ea91 and main. slime isn't on PyPI, so the job installs:
  • slime's checkout, with its requirements and CPU torch;
  • SGLang's and Megatron-LM's checkouts on PYTHONPATH, at the SGLang tag and Megatron-LM commit slime's Dockerfile builds on;
  • the transformers that SGLang tag pins;
  • the few packages SGLang's tool-call parser and slime's argument parser import on CPU.
• The optional-extras test covers the slime extra and the shared _chat module.

Run on a Linux box:

• ruff;
• tst/unit/training, the extras test and tst/slime without slime: 151 passed, and the tier skipped on its capability;
• tst/slime against slime main 540ea91: 4 passed;
• after the eval and cancellation changes, against slime main 8a53b68: 6 passed. The cancellation case passed 4 of 4 runs on its own.

🤖 Generated with Claude Code

Confidence Score: 5/5

The PR appears safe to merge; no actionable new issue was found.

Summary

Adds slime training through its OpenAI adapter and an OpenEnv server.

• Slime can train on tasks served through OpenEnv.
• Miles and slime share task rows and episode play.
• Trainer tests use one shared set of sample tasks.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  Rows[Chat rows] --> Generate[slime generate]
  Generate --> Adapter[slime OpenAI adapter]
  Adapter --> Model[SGLang]
  Generate --> Episode[OpenEnv episode]
  Episode --> Reward[Episode reward]
  Reward --> Samples[Training samples or one evaluation sample]

Loading

_{Reviews (4) · Last reviewed commit: "fix(slime): load the adapter's tokenizer..." · Reviewed by Greptile}