Skip to content

perf(deploy): pre-pull the worker image on every GPU node - #373

Draft
hodaaaaaaaaaa wants to merge 1 commit into
openrl-dev:mainfrom
hodaaaaaaaaaa:perf/worker-image-prepull
Draft

hodaaaaaaaaaa wants to merge 1 commit into
openrl-dev:mainfrom
hodaaaaaaaaaa:perf/worker-image-prepull

Conversation

@hodaaaaaaaaaa

Copy link
Copy Markdown
Collaborator

Problem

A new worker on a node without the worker image pulls it first, several GB that take minutes, and a cold job pays it for its trainer and its sampler.

Change

The base bundle adds the open-rl-worker-prepull DaemonSet, which pulls the worker image onto every GPU node with openrl.io/enabled=true at deploy time and only sleeps. The kustomize images transformer rewrites its image like every other OpenRL image, so a release that bumps the tag re-pulls on every node. The kind overlays drop it, since their workers pull from a cluster-local registry.

Based on droot's prototype in #343.

Results

Measured on GKE with eight GSM8K GRPO jobs launched together (gemma-4-E2B and Qwen3-0.6B, LoRA and full fine-tuning, A100 80GB and L4), with the worker image removed from every node before each run, two runs each:

  • A cold worker image pull took about 3 minutes per worker.
  • With the DaemonSet rolled out first, every job reached its first training step 330–400 s sooner, and the whole workload took 749 s instead of 1111 s.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant