k8s/deploy/demo: live lights, an idle fleet view, and the recipe on t…

k8s/deploy/demo/README.md

@@ -26,18 +26,28 @@ a Job is matched to its run.
   presets. Start submits one client Job per row and count, each running the
   repo's `fft-gsm8k-rl` or `lora-gsm8k-rl` scenario with
   `TINKER_TAGS=openrl.recipe=rl`. Stop releases the run's workers through
-  `delete_model` and deletes the Job. Jobs are listed newest first; sweeps get
-  a letter (A is the newest) so job numbers stay unambiguous across sweeps.
-- **Fleet.** Counts, the job table with step progress, and the pool: a box
-  per node with one row per GPU and a chip per worker holding it. A chip is
-  lit while its worker is computing (a time-slicer turn, or a shared LoRA
-  runtime serving one of its runs) and soft while it waits for its turn.
-  Below the pool: the queue (jobs without a GPU yet; outlined while the pod
-  is still starting) and the share of the pool's GPU time held by a job since
-  the first submit, computed from the placements the server recorded. Click
-  a GPU row or a job row to open it on the Timeline tab. Counts, bars and
-  states move to each new poll instead of jumping; a worker that gets a GPU
-  flies from its job's tag in the queue to its row.
+  `delete_model` and deletes the Job. Under the spec, cards show the recipe
+  every job runs (GSM8K, GRPO, the batch, sampling and learning rates, from
+  `RECIPE` in `app.py`, the same settings the Jobs get) and sum up the spec as
+  it is edited. The job list below is collapsed until opened; jobs are listed
+  newest first, and sweeps get a letter (A is the newest) so job numbers stay
+  unambiguous across sweeps.
+- **Fleet.** A headline (how many jobs share how many GPUs), counts (running,
+  the queue while anything waits, done, steps done), the job table, and the
+  pool: a box per node with one row per GPU and one chip per job a worker
+  serves, so the jobs on a shared LoRA runtime show one by one like full
+  fine-tuning workers. A chip is lit while that job's work is on the GPU (a
+  time-slicer turn, or the job's ops on a shared runtime) and outlined while
+  it waits for its turn. Below the pool: the queue (only while a job waits)
+  and the share of the pool's GPU time held by a job since the first submit,
+  computed from the placements the server recorded. Click a GPU row or a job
+  row to open it on the Timeline tab. Counts, bars and states move to each
+  new poll instead of jumping; a worker that gets a GPU flies from its job's
+  tag in the queue to its row. The sweep selector opens on no sweep: the pool
+  with no job's counts or chips, any worker holding a GPU drawn as a neutral
+  chip, the shot before a demo starts. Starting a sweep switches it to the
+  active sweeps; it also offers every sweep or one sweep, and `?sweep=active`,
+  `?sweep=all` or `?sweep=<run>` opens on those.
 - **Timeline.** One occupancy trace per GPU: 1 while a worker is on it,
   coloured by the holder, 0 when idle; the thin band at the base is a worker
   that holds the GPU between time-slicer turns. The default window, fit,
@@ -55,6 +65,13 @@ a Job is matched to its run.
   top, or one job alone. Read from the `metrics.jsonl` the client writes to
   the log directory the app assigned it (`RUNS_ROOT/<job>`).
 
+## Live state
+
+The lights and the timeline's current turn come from `/api/v1/inspect/now`,
+polled twice a second, which serves each worker's state as it wrote it the
+moment it changed. On an API server without that endpoint, the page replays
+the activity history instead, about 8 s behind.
+
 ## Presenting
 
 Press `p` (or open `/?present=1#fleet`) for presentation mode: the canvas

k8s/deploy/demo/app.py

@@ -46,8 +46,22 @@
 RUN_LABEL = "openrl.io/demo-run"
 HERE = Path(__file__).parent
 
+# Every client Job runs the e2e script's GSM8K RL scenario (tinker_cookbook's math_rl recipe) with these settings;
+# the Sweep tab shows RECIPE under the spec, so what it says is what the Jobs run.
+GROUP_SIZE, GROUPS_PER_BATCH, MAX_TOKENS = 8, 8, 512
+RECIPE = {
+  "dataset": "GSM8K",
+  "group_size": GROUP_SIZE,
+  "groups_per_batch": GROUPS_PER_BATCH,
+  "max_tokens": MAX_TOKENS,
+  "temperature": 1.0,  # run_training_e2e.run_gsm8k_rl
+  "learning_rate": {"lora": "1e-4", "fft": "1e-5"},  # run_training_e2e.run_gsm8k_rl
+  "lora_rank": 32,  # tinker_cookbook math_rl CLIConfig default
+  "loss": "importance sampling",  # tinker_cookbook math_rl CLIConfig default, with kl_penalty_coef 0
+}
+
 PRESETS = {
-  "keynote-20": {
+  "mixed-20": {
     "steps": 4,
     "jobs": [
       {"kind": "lora", "model": "google/gemma-4-e2b", "count": 5, "steps": 8},
@@ -56,13 +70,6 @@
       {"kind": "fft", "model": "Qwen/Qwen3-0.6B", "count": 5, "steps": 4},
     ],
   },
-  "qwen-8": {
-    "steps": 5,
-    "jobs": [
-      {"kind": "lora", "model": "Qwen/Qwen3-0.6B", "count": 4, "steps": 5},
-      {"kind": "fft", "model": "Qwen/Qwen3-0.6B", "count": 4, "steps": 5},
-    ],
-  },
   "smoke-2": {"steps": 2, "jobs": [{"kind": "fft", "model": "Qwen/Qwen3-0.6B", "count": 2, "steps": 2}]},
 }
 
@@ -111,9 +118,9 @@ def job_manifest(run: str, job: str, kind: str, model: str, steps: int, seed: in
                 f"log_dir={RUNS_ROOT}/{job}",
                 f"base_model={model}",
                 f"steps={steps}",
-                "group_size=8",
-                "groups_per_batch=8",
-                "max_tokens=512",
+                f"group_size={GROUP_SIZE}",
+                f"groups_per_batch={GROUPS_PER_BATCH}",
+                f"max_tokens={MAX_TOKENS}",
                 f"extra=seed={seed}",
               ],
               # openrl.recipe=rl makes the API server start the sampler with the trainer instead of on the first sample
@@ -319,7 +326,7 @@ async def metrics(jobs: str):
 
 @app.get("/api/sweep/presets")
 async def presets():
-  return {"presets": PRESETS, "namespace": NAMESPACE, "cluster": CLUSTER_NAME}
+  return {"presets": PRESETS, "recipe": RECIPE, "namespace": NAMESPACE, "cluster": CLUSTER_NAME}
 
 
 @app.get("/api/sweep/jobs")
@@ -329,7 +336,7 @@ async def jobs():
 
 @app.post("/api/sweep/start")
 async def start(body: dict):
-  run = re.sub(r"[^a-z0-9-]", "-", str(body.get("run") or time.strftime("demo-%H%M%S")).lower())[:30].strip("-")
+  run = re.sub(r"[^a-z0-9-]", "-", str(body.get("run") or time.strftime("sweep-%H%M%S")).lower())[:30].strip("-")
   specs = body.get("jobs") or []
   if not specs:
     raise HTTPException(400, "no jobs")