[droot]

Draft, for review. This branch stacks on #383 (the five commits of dashboard/main, rebased onto main at 0644063) and adds what came out of trying that PR on a fresh GKE cluster (open-rl-dra-use5, us-east5, 8 x H100, DRA). The write-up is in docs/briefs/pr383-dashboard-trial/findings.md; notes.md next to it is the step-by-step log with every number and screenshot.

Commits on top of #383:

• Record a sparse delta's density and size on the save op. The FFT trainer already computes how much of the model a sparse delta changed and returns density_pct from the save, but the command record keeps only result["metrics"], so the figure never reached the inspection API (finding 13 in the report). The worker now keeps last_save_metrics (delta/density_pct, delta/changed_elements, delta/total_elements, delta/layers, time/delta_collect, time/delta_write; empty after a full checkpoint), the processor attaches them to save results, and the API server drops them from the client's save responses, which the SDK's save types do not carry. /runs/{id}/steps and the dashboard's trainer charts show them with no change on the dashboard side. Two unit tests. Verified on the cluster: a three-step Qwen3-0.6B run reported 11.6 %, 7.8 % and 6.0 % of 596 M elements per step.
• e2e: RL scenarios ask the API server to start the sampler with the trainer. openrl.recipe=rl is what makes the API server launch the sampler alongside the trainer instead of on the first sample request; the harness never set it, so every RL scenario paid the sampler's start on its first step (about 20 s on an idle H100). Scenarios whose name carries rl or textsql now send it in TINKER_TAGS.
• Demo app and the use5 overlay (k8s/deploy/gke-use5/). A small in-cluster FastAPI app on the API server image, code from a ConfigMap, that starts and stops sweeps of RL client Jobs from a page and shows the fleet from Run telemetry and the dashboard #383's inspection API: Sweep (spec editor, start, stop, stop-all), Fleet (the keynote mock's scene, live), Timeline (one occupancy trace per accelerator or per job, other-jobs overlay, drag to select a window, hover details), Weights (the sparse delta per step). The overlay also carries what the trial needed around Run telemetry and the dashboard #383: a dedicated dashboard service account for Workload Identity, the DCGM exporter and three PodMonitorings (exporter, scheduler's openrl_gpu_held, sampler pods), spread placement for the scheduler. It is specific to one cluster and one registry; it is here so the review can see it, not to merge as is.

What the report says about #383, in one paragraph: the mechanism works as described (placements stamped, records written, every route answers, the UI renders); the items that need attention are goodput charging the server's dispatch wait to "client", a negative since_last_step_s from the cached snapshot, the client-directory depth missing the repo's own e2e layout, nothing scraping openrl_gpu_held, no DCGM exporter in the bundle the dashboard ships with (and GKE's managed one skipping DRA pools), GKE IAM undocumented, node inventory only from DCGM, a trailing empty step, duty on a shared GPU shown per lane, a run whose client died on a 400 still reading as healthy, workers sanitising NaN to 0 before telemetry sees it, placement records per run on a shared runtime and 30 s behind, and the delta density above. Two placement observations about OpenRL itself (binpack co-locating trainer and sampler with idle GPUs around; a shared runtime's halves placed independently, so a trainer can hold a GPU idle while its sampler waits) are in the report as well.