DTaaS viz: live + replay dashboard (Explorer and standalone) and architecture figure - #3
Merged
Conversation
Four role lanes on one canvas: the client with a sub-lane per session, the broker with one card per twin, and the task and ex-situ endpoints grouped under an HPC-resources frame. The lanes are roles rather than hosts, so all four are drawn even when a deployment collapses them onto one endpoint. The data layer takes a stream of timestamped frames and folds them into one world -- snapshots of `admin/sessions` and the gateway's SSE notifications -- so it cannot tell a recording from a live stack. This commit brings only the recording half: `index.html` plays the bundled `dt_sample.js` on load and loops it, and play/pause and the speed slider act on the frames rather than on an animation drawn from them. The sample is a real capture of a live stack (one broker hosting `dt`, two rhapsody endpoints): a ROSE learner twin converging on its `rmse` criterion, an in-situ twin created and closed mid-recording, and a twin whose component crashes. It is a .js file assigning one JSON object for exactly one reason -- a classic <script src> loads from `file://` where `fetch()` does not, which is what makes the page self-demo with nothing running and no server. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The data layer gains its second source: a 1 Hz `admin/sessions` poll and
the gateway's SSE feed, turned into the frames the replay half already
consumes -- so `rec` captures the live stream to a recording that plays
back through exactly the same model, and none of the renderer knows which
it is watching.
Live has to be same-origin with the broker, and that is not a choice.
The gateway's CORS allow-list holds a handful of localhost origins, and
the `orbit_broker_token` cookie an EventSource rides is SameSite=Strict:
a page opened from anywhere else cannot reach a live broker even holding
the right token. So the plugin serves the page itself, at
`{namespace}/ui`, from an allow-list of assets -- `{asset}` is a
client-supplied path segment.
Two things the dashboard needs that nothing else did. A learner now
mirrors its per-window criterion into a filtered, read-only `metrics`
dict -- value, threshold, the operator between them, whether the window
met it, the window count and a bounded history, and never the model,
which can be megabytes -- which `DTRuntime.metrics()` collects off the
graph by duck typing (the runtime must not have to know about ROSE) and
`twin_list` / `admin/sessions` carry per twin. And a session summary now
names the endpoint behind each engine role, so the two HPC lanes can say
which hardware they are, and the ex-situ one can say when it aliases the
task one.
The create / destroy / state-change verbs on the arcs stay *inferred*
from the delta between two polls: in v1 nothing on the wire announces
them, and `twin_list` polling is the documented observation mechanism.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
ORBIT's plugin-UI machinery turns out to support arbitrary plugin JS: a
`Plugin.ui_module` path is read by `BrokerPluginHost.get_ui_modules()` and
served by the gateway at `/plugins/<plugin_name>.js`, which the Explorer
imports by convention and drops into a page of its own -- same origin,
same scope, no iframe and no CSP, so a canvas is entirely fine. So the
`dt` plugin declares one.
`dt_explorer.js` is an adapter, not a second dashboard: it hands the
Explorer a template and then dynamically imports the very file the
standalone page loads with a <script src> (it has no imports and no
exports, so it is valid either way) and calls the same `mount()`, in
compact mode and live -- the Explorer page is already same-origin with
the broker, so the auth cookie is simply there. It does not use the
Explorer's `onNotification` hook: that hook only delivers *this* plugin's
events, and the simulation tiles are rhapsody's, so the dashboard keeps
its own EventSource in both hosts.
One limitation to know about, since ORBIT's own docs present `ui_config`
and `ui_module` as interchangeable: `ui_module` is read on the broker
plugin host only. Endpoint-hosted, the Explorer falls back to the
declarative `ui_config` tile and the page has to be opened at
`{namespace}/ui` instead. The gateway also caches a plugin's JS for the
life of the broker process, which the plugin's own `ui/` route does not.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
`round(x, 6)` counts decimal places, so a criterion threshold of 1e-8 -- an ordinary target -- reached the wire as 0.0, and the dashboard drew its target tick at the bottom of the track. Six *significant* figures instead, in the value and the history alike, and infinities and NaN drop out rather than becoming invalid JSON. The history now stops at 24 to match what the sparkline actually keeps. Three things that had no ceiling: - `rec` grew until the tab died. Under the orbit data plane every stream message is an event on the same feed, so a recording left running on a chatty twin is unbounded by construction. It now stops at a frame cap and hands over what it captured, and a stream frame keeps its topic but not its payload -- the topic is what draws the pulse; the payload is a cloudpickled blob no replay can read. - A task whose terminal event fell into an SSE reconnect gap pulsed RUNNING forever, holding its tile and its place in the tally. Tasks are now aged out on silence, which trades a task running longer than the TTL going undrawn for a slot that always comes back. - The Explorer removes a page's node on disconnect without telling the module that drew it, so every reconnect cycle left another poll, EventSource and RAF loop running against a canvas nobody can see. The frame loop notices it has been detached and tears itself down. Checked against the real thing: one dashboard polls 6 times in 6 s, and so does one that has been through a disconnect/reconnect cycle. The poll was a `setInterval` with nothing stopping two from overlapping; out-of-order snapshots read as twins vanishing and coming straight back, complete with inferred arcs. It chains from completion now, carries a request deadline, and drops a response that lands after its source was replaced. A CLOSED EventSource is re-opened with backoff -- the browser only reconnects one that is still CONNECTING, and a dashboard that has silently stopped seeing tasks looks like an idle service. Also: the stream topic's terminator is a NUL, not the pipe the pulse parser was stripping, so a dtype label kept a stray character; one synthetic `dt_stream` frame in the sample (labelled as such -- the recorded stack ran the zmq plane) now pins that path; percent-encoded traversal cases on the asset route; a note against putting a broker token in a URL; and a comment recording that a `Response` really is handled by all three dispatch paths, since this is the one route here that does not return JSON. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Deck-native dark idiom, level colors per the AmSC layer stack (L5 DT / L4 ROSE / L3 AsyncFlow / L2.5 ORBIT / L2 Rhapsody / L1 resources), roles drawn as separate hosts. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The dashboard used to guess which twin a task belonged to: the events carry a uid, an endpoint and nothing else, so attribution went through candidate narrowing, a session block and finally a drawn brace. It could also lie -- a task whose twin stopped before its event arrived was confidently credited to a sibling. Ownership is now recorded where the task is submitted. The runtime hooks its engine once and keeps a bounded ring of task uids per twin, a ContextVar carries the owning twin into every child task, and an instance wrapper on the learner's task registration covers the ROSE side. twin_list reports the ring, and the dashboard joins events against it: 99.6% of task arcs in the bundled trace, 100% in the multi-session one, where every arc used to start at the broker frame. That deletes the guessing subsystem entirely, and the dashboard shrinks by about 550 lines. Arcs anchor two thirds down a card and bow into the empty lane below the top-aligned grid, so a trail belongs visibly to one twin. Lanes now resolve per session, which stops a second session's ex-situ traffic from drawing on the task lane. A stale copy of the dashboard cost a debugging round, so the header carries its version and the page busts its own cache, with a test keeping the two in step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Every engine's `OrbitExecutionBackend` registers with the broker as a participant, and the broker draws it in the topology -- until now as an anonymous `rhapsody.<uuid>`, two of them per dual-engine session, unexplained. Now the session names them for what they are: `rhapsody.<session>.<role>`. Uniqueness holds by construction (one engine per role per session; `engine` caches and `_lost` forbids rebuilds), and the kwarg is signature-guarded so a rhapsody without the new `participant_name` parameter keeps working unchanged. The rhapsody half is fix/orbit-participant-name @ 6b00e14. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Everything a host needs to join the live demo, checked in where the
demo lives:
deploy/install.sh one pinned install for every role. Python 3.12
on every host (the wire contract pins the
minor); asyncflow, rhapsody[telemetry], ROSE
pinned as git installs -- naive PyPI resolution
yields a rhapsody without the orbit backend, an
asyncflow without the non-main-thread fix, and
an unrelated package named `rose`; orbit rides
its 0.5.0 release. A fail-fast sanity block
proves all of it at install time.
deploy/run-broker.sh broker + dt plugin; exports its own broker URL
(twin creation needs it) and the orbit data
plane (the dashboard's pulses need it).
deploy/run-endpoint.sh one rhapsody endpoint per role, tagged so task
bodies can report where they ran.
test/12-dtaas-live/ the narrated two-phase driver (build, walk
away, reattach with only the sid, tear down --
session included), its twin components, and the
operator runbook with the failure table the
rehearsals actually filled.
Rehearsed end to end against a remote broker: twin round trip, ex-situ
learning across two endpoints, reattach, teardown to an empty service.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
andre-merzky
force-pushed
the
feature/dtaas-viz
branch
from
August 19, 2026 10:29
8d96584 to
2bca62d
Compare
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
andre-merzky
force-pushed
the
feature/dtaas-viz
branch
from
August 19, 2026 11:09
c7683dd to
d053489
Compare
Two topology-legibility fixes in one server round:
* Stream clients on the orbit data plane registered as anonymous
`dt_stream.<uuid>` consumers -- one per twin, unexplained in the
Explorer. Now they are `dt_stream.<twin-prefix>.<suffix>` with the
advertised role `stream`: matchable to a dashboard card at a glance.
The suffix stays random because a namespace can legitimately carry
two clients (the twin's own, plus a consumer which opened the twin's
reported config).
* The dashboard page at `{namespace}/ui` (no trailing slash) resolved
its relative script against the parent path and fell back to "did
not load". The assets now answer next to the page as well as under
it, so both spellings work; the runbook links the canonical one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Twin B moves from step 6 to step 5: its first training window (~15s after start) then lands inside the ten-call query span, and one talking window shows the task-lane arcs, the exsitu tiles and the convergence bar together. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The first cut read the asset name off `request.url`, which exists on the ASGI path and not on the broker-hosted one: `BrokerPluginHost` hands a request shim, and the route 500ed exactly where the demo serves it. Caught on a live broker, not by the unit suite -- the shim dispatch has no test double here yet. The name is per-route anyway, so the handler now closes over it and touches the request not at all. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… looks Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Collaborator
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Live + replay visualization dashboard for the DT service, plus a slide-ready architecture figure.
Stacked on #6 (M3) → #5 → #4 → #3 — review from commit
58d4be4onward, or merge in order.What's in here
src/digitaltwin/service/ui/dt_dash.js(vanilla, no build step, canvas idiom after the AmSC deck's flow diagrams) mounted by a standalone page (works fromfile://, self-demos with a bundled real 61 s recording) and by the ORBIT Explorer viaPluginDT.ui_module(verified end-to-end in headless Chrome against a live broker).admin/sessionspoll (chained, abort-bounded, ordered) + SSE with reconnect backoff and task TTLs; record button captures a bounded replay file (stream payloads stripped); replay uses the play/pause/speed controls.metricsdict fed from ROSE'sIterationState(significant-figure rounding — a1e-8threshold survives the wire), per-sessionendpointsmap, pluginuiroutes with an allow-list (encoded-traversal test-pinned).docs/dtaas-architecture.svg— high-level architecture in the deck's idiom with consistent per-layer colors.Internally reviewed (request-changes round applied in full; security review of the new routes clean). Upstream notes recorded in the README: task→twin attribution needs a rhapsody payload field;
ui_moduleis broker-hosted-only and ORBIT'sui_config/ui_moduledocs conflate the two; gateway sessions carry nox-orbit-srcowner.🤖 Generated with Claude Code