Skip to content

Pool-backed engine roles (#7 phase 2, DT side) - #18

Merged
andre-merzky merged 1 commit into
develfrom
feature/dt-pool-wiring
Aug 24, 2026
Merged

Pool-backed engine roles (#7 phase 2, DT side)#18
andre-merzky merged 1 commit into
develfrom
feature/dt-pool-wiring

Conversation

@andre-merzky

Copy link
Copy Markdown
Member

The DT half of #7 phase 2. Companion PRs: radical-cybertools/radical.orbit#117 (dispatcher speaks the rhapsody execution dialect, batched persists/notifications) and radical-cybertools/rhapsody#88 (OrbitExecutionBackend pool mode).

Config: a role may name a dispatcher-managed pool instead of an endpoint, with the pool configs declared once at session level:

ENGINES = {
  "pools":   [{"name": "exsitu", "endpoint_name": "hpc1", ...}],   # dispatcher PoolConfig schema
  "engines": {"inference": {"endpoint_name": "dt_inference_ep"},
              "learning":  {"pool": "exsitu"}},
}

Semantics:

  • one dispatcher session per DT session, keyed by the DT sid and shared by every pool-backed role; pools are declared with it (idempotent re-declare, so role build order does not matter)
  • pool-backed roles opt out of the R8 endpoint-loss fail-fast: pilot loss requeues tasks inside the dispatcher; the twin's failure surface is a task that ultimately fails
  • contract: requeued tasks re-execute — twin tasks stay idempotent (documented at the seam and in register_session)
  • a pool config against a rhapsody without pool support fails the engine build with a clear error; without a pool, nothing changes — CI stays green on the current pins

Tests: pool kwargs land on the backend (dispatcher plugin, shared sid, declared pools, pool:<name> endpoint record), the old-rhapsody guard, and R8 sparing pool-backed roles. 171 unit + 29 integration pass.

Merge order: after the two companion PRs land, the CI/install pins move to versions carrying them; until then the pool path is config-gated and inert.

🤖 Generated with Claude Code

https://claude.ai/code/session_016Npyz3Hbnwos12ESsdJ2YU

`{"pool": "<name>"}` in a role's engine config routes that role's tasks
through the task dispatcher's pilot pools: the backend targets the
broker-hosted dispatcher (rhapsody's pool mode), one dispatcher session
per DT session -- keyed by the DT sid, shared by every pool-backed role,
with the session-level `pools` configs declared on it (re-declaring is
idempotent, so role build order does not matter).

Pool-backed roles opt out of the R8 endpoint-loss fail-fast: their
`_endpoints` entry is `pool:<name>`, which no lost endpoint ever
matches.  Losing a pilot's endpoint requeues its tasks inside the
dispatcher; the twin's failure surface for pools is a task that
ultimately fails.  The requeue contract makes twin tasks re-executable,
so they must stay idempotent -- noted at the seam and in the
register_session docs.

A pool config against a rhapsody without pool support raises a clear
error at engine build; without a pool nothing changes.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016Npyz3Hbnwos12ESsdJ2YU
@andre-merzky
andre-merzky merged commit a28e661 into devel Aug 24, 2026
2 checks passed
@andre-merzky
andre-merzky deleted the feature/dt-pool-wiring branch August 24, 2026 23:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant