[multi-lora] 1/n operation backend: explicit training with Tinker compatibility

[yushengsu-thu]

Summary

Replaces the old dataset-driven Multi-LoRA path with a service-only, client-driven operation backend. Multiple clients share one Megatron base model through fixed LoRA slots and explicitly control forward/backward, optimizer, checkpoint, and sampler-publication boundaries.

This PR provides the backend and Ray-side operation contract. Stacked PR #2346 adds the official tinker==0.24.1 JSON/REST frontend, and #2365 adds the complete client-owned SFT/RL guide.

Reviewer

Tinker SDK client
    │
    │ JSON/REST translation (#2346; not part of this PR)
    ▼
TinkerFrontendHTTPServer (#2346)
    │ calls the backend inside the head-node controller actor
    ▼
MultiLoraOperationController
    └── MultiLoraOperationBackend
        ├── AdapterRegistry
        │   └── fixed slot for (name, registration_id)
        ├── OperationLedger
        │   └── ordering, retries, gaps, ACK/backpressure, and fencing
        ├── FixedSlotResidency
        │   └── claim-and-bind → immutable BatchExecutionLease
        ├── GradientWindowTracker
        └── RouterInferenceAdmin

train_multi_lora_operations.py
    ├── DATA: forward / forward_backward
    │   └── MultiLoraOperationBatchFn
    │       ├── claim one whole client batch per ready registration
    │       ├── coalesce only the same operation kind
    │       ├── acquire one BatchExecutionLease
    │       ├── RolloutManager conversion + zero-weight DP padding
    │       └── Megatron forward[/backward]
    │           └── per-operation results → commit or poison
    │
    └── CONTROL
        ├── optim_step
        │   └── run_optim_controls(ParameterExecutor)
        │       ├── MultiLoraParameterExecutor               [wired]
        │       │   ├── validate registration + physical slot from lease
        │       │   ├── discard a poisoned gradient window
        │       │   └── slot-sorted, per-call Adam step
        │       │       ├── all-rank non-finite veto
        │       │       └── norm-blind veto + gradient cleanup
        │       │
        │       └── FullParameterExecutor                    [seam only;
        │                                                     not wired]
        │
        └── save_state / load_state / save_weights_for_sampler
            └── Multi-LoRA trainer handlers
                ├── adapter + optimizer state shards
                ├── ownership/shape-fenced restore
                └── ActorGroupWeightUpdater → SGLang
                    └── publish completes only after weights are live

MultiLoraParameterExecutor is deliberately narrower than the whole backend: it owns optimizer step/discard for lease-bound adapter slots. Data conversion and forward/backward live in MultiLoraOperationBatchFn; checkpoint and publication remain Multi-LoRA trainer controls. create_rollout_components() also exposes separate InferenceControllerPort and RolloutExecutorPort roles. Today they are two adapters over the same combined RolloutManager; the physical controller/executor split remains future integration work.

Operation semantics

Supported operations are:

• forward
• forward_backward
• optim_step
• save_weights_for_sampler
• save_state
• load_state

Important invariants:

• Operations execute in strict ordinal order per registration, while out-of-order arrival is gap-buffered.
• Identical retries are idempotent; reused identities with different content are conflicts.
• (name, registration_id) prevents stale handles from targeting a re-registered adapter with the same display name.
• Claim-and-bind produces one immutable execution lease, which trainer ranks validate before physical mutation.
• forward_backward calls accumulate one client-owned gradient window. A failed batch poisons that window, and the next optim_step discards it instead of applying partial gradients.
• Results remain available until ACK; capacity pressure rejects new work instead of evicting terminal results.
• Sampler publication completes only after staged weights are live.

Sampling, scoring, rewards, advantages, batch scheduling, and SDK Datum
construction remain client-owned.

Full-parameter seam

The reusable seam is intentionally narrower than the complete backend:

• BatchExecutionLease
• TrainerResidencyPort
• ParameterExecutor
• run_optim_controls

FullParameterExecutor implements singleton whole-model Adam step/discard
behavior against a stock Megatron optimizer. It is not connected to launch
configuration, registration, data conversion, forward/backward execution,
checkpointing, controller routing, or sampler publication. This PR therefore
does not claim working full-parameter SFT/RL or full-parameter GPU/E2E support.

Validation

Current head 7189b1e54 has:

• 29 successful GitHub checks, 2 expected skips, and no failures or
  pending checks.
• CPU test shards, pre-commit, and CodeQL.
• 2/4/8-GPU H200, 8-GPU H100, and 4-GPU MI350 CI coverage.
• Development-time 2×H200 acceptance covering Qwen SFT/RL, GPT-OSS
  multi-adapter training/checkpointing, sampler publication, restore,
  registration fencing, and slot reuse.

Limits

• Text-only, 1-D shifted targets with cross_entropy,
  importance_sampling, or ppo.
• Fixed adapter slots; no eviction, idle-slot GC, or per-tenant quota.
• Megatron backend with Adam-family per-slot optimization.
• Disaggregated execution only; pipeline parallelism must be 1 and
  qkv_format must be thd.
• Latest-only sampler publication; no immutable version-pinned snapshots.
• Restore rejects incompatible world topology, LoRA shape, or per-rank
  optimizer ownership.
• Recovery of work lost after claim during downstream process death is not
  guaranteed; executor-side reconciliation remains future work.

Stack and dependencies

• [multi-lora] 2/n Tinker frontend: tinker==0.24.1 JSON/REST adapter #2346 — tinker==0.24.1 JSON/REST frontend.
• [multi-lora] 3/n Tinker full stack: backend, frontend, and RL/SFT docs #2365 — full-stack SFT/RL examples and documentation.
• [optimizer] Fix LayerWise expert ownership Megatron-LM#82 — LayerWise expert ownership; merged into
  miles-main.
• peft: fix multi-LoRA activation-recompute replay and grouped-expert adapter DDP flagging Megatron-Bridge#27 — Multi-LoRA recompute and grouped-expert
  support; merged into the tracked bridge branch.

0821 liveness fixes: capped tolerance for consecutive generate failures (--multi-lora-max-consecutive-generate-failures), a claimed-operation TTL backstop that unblocks orphaned CLAIMED queue heads (--tinker-operation-claimed-ttl), and FAILED child runtimes self-healing to IDLE after a cooldown.

• 0821: fixed the publish-path function-local import stranded by the api_backends regrouping (update_weight .../mixin.py → api_backends.multi_lora.model) + tests/fast/test_import_integrity.py static import-integrity regression; 2-GPU mini-loop re-verified publish+sample (PASS).

• 0822 test cleanup: launch-unreachable multi-LoRA tests removed (stamped-slot fallback deleted from production, now a loud ValueError), duplicate trim test dropped, sglang abort regression moved to its own PR [multi-lora] 1/n - 7, test: pin update_weight_version abort_all_requests=False regression #2713, RL-quality harness now exits non-zero on aborted loops.

[yushengsu-thu]

clean

[yushengsu-thu]

@claude review