[multi-lora] 3/n Tinker full stack: backend, frontend, and RL/SFT docs

[yushengsu-thu]

Summary

This is the documentation/integration leaf of the Multi-LoRA Tinker stack.

Because this PR formally targets main, GitHub's Files changed tab
includes the backend and frontend inherited from the first two PRs. The
logical review order is:

main
  └── #2273  operation backend and Multi-LoRA execution
        └── #2346  tinker==0.24.1 JSON/REST frontend
              └── #2365  client-owned SFT/RL quick start and full-stack guide

Relative to the current #2346 head, this PR is documentation-only:

• examples/multi_lora_operations/README.md is the source of truth.
• docs/examples/multi-lora-operations.md is its generated docs mirror.

No runtime or test code is unique to this layer.

Reviewer

client application
  ├── prompts and SFT examples
  ├── sampling decisions and parameters
  ├── rewards, scoring, and advantages
  ├── tinker.Datum construction
  └── sample → train → publish loop
                 │
                 │ official tinker==0.24.1 SDK
                 ▼
Tinker JSON/REST frontend (#2346)
  ├── sampling ───────────────────────────────> SGLang router
  └── ordered training requests
                 │
                 ▼
MultiLoraOperationBackend (#2273)
  ├── registration and operation ledger
  ├── DATA: MultiLoraOperationBatchFn
  │     └── forward / forward_backward → Megatron
  └── CONTROL
        ├── MultiLoraParameterExecutor
        │     └── adapter-slot step or poisoned-gradient discard
        ├── save/load training state
        └── explicit weight publication ──────> SGLang router

Miles executes ordered training operations and serves explicitly published
adapter weights. The application still owns its data, sampling loop, reward
function, scoring, advantages, and update schedule.

What this layer adds

The guide shows:

• how to prepare and start the shared training/sampling deployment;
• how to enable the official SDK endpoint with --tinker-frontend;
• the division of ownership between client code and Miles;
• pure SFT data preparation without server-owned datasets or rewards;
• the RL sequence sample → score → build Datum → forward_backward → optim_step → publish;
• why the sampling client is reacquired after each latest-only publication;
• where to find the complete runnable GRPO example.

Quick start:

python examples/multi_lora_operations/run_multi_lora_operations.py prepare

python examples/multi_lora_operations/run_multi_lora_operations.py serve \
  --extra-args "--tinker-frontend"

The full runnable RL version is
tests/e2e/tinker_frontend/tinker_sdk_rl_quality.py.

Validation

At current head 83b55d13c:

• Verified that [multi-lora] 2/n Tinker frontend: tinker==0.24.1 JSON/REST adapter #2346 head e87e4ef2d is an ancestor.
• The final-tree delta from [multi-lora] 2/n Tinker frontend: tinker==0.24.1 JSON/REST adapter #2346 is exactly the source README and its
  generated docs mirror: two files, +188/-2 total.
• git diff --check: passed.
• Pre-commit passed on the two effective files on an H200 devbox and in the
  current GitHub run.
• All selected Stage A/B CPU jobs pass. GPU jobs are skipped for this
  documentation-only leaf; runtime GPU/E2E evidence remains in [multi-lora] 1/n operation backend: explicit training with Tinker compatibility #2273 and
  [multi-lora] 2/n Tinker frontend: tinker==0.24.1 JSON/REST adapter #2346.

Scope and limits

• Official tinker==0.24.1 JSON core-loop subset, not the 0.25+ protobuf
  protocol.
• Text-only Multi-LoRA with cross_entropy, importance_sampling, and ppo.
• Sampling publication is latest-only; pinned off-policy sampler snapshots are
  not implemented.
• Client code supplies datasets, rewards, scoring, and Datum construction.
• FullParameterExecutor is an implemented but unwired optimizer-control seam
  from [multi-lora] 1/n operation backend: explicit training with Tinker compatibility #2273. Full-parameter launch, training flow, checkpointing,
  publication, and GPU/E2E support are outside this stack.

See #2273 and #2346 for the detailed backend and frontend contracts.

0821 liveness fixes merged forward from #2273/#2346: capped generate-failure tolerance in the driver, claimed-operation TTL backstop, FAILED child-runtime cooldown self-heal.

• 0821: merge-forward — publish-path api_backends import fix + import-integrity test ([multi-lora] 1/n operation backend: explicit training with Tinker compatibility #2273) and prompt-logprobs/compute_logprobs support ([multi-lora] 2/n Tinker frontend: tinker==0.24.1 JSON/REST adapter #2346); GPU mini-loop + logprob-parity PASS on this stack.