Add SGLang-Omni RL rollout integration - #2

Closed

Hayden727 wants to merge 24 commits into

mainfrom

hayden/omni-rl-clean-20260708

Commits on Jul 8, 2026

feat: add omni rollout contract and audio encode for sglang-omni RL
Hayden727
committed
041ad2d
fix(omni): harden OmniGenerateFn budget/audio handling + audio_data wiring
Hayden727
committed
db08da2
fix(omni): partial-rollout loss_mask, call_processor audio route, unified audio keys
Hayden727
committed
e183512
feat(omni): thinker-submodule checkpoint extractor for FSDP training
Hayden727
committed
f450d5b
feat(omni): text-only math reward for GATE-A thinker RL smoke
Hayden727
committed
0d82cfc
feat(omni): GATE-A closed-loop GRPO LoRA smoke + math dataset
Hayden727
committed
1230255
feat(omni): full on-policy GATE-A with NCCL weight-sync to served thinker
Hayden727
committed
6e6626d
fix(omni): bind CUDA device before NCCL collectives in full GATE-A
Hayden727
committed
2a72ca2
feat(omni): make weight-sync GROUP_NAME env-configurable
Hayden727
committed
9a7f7bf
docs(omni): record NCCL_P2P_DISABLE fix that makes GATE-A weight-sync work
Hayden727
committed
059c28d
feat(omni): composite TTS reward (ASR CER + audio guards) for GATE-B
Hayden727
committed
76d75bc
feat(omni): GATE-B rollout->reward->advantage loop on real Higgs TTS
Hayden727
committed
074b5e7
feat(omni): trainer-side Higgs TTS actor + logprob-parity gate
Hayden727
committed
285340b
feat(omni): GATE-B full on-policy loop (GRPO + NCCL tts_engine weight-sync)
Hayden727
committed
edf87ea
feat(omni): parametrize GATE-B rollout temperature/max_new_tokens
Hayden727
committed
e5ae79a
chore(omni): add corrected higgs codec-logprob patch (supersedes broken copy)
Hayden727
committed
1d3a32a
chore(omni): remove stale broken higgs codec-logprob patch
Hayden727
committed
7a358d5
feat(omni): non-saturating eval sets + MAX_NEW knob for GATE-A/B learning runs
Hayden727
committed
8b0521c
refactor(omni): rename gate_a/gate_b examples to function-descriptive names
Hayden727
committed
69764df
refactor(omni): scrub GATE-A/GATE-B terminology + fix internal path refs
Hayden727
committed
dd5d2ad
feat(rollout): wire --rollout-external to use an external sglang server
Hayden727
committed
d55adfc
refactor(omni): consume Higgs rollout contract
Hayden727
committed
3186a5a
fix(omni): preserve external rollout metadata paths
Hayden727
committed
aae8700

Commits on Jul 11, 2026

fix(omni): train all Higgs codebook actions
Hayden727
committed
7b8fb14

Navigation Menu

Add SGLang-Omni RL rollout integration - #2

Add SGLang-Omni RL rollout integration#2
Hayden727 wants to merge 24 commits into
mainfrom
hayden/omni-rl-clean-20260708

Commits on Jul 8, 2026

feat: add omni rollout contract and audio encode for sglang-omni RL

fix(omni): harden OmniGenerateFn budget/audio handling + audio_data wiring

fix(omni): partial-rollout loss_mask, call_processor audio route, unified audio keys

feat(omni): thinker-submodule checkpoint extractor for FSDP training

feat(omni): text-only math reward for GATE-A thinker RL smoke

feat(omni): GATE-A closed-loop GRPO LoRA smoke + math dataset

feat(omni): full on-policy GATE-A with NCCL weight-sync to served thinker

fix(omni): bind CUDA device before NCCL collectives in full GATE-A

feat(omni): make weight-sync GROUP_NAME env-configurable

docs(omni): record NCCL_P2P_DISABLE fix that makes GATE-A weight-sync work

feat(omni): composite TTS reward (ASR CER + audio guards) for GATE-B

feat(omni): GATE-B rollout->reward->advantage loop on real Higgs TTS

feat(omni): trainer-side Higgs TTS actor + logprob-parity gate

feat(omni): GATE-B full on-policy loop (GRPO + NCCL tts_engine weight-sync)

feat(omni): parametrize GATE-B rollout temperature/max_new_tokens

chore(omni): add corrected higgs codec-logprob patch (supersedes broken copy)

chore(omni): remove stale broken higgs codec-logprob patch

feat(omni): non-saturating eval sets + MAX_NEW knob for GATE-A/B learning runs

refactor(omni): rename gate_a/gate_b examples to function-descriptive names

refactor(omni): scrub GATE-A/GATE-B terminology + fix internal path refs

feat(rollout): wire --rollout-external to use an external sglang server

refactor(omni): consume Higgs rollout contract

fix(omni): preserve external rollout metadata paths

Commits on Jul 11, 2026

fix(omni): train all Higgs codebook actions