[Feature] Add experimental deferred Ouro deep batching - #125

[Feature] Add experimental deferred Ouro deep batching#125

Camellia86 wants to merge 1 commit into

InferMatrixLab:mainInferMatrixLab/vllm-rlt:mainfrom

Camellia86:feat/ouro-deferred-decodeCamellia86/vllm-rlt:feat/ouro-deferred-decodeCopy head branch name to clipboard