Skip to content

feat(sched): bound the uninterruptible prefill step — 41.9 s to 23.2 s TTFT p50, and 4 users to 101 at K=128 [MEASURED] - #119

Closed
heydryft wants to merge 10 commits into
masterfrom
perf/prefill-admission-cap
Closed

feat(sched): bound the uninterruptible prefill step — 41.9 s to 23.2 s TTFT p50, and 4 users to 101 at K=128 [MEASURED]#119
heydryft wants to merge 10 commits into
masterfrom
perf/prefill-admission-cap

feat(sched): bound how much prefill can accumulate in one uninterrupt…

71397b8
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar