refactor: one post-sample finish step for every CLI and server decode site #2168

Part of #2166. Phase 1. Depends on #2167 (Phase 0 baseline and threshold). Can run in parallel with Phases 2 and 3.

Problem / Background

After a token is sampled, every decode site decides whether the sequence is finished. That logic is open-coded nine times, and the copies have already drifted. Stop strings (#1466) had to be added to five server sites. The speculative burst stream never got loop detection: src/server/batch/speculative_burst.rs contains no call to detect_repetition_loop, so a looping request served by MTP or DFlash runs to max_tokens while the same request on classic decode stops early with RepetitionLoop.

Current Behavior

Server sites, all in src/server/batch/:

1. The per-row loop in execute_batched_decode, at scheduler/decode_tick.rs:1088-1203 (loop detection at :1172).
2. apply_fused_decode_tokens, at scheduler/decode_tick.rs:1282-1400 (loop detection at :1369).
3. decode_single_step, at scheduler/decode_tick.rs:1412-1722 (finish block at :1608-1719, loop detection at :1687).
4. finish_prefill, at scheduler/prefill.rs:1319 (EOS at :1473, finish block at :1529-1612).
5. stream_burst_tokens and finalize_burst_stream, at speculative_burst.rs:2010-2185. Neither calls detect_repetition_loop.

CLI sites: the four CxxGenerator loops in src/lib/mlxcel-core/src/generate.rs. They are generate_streaming (:1914, loop check at :2168), generate_streaming_with_embeddings (:2233, :2381), generate_with_stats_and_embeddings (:2416, :2618) and generate_with_stats (:2683, :2901).

One finish path is deliberately not a post-sample site. When a decode lookahead teardown cannot rewind a model-owned sequence (#2182), apply_lookahead_trim (scheduler/decode_tick.rs:464-512) calls fail_desynchronized_sequence (:518-526), which finishes the request with FinishReason::Error and overrides a Length or Stop already recorded on the same tick, so the desynchronized state is never donated to the prompt cache. It stays outside finish_step, and nothing in finish_step may make a recorded finish immune to that override.

The server order at site 1, which is the reference:

1. EOS: finish Stop, and do not push the token.
2. Push the token to generated_tokens, and to token_history when needs_token_history().
3. Stop string through decode_state.on_token and stream_decoded_text: StopSequence.
4. finish_on_generation_bound (decode_tick.rs:23).
5. Structured-output stop: Stop.
6. max_tokens: Length.
7. context_bound_stop_due (decode_tick.rs:44): Length, and set context_exhausted.
8. detect_repetition_loop: RepetitionLoop.
9. If the sequence is not finished, apply the should_clear_cache_at cadence.

Proposed Solution

Add src/lib/mlxcel-core/src/decode_finish.rs. It has to live in the core so that CxxGenerator can call it. It exports:

pub enum FinishCause { Eos, StopSequence, StructuredStop, Length, ContextExhausted, RepetitionLoop }

pub trait FinishHooks {
    /// Feed the token through the text stop matcher; true when a stop string completed.
    fn stop_text(&mut self, token: i32) -> bool;
    /// A generation bound (b10621 #1477) has fired.
    fn bound_stopped(&self) -> bool;
    /// The next token would overflow the KV window with context shift disabled.
    fn context_bound_due(&self) -> bool;
}

pub struct FinishInput<'a> {
    pub token: i32,
    pub eos: &'a [i32],
    pub generated: &'a mut Vec<i32>,
    pub history: Option<&'a mut Vec<i32>>,
    pub max_tokens: usize,
    pub structured_stopped: bool,
    pub loop_detection: &'a LoopDetectionConfig,
}

pub fn finish_step(input: FinishInput<'_>, hooks: &mut dyn FinishHooks) -> Option<FinishCause>;

finish_step applies the nine steps above, in that order. The server implements FinishHooks on a borrow of SequenceInfo, using decode_state, stop_matcher, bound_stopped() and context_bound_stop_due. It maps FinishCause to FinishReason in one helper (Eos and StructuredStop to Stop, ContextExhausted to Length with retention.context_exhausted = true). The CLI passes a NoStopHooks that always returns false, so CLI behavior is unchanged. Every one of the nine sites calls finish_step, and no site keeps its own copy.

For the burst stream (site 5), per-token finishing moves into stream_burst_tokens through finish_step. As a result, burst requests get loop detection. finalize_burst_stream keeps only state transition and Done emission. The "drafter bailed early, report Stop" fallback stays as it is.

Scope

In scope: the new core module, its re-export from mlxcel-core, the nine call sites, and tests.

Out of scope: sampling (Phase 2), prefill (Phase 3), and any change to which finish reasons exist or to the order above.

Implementation Notes

• Reuse: detect_repetition_loop (src/lib/mlxcel-core/src/loop_detection.rs:148), should_clear_cache_at (src/lib/mlxcel-core/src/memory.rs:201), SequenceInfo::bound_stopped (src/server/batch/sequence.rs:555) and StopMatcher (src/server/batch/stop_matcher.rs:67). Do not reimplement any of them.
• Constraints: the CLI hot loop keeps its pipelining. finish_step must not force a host sync that the loop does not already do. It reads only the token id the loop has already read.
• Edge cases: EOS on the first token (prefill) is not pushed. max_tokens == 1 finishes Length at prefill. A stop string and max_tokens on the same token report StopSequence. With loop detection disabled (the default), it costs one branch.
• Error handling: state-transition errors keep today's tracing::error! and do not abort the request.

Acceptance Criteria

• rg -n 'detect_repetition_loop' src/server src/lib/mlxcel-core/src/generate.rs --glob '!*_tests.rs' finds no production caller, because the only caller is decode_finish.rs. The one remaining hit allowed is the apply_loop_guard helper in sequence.rs's test module (src/server/batch/sequence.rs:859-870), which replicates the decode-site guard for a state-machine test. rg -n 'generated_tokens.len\(\) >= .*max_tokens' src/server --glob '!*_tests.rs' finds no open-coded finish block at any of the nine sites (today it matches decode_tick.rs:1139, :1336, :1657, prefill.rs:1554 and speculative_burst.rs:2035, :2097, :2128)
• One unit test per site class (batched per-row, fused, single-step, prefill, burst, CLI loop) feeds the same token sequence and asserts the same FinishCause for EOS, stop string, bound, max_tokens, context bound and repetition loop
• A burst-path test fails on current main and passes after the change: a repeating token stream through stream_burst_tokens finishes RepetitionLoop
• The server and CLI suites are green, and greedy CLI output on a real checkpoint is byte-identical before and after
• Phase 0 B=1 and bench_decode numbers show no change beyond noise
• finish_step is the code path the server and CLI actually run, with no unused parallel module

Verification

make verify                                   # macOS gate
make verify-fmt verify-test-cuda              # Linux/CUDA gate
cargo test --profile test-fast --features cuda -p mlxcel-core decode_finish
cargo test --profile test-fast --features cuda --lib server::batch

References verified against main at fc21784 (2026-10-07).

Progress (PR #2201): the rg checks, the per-site-class tests, the burst loop test and the integration are done; the server suites are green module by module. Byte-identical real-checkpoint output (engine-parity) and the Phase 0 B=1 / bench_decode comparison are left to the epic orchestrator.