Skip to content

fix(pixel-edge): accumulate content before finish reason in stream fallback guard - #5604

Open
vaibhavsrv wants to merge 1 commit into
Osmantic:public-betafrom
vaibhavsrv:fix/pixel-edge-stream-single-chunk
Open

vaibhavsrv wants to merge 1 commit into
Osmantic:public-betafrom
vaibhavsrv:fix/pixel-edge-stream-single-chunk

Conversation

@vaibhavsrv

Copy link
Copy Markdown
Contributor

Why this matters

In the baseline Pixel Edge streaming proxy (_stream_upstream), concise model completions or single-chunk model responses (common with smaller, faster models or inference engines emitting the final token and finish_reason: "stop" concurrently in the same chunk) trigger the empty-reply fallback prematurely. Because finish_reason was evaluated before accumulating content into pending_text, pending_text was evaluated as empty "", causing the edge to inject empty_reply_fallback ("I couldn't produce a useful response to that request...") before the actual model content. Furthermore, when replace_pending was called on unverified turns, the unparsed line was written unconditionally afterwards, leaking reserved phrases like NO_REPLY into the client stream.

This change accumulates incoming SSE content into pending_text before evaluating finish_reason, and cleanly synthesizes finish_reason: "stop" when replacing pending empty or reserved replies (synthesize_finish=True) without leaking the unverified or reserved payload. Existing SSE chunk framing limits, CRLF normalization, and reserved reply definitions remain completely untouched.

Validation

  • Baseline reproduction: On unpatched code, a model response containing {"choices": [{"delta": {"content": "Paris"}, "finish_reason": "stop"}]} failed with 'FALLBACK_CALLED' unexpectedly found in the stream output, erroneously prepending the fallback error message before "Paris". Single-chunk NO_REPLY responses leaked 'NO_REPLY' into the output after the fallback message.
  • Post-fix verification: The single-chunk response delivers "Paris" cleanly without triggering the fallback; reserved NO_REPLY payloads are cleanly replaced with synthesized fallback frames without leaking NO_REPLY. The full test suite in test_pixel_edge.py (102 tests) and new regression test_pixel_edge_single_chunk.py (3 tests) pass cleanly.
  • Telemetry statement: "Pixel Edge suites: 105 passed. New-test Ruff, PyCompile, and diff checks pass; new regressions wired into Linux CI."

Overlap check

Risk / AI disclosure

AI-assisted investigation, implementation and CLI regressions. This strengthens an SSE stream delivery check, not runtime admission or model authorization. Independent human review and platform/runtime qualification remain gates. No running configuration, deployment or upstream merge changed.

Follow-up integration evidence

Composed with #5600 at 3f44dce without conflicts. Production and test diffs passed together; pixel edge and inference checks remain intact.
Backlog composition was local-only (production/test diffs, excluding workflow/Makefile wiring); it is not an upstream merge or independent human approval. Declared live-review gates remain open.

@vaibhavsrv
vaibhavsrv force-pushed the fix/pixel-edge-stream-single-chunk branch from 7606c52 to 736cb40 Compare September 17, 2026 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant