Skip to content

fix(pixel-edge): accumulate trailing buffered content and synthesize fallback finish - #5608

Open
vaibhavsrv wants to merge 1 commit into
Osmantic:public-betafrom
vaibhavsrv:fix/pixel-edge-buffered-stream-flush
Open

vaibhavsrv wants to merge 1 commit into
Osmantic:public-betafrom
vaibhavsrv:fix/pixel-edge-buffered-stream-flush

Conversation

@vaibhavsrv

Copy link
Copy Markdown
Contributor

Why this matters

In ods/extensions/services/pixel-edge/pixel_edge.py, _stream_upstream() handles chunked Server-Sent Events (SSE) streaming from model inference runtimes, buffering text to detect and replace empty or reserved responses before streaming to client applications. When an upstream model finishes streaming and the final SSE event does not contain a trailing newline before EOF, the remaining bytes are parsed from buffered (lines 1350-1362) and queued into pending via queue_pending(line, event, content, finish_reason). However, unlike chunk processing in the main loop (line 1338), _stream_upstream() omitted updating pending_text += content for this trailing chunk. Consequently, when a model finishes generation in a single un-terminated chunk or emits its final text across EOF, pending_text remains empty (""), triggering the if not normalized or normalized in _RESERVED_ASSISTANT_REPLIES: guard at line 1365. This causes the valid model response to be thrown away and replaced with empty_reply_fallback. Furthermore, replace_pending(template, synthesize_finish=False) at line 1370 passed synthesize_finish=False at EOF, emitting the fallback payload without a terminal finish_reason: "stop" chunk and leaving client SSE consumers stalled.

This fix surgically accumulates if content is not None: pending_text += content when queuing trailing buffered events, preventing false-positive empty reply replacement of valid model answers. It also inspects has_finish = any(item[3] is not None for item in pending) and synthesizes a terminal finish event when executing genuine empty reply fallback at EOF. Existing line length caps, prelude byte limits, and activity tracking remain untouched.

Validation

  • Baseline reproduction: Passing single un-terminated SSE chunks (e.g. b'data: {"model":"m","choices":[{"delta":{"content":"Tokyo"}}]}') left pending_text as "", causing _stream_upstream() to replace "Tokyo" with empty_reply_fallback without a finish reason ("finish_reason": null).
  • Post-fix behavior: Valid trailing buffered content is preserved and flushed to client responses, while genuinely empty streams synthesize fallback responses with terminal "finish_reason": "stop".
  • Telemetry statement: "[Pixel-edge] suites: 2 passed. New-test Ruff, ShellCheck, and diff checks pass; new regressions wired into Linux CI."

Overlap check

Risk / AI disclosure

AI-assisted investigation, implementation and CLI regressions. This strengthens SSE trailing buffer accumulation and fallback termination, not runtime admission. Independent human review and platform/runtime qualification remain gates. No running configuration, deployment or upstream merge changed.

Follow-up integration evidence

Composed with #5607 at 6e118f4 without conflicts. Production and test diffs passed together; pixel edge stream and single chunk checks remain intact.
Backlog composition was local-only (production/test diffs, excluding workflow/Makefile wiring); it is not an upstream merge or independent human approval. Declared live-review gates remain open.

@vaibhavsrv
vaibhavsrv force-pushed the fix/pixel-edge-buffered-stream-flush branch 2 times, most recently from 699b935 to 7706ffc Compare September 17, 2026 04:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant