Skip to content

fix(pixel-teams): handle split multibyte utf8 sequences in sse stream decoding - #5722

Closed
Vishaaallll wants to merge 1 commit into
Osmantic:public-betafrom
Vishaaallll:fix/pixel-teams-multibyte-utf8-stream
Closed

Vishaaallll wants to merge 1 commit into
Osmantic:public-betafrom
Vishaaallll:fix/pixel-teams-multibyte-utf8-stream

Conversation

@Vishaaallll

Copy link
Copy Markdown
Contributor

Why this matters

In ods/extensions/services/dashboard-api/routers/pixel_teams.py, the _run coroutine decoded SSE chunks from response.body_iterator using buffered += chunk.decode("utf-8") if isinstance(chunk, bytes) else chunk. When a multi-byte UTF-8 character (such as emoji, symbols, or localized characters) is split across network TCP or chunk boundaries by the upstream inference stream, calling chunk.decode("utf-8") on an incomplete byte slice raises an unhandled UnicodeDecodeError, terminating the team worker task abruptly.

This surgical fix introduces Python's standard codecs.getincrementaldecoder("utf-8")() to buffer incomplete multi-byte sequences across chunk boundaries before appending decoded text to the line buffer.

Validation

  • Baseline reproduction: verified that direct .decode("utf-8") on split chunk b'\xf0\x9f' raises UnicodeDecodeError.
  • Post-fix behavior: incremental decoder buffers partial code units, decoding full characters across chunk boundaries without error.
  • Telemetry: pixel-teams suite: 2 passed. New-test syntax and diff checks pass; new regression wired into Linux CI.

Overlap check

Inspected live open/closed PRs. PR #5695 touched chat cancel; PR #5689 touched team history files. Zero overlap with SSE streaming chunk decoding in pixel_teams.py.

Risk / AI disclosure

AI-assisted investigation, patch, and regression tests. Uses Python's standard incremental decoder without changing downstream SSE parsing contracts. Independent review remains a gate.

Follow-up integration evidence

Composed locally with public-beta at 4fa6ad170 without conflicts. Unit and CI checks pass.

@Lightheartdevs

Copy link
Copy Markdown
Collaborator

Thanks for this contribution. public-beta was promoted into main on 2026-09-24 and no longer receives changes, so we're closing pull requests that target it. This isn't a judgment on the change itself. If it's still needed, please rebase onto main and open a focused PR. See #7253 for details and the contribution policy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants