fix(code-interpreter-js): make readLines linear on long single-line outputs

[Kylinny]

Fixes #1990.

What changed

readLines() in packages/code-interpreter-js/src/utils.ts appended every
decoded chunk to a single string buffer and re-scanned the whole buffer with
buffer.indexOf('\n') (plus buffer.slice() copies) on every read. For a
long line with no newlines — a base64-encoded image, a wide DataFrame, large
JSON — that is O(n²) in line length and blocks the Node event loop.

The fix decodes and scans only the newly arrived chunk, accumulates line
fragments in an array, and joins once per emitted line. Streaming
TextDecoder usage (multi-byte characters split across chunks still decode
correctly), the trailing partial line yielded at stream end, \r preservation
(lines are split on \n only), and reader.releaseLock() in finally are
all unchanged.

Per the repo's "equivalent changes in JS and Python" rule I checked the
Python side: both the sync and async code-interpreter stream lines through
httpx's iter_lines() / aiter_lines(), whose LineDecoder already
accumulates fragments in a list and joins once per emitted line — no
equivalent bug there, so no Python change is needed.

Reproducer

Save as repro-1990.mjs and run with node repro-1990.mjs (Node 22+, no
dependencies). It feeds a single-line stream with no newlines through
readLines() in 64 KiB chunks — readLinesBefore is the implementation on
main before this PR, readLinesAfter is this PR's version:

// Reproducer for https://github.com/e2b-dev/E2B/issues/1990
// Run with: node repro-1990.mjs   (Node 22+, no dependencies)
//
// Feeds a single-line (no newlines) byte stream — like a base64-encoded
// image, a wide DataFrame, or large JSON — through readLines() in 64 KiB
// chunks and measures wall time.
//
// `readLinesBefore` is the implementation on main before the fix
// (packages/code-interpreter-js/src/utils.ts @ 62a17df), copied verbatim.
// `readLinesAfter` is the fixed version from this PR.

async function* readLinesBefore(stream) {
  const reader = stream.getReader()
  const decoder = new TextDecoder()
  let buffer = ''

  try {
    while (true) {
      const { done, value } = await reader.read()

      if (value !== undefined) {
        buffer += decoder.decode(value, { stream: true })
      }

      if (done) {
        buffer += decoder.decode()
        if (buffer.length > 0) {
          yield buffer
        }
        break
      }

      let newlineIdx = -1

      do {
        newlineIdx = buffer.indexOf('\n')
        if (newlineIdx !== -1) {
          yield buffer.slice(0, newlineIdx)
          buffer = buffer.slice(newlineIdx + 1)
        }
      } while (newlineIdx !== -1)
    }
  } finally {
    reader.releaseLock()
  }
}

async function* readLinesAfter(stream) {
  const reader = stream.getReader()
  const decoder = new TextDecoder()
  const fragments = []

  try {
    while (true) {
      const { done, value } = await reader.read()

      if (value !== undefined) {
        const text = decoder.decode(value, { stream: true })
        let start = 0
        let newlineIdx = text.indexOf('\n', start)
        while (newlineIdx !== -1) {
          fragments.push(text.slice(start, newlineIdx))
          yield fragments.join('')
          fragments.length = 0
          start = newlineIdx + 1
          newlineIdx = text.indexOf('\n', start)
        }
        if (start < text.length) {
          fragments.push(text.slice(start))
        }
      }

      if (done) {
        const trailing = decoder.decode()
        if (trailing.length > 0) {
          fragments.push(trailing)
        }
        if (fragments.length > 0) {
          yield fragments.join('')
        }
        break
      }
    }
  } finally {
    reader.releaseLock()
  }
}

const CHUNK = 64 * 1024

function makeStream(totalBytes) {
  let remaining = totalBytes
  return new ReadableStream({
    start(controller) {
      while (remaining > 0) {
        const n = Math.min(CHUNK, remaining)
        controller.enqueue(new Uint8Array(n).fill(0x78)) // 'x', no newlines
        remaining -= n
      }
      controller.close()
    },
  })
}

async function bench(label, readLines, mb) {
  const lines = []
  const t0 = performance.now()
  for await (const line of readLines(makeStream(mb * 1024 * 1024))) {
    lines.push(line)
  }
  const ms = performance.now() - t0
  const chars = lines.reduce((a, l) => a + l.length, 0)
  console.log(`${label}: ${mb}MB -> ${lines.length} line(s), ${chars} chars, ${ms.toFixed(0)}ms`)
  if (lines.length !== 1 || chars !== mb * 1024 * 1024) {
    throw new Error('wrong output')
  }
}

for (const mb of [10, 40]) await bench('before', readLinesBefore, mb)
for (const mb of [10, 40]) await bench('after ', readLinesAfter, mb)

Output (verbatim, measured on this machine):

before: 10MB -> 1 line(s), 10485760 chars, 473ms
before: 40MB -> 1 line(s), 41943040 chars, 8987ms
after : 10MB -> 1 line(s), 10485760 chars, 43ms
after : 40MB -> 1 line(s), 41943040 chars, 111ms

4× the input → ~19× the time before the fix (quadratic), ~2.6× after
(linear); the 40MB line goes from ~9s of blocked event loop to ~0.1s.

Tests

• New regression + edge-case tests in packages/code-interpreter-js/tests/utils.test.ts:
  a 1MB single line assembled from 1 KiB chunks yields one intact line;
  \r kept when splitting on \n; newline straddling a chunk boundary;
  empty stream; leading empty chunks.
• pnpm vitest run tests/utils.test.ts → 15/15 pass. (The package's other
  test files are live-API integration tests requiring E2B_API_KEY and fail
  identically without the key on unmodified main.)
• pnpm run format, pnpm run lint (0 errors; 1 pre-existing warning in
  untouched src/messaging.ts), pnpm run typecheck all pass.

Changeset

Patch changeset for @e2b/code-interpreter included.

---

[cla-bot]

We require contributors to sign our Contributor License Agreement, and we don't have @Yao-Junyi on file. You can sign our CLA at https://e2b.dev/docs/cla . Once you've signed, post a comment here that says '@cla-bot check'

[changeset-bot]

🦋 Changeset detected

Latest commit: 61cff83

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package

Name	Type
@e2b/code-interpreter	Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-15T19:49:56.655545Z	61cff83	PR opened

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[devin-ai-integration]

TASTE.md review: complies, 0 violations. readLines is internal (not re-exported from index.ts) so no public surface changes; checked T-29 (streaming UTF-8 decode with replacement, flushed at stream end: kept, covered by tests) and T-1/T-2 (parity: Python uses httpx iter_lines/aiter_lines, which doesn't have this bug).

Written by Devin

[devin-ai-integration]

🔍 Devin Review: 1 flag

Not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)