feat(console): render arduino-cli colour and collapse progress redraws

[thiagoralves]

Problem

Build output is captured from a pipe and pushed to the console verbatim, so the two terminal control sequences arduino-cli uses were both mishandled.

Carriage returns. A download progress bar redraws by rewriting one line with \r. Each redraw arrived as its own chunk and became its own timestamped entry, so one core install produced hundreds of near-identical lines that pushed the real output out of view:

[09:12:34]: ...54.94 MiB / 93.67 MiB [=====>-----]  58.65%
[09:12:35]: ...54.94 MiB / 93.67 MiB [=====>-----]  58.65%
[09:12:35]: ...57.68 MiB / 93.67 MiB [======>----]  61.58%

SGR colour. arduino-cli colours its compile summary table. The editor suppressed this with output.no_color in arduino-cli.yaml, inherited from the 2022 Python editor, which added --no-color because the raw ESC[92m bytes were being printed literally.

Worth recording, since it contradicts a natural assumption: arduino-cli emits colour through a pipe just as much as under a PTY — it does not TTY-gate. Measured identically on 1.0.3 (the version that prompted the original --no-color) and the bundled 1.4.1: 10 escape sequences each, piped and under a PTY. Nothing changed upstream; the config was doing the suppressing all along.

Changes

• frontend/utils/terminal-output.ts (new, shared): \r collapsing plus a small SGR parser. Explicitly not a terminal emulator — cursor addressing, scroll regions and erase-in-line are stripped rather than interpreted, because build output never uses them.
• Carriage returns. A chunk collapses to the frame a terminal would leave on screen, and the entry is marked transient while the line is still open. The next redraw overwrites it; a trailing newline commits it so the following download starts its own line. That commit step matters — without it every download in a session collapses onto one line and you lose the … downloaded record of each.
• Colour is split off once, in the console slice: message always holds clean text and segments carries styling only when there was any. Search, level filters and copy therefore needed zero changes, and uncoloured logs (nearly all of them) keep their exact previous shape and allocate nothing extra.

Dead code removed rather than left behind

• output.no_color dropped from ARDUINO_DATA, and existing configs migrated. This was the trap: the config was written with { flag: 'wx' } and skipped on EEXIST forever, so every machine that had ever launched an older build would have kept no_color: true and made the new renderer unreachable. Reconciliation lives in its own testable module using the yaml Document API, so comments, ordering and user-added indexes survive. It only adds missing URLs, drops no_color, and prunes output if that leaves it empty — and bails on an unparseable file rather than clobbering it.
• ArduinoCliConfigSchema / ArduinoCliConfig deleted: a zod schema describing the config's no_color shape, imported by nothing.

Also folded a duplicated ternary in log.tsx (the compileError branch was identical in both arms) into one MessageBody component.

A fix that was tried and reverted

The bar still wraps across ~2 visual lines, because arduino-cli sizes it to a width it guesses from its own environment. I pushed a fix that stripped the ASCII bar with a regex and reverted it (2dfbb557e → 38f0f7859): pattern-matching another tool's cosmetic output is fragile and would break silently if the bar glyphs ever change. The wrap is accepted, deliberately.

For anyone tempted to retry: there is no flag for it. COLUMNS, TERM and a real PTY all make no difference, and the full global flag list has nothing for progress or width. The only source-side lever is --json, which suppresses download progress entirely (measured: zero bytes of output while the download still runs) and would require re-plumbing compile output through a JSON envelope.

Tests

29 new, across the parser, the CR state machine, the slice's overwrite rule, and the config migration including the upgrade-from-no_color path. Checked they are not vacuous: reverting the three source hunks fails 5 of them.

Verified against real captured arduino-cli bytes: 24 CR frames collapse, the summary table maps to green/plain/green/plain/grey exactly as its 92/92/90 codes specify, and neither an escape nor a carriage return survives into stored text.

Full editor suite 6362 passing; typecheck, lint and prettier clean across all 15 changed files.

Merge note

Branched from development, so it does not contain #1001. Both touch #checkIfArduinoCliConfigExists, so they will conflict. This branch's reconcileArduinoCliConfig is a superset — it does #1001's URL merging and the no_color migration — so resolution is "take this branch's version", not a hand-merge. Merge #1001 first, then this.

It also resolves a review finding on #1001: the regex there could not see additional_urls: [] (the shape arduino-cli config init writes). The yaml rewrite handles it, and the substring-match issue with it.

Paired with openplc-web (shared-core parity).

🤖 Generated with Claude Code

Summary by CodeRabbit

• New Features
  • Console output now preserves supported text colors and bold styling.
  • Live compiler progress updates redraw a single line instead of creating repeated entries.
  • Compiler output handling better preserves readable formatting, including error details and indentation.
• Bug Fixes
  • Existing Arduino CLI configuration is updated safely while preserving custom settings and comments.
  • Obsolete output settings are removed, and missing board-manager URLs are added.
• Tests
  • Added coverage for styled logs, progress redraws, configuration updates, and malformed input handling.

[coderabbitai]

Walkthrough

The change reconciles existing Arduino CLI YAML files with shipped settings and removes obsolete output.no_color. It also adds ANSI styling and carriage-return progress handling across terminal parsing, console state, event processing, and rendering.

Changes

Arduino CLI configuration reconciliation

Layer / File(s)	Summary
Configuration reconciliation src/backend/editor/compiler/types.ts, src/backend/editor/services/user-service/data/...	The reconciler preserves custom settings and URLs, adds missing shipped URLs, removes obsolete output settings, and avoids unnecessary rewrites. Tests cover upgrades, preservation, idempotence, invalid YAML, and fresh data.
Configuration initialization wiring src/backend/editor/services/user-service/index.ts	Arduino CLI initialization creates missing files and reconciles existing files. Errors are logged without throwing.

Console terminal output

Layer / File(s)	Summary
Terminal metadata and parsing src/middleware/shared/ports/types.ts, src/frontend/utils/terminal-output.ts, src/frontend/utils/__tests__/terminal-output.test.ts	Log entries support styled segments and transient state. Terminal utilities parse ANSI styles, strip control sequences, and collapse carriage-return frames.
Progress event and store handling src/frontend/utils/debugger-session.ts, src/frontend/store/slices/console/..., src/frontend/store/__tests__/console-slice.test.ts, src/frontend/utils/__tests__/debugger-session.test.ts	Compiler output marks unfinished redraw lines as transient. The store replaces transient entries during redraws and stores clean messages with parsed styling.
Styled console rendering src/frontend/components/_organisms/console/...	The console passes segments to LogComponent. MessageBody renders styled segments with search highlighting and preserves plain-message fallback behavior.

Estimated code review effort: 4 (Complex) | ~45 minutes

Poem

I’m a rabbit with logs in a row,
Clean colors now sparkle and glow.
Old settings hop out of sight,
Progress redraws just right.
YAML keeps custom tunes,
While carrots compile beneath the moons.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly summarizes the main console changes: Arduino CLI color rendering and progress redraw collapsing.
Description check	✅ Passed	The description thoroughly explains the problem, implementation, migration, tests, accepted limitations, and merge considerations.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/console-terminal-output

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

Actionable comments posted: 8

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In
`@src/backend/editor/services/user-service/data/__tests__/arduino-cli-config.test.ts`:
- Around line 19-20: Remove the prohibited type assertions in urlsOf and the
updated-config result checks. Add a requireUpdated helper that throws when the
update result is null, use a non-asserted type predicate to narrow parsed YAML
before accessing board_manager.additional_urls, and replace the affected result
assertions with requireUpdated-based handling.

In `@src/backend/editor/services/user-service/data/arduino-cli-config.ts`:
- Around line 28-35: Update shippedBoardManagerUrls to remove the parse(shipped)
type assertion and validate the parsed YAML value at runtime before accessing
board_manager.additional_urls. Add or reuse a type guard that confirms the
nested object shape and filters string URLs, while preserving the existing
empty-array fallback for invalid or missing data.
- Around line 53-76: Guard nested YAML operations in the configuration migration
with isMap checks: only access BOARD_MANAGER_URLS_PATH via getIn/setIn when the
board_manager parent is a map, and only call hasIn(NO_COLOR_PATH),
deleteIn(NO_COLOR_PATH), or delete('output') when output is a map. Preserve the
existing list merge and cleanup behavior for valid map nodes.

In `@src/backend/editor/services/user-service/index.ts`:
- Around line 171-177: Update the migration flow around
reconcileArduinoCliConfig to write updated content to a temporary file in the
same directory, then rename that file to pathToArduinoCliConfig only after the
write succeeds, preserving the existing early return when no update is needed
and ensuring the temporary file targets the same filesystem for atomic
replacement.

In `@src/frontend/components/_organisms/console/log.tsx`:
- Around line 89-92: Update the segment rendering in the log component so search
ranges are computed against the full message, then intersected with each
segment’s source range before rendering highlights. Preserve segment styling
while highlighting matches that cross boundaries, and add a render test covering
a search term spanning two adjacent segments.

In `@src/frontend/utils/__tests__/debugger-session.test.ts`:
- Around line 167-175: Remove the type assertion from createRedrawCollector by
declaring log with the callback type expected by logCompilerEvent when it is
defined. Return the typed log function directly while preserving its existing
writes behavior and optional redraw handling.

In `@src/frontend/utils/debugger-session.ts`:
- Around line 80-105: Update logCompilerEvent and its carriage-return handling
so complete CRLF sequences are normalized before redraw detection, while a
trailing \r is retained across events until the next chunk determines whether it
forms \r\n. Ensure CRLF is never marked as a redraw and add a regression test
covering successive events containing "Windows line\r" and "\n".

In `@src/frontend/utils/terminal-output.ts`:
- Around line 35-36: Extend CSI_PATTERN and the tokenizer used by stripAnsi and
parseAnsi to consume OSC sequences, including their terminated payloads, and
remove any remaining ESC control bytes so stored messages and rendered segments
contain no escape characters. Preserve existing CSI parsing behavior and add
coverage for at least one OSC sequence, such as a hyperlink.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 4c12d5eb-0288-491c-819c-4016c0648e91

📥 Commits

Reviewing files that changed from the base of the PR and between 4f05cf6 and 38f0f78.

📒 Files selected for processing (15)

• src/backend/editor/compiler/types.ts
• src/backend/editor/services/user-service/data/__tests__/arduino-cli-config.test.ts
• src/backend/editor/services/user-service/data/arduino-cli-config.ts
• src/backend/editor/services/user-service/data/types.ts
• src/backend/editor/services/user-service/index.ts
• src/frontend/components/_organisms/console/index.tsx
• src/frontend/components/_organisms/console/log.tsx
• src/frontend/store/__tests__/console-slice.test.ts
• src/frontend/store/slices/console/slice.ts
• src/frontend/store/slices/console/types.ts
• src/frontend/utils/__tests__/debugger-session.test.ts
• src/frontend/utils/__tests__/terminal-output.test.ts
• src/frontend/utils/debugger-session.ts
• src/frontend/utils/terminal-output.ts
• src/middleware/shared/ports/types.ts

💤 Files with no reviewable changes (1)

• src/backend/editor/services/user-service/data/types.ts

[coderabbitai]

🧹 Nitpick comments (1)

src/backend/editor/services/user-service/data/__tests__/arduino-cli-config.test.ts (1)

122-124: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Assert the expected outcomes for scalar YAML cases.

Both tests only check that reconciliation does not throw. The scalar board_manager case should return null. The scalar additional_urls case should return rewritten YAML containing the shipped URLs. A regression can otherwise pass while returning null or preserving additional_urls: 7.

Proposed assertions

-    expect(() => reconcileArduinoCliConfig('board_manager: 5\n', ARDUINO_DATA)).not.toThrow()
+    expect(reconcileArduinoCliConfig('board_manager: 5\n', ARDUINO_DATA)).toBeNull()

-    expect(() => reconcileArduinoCliConfig('board_manager:\n  additional_urls: 7\n', ARDUINO_DATA)).not.toThrow()
+    const updated = requireUpdated(
+      reconcileArduinoCliConfig('board_manager:\n  additional_urls: 7\n', ARDUINO_DATA),
+    )
+    const updatedUrls = urlsOf(updated)
+    for (const url of urlsOf(ARDUINO_DATA)) expect(updatedUrls).toContain(url)

These outcomes match src/backend/editor/services/user-service/data/arduino-cli-config.ts, Lines 49-92.

Also applies to: 140-142

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In
`@src/backend/editor/services/user-service/data/__tests__/arduino-cli-config.test.ts`
around lines 122 - 124, Strengthen the scalar YAML tests for
reconcileArduinoCliConfig: in the board_manager scalar test, assert the result
is null; in the additional_urls scalar test, assert the returned YAML is
rewritten and contains the shipped URLs rather than preserving additional_urls:
7. Update the existing tests around reconcileArduinoCliConfig without changing
production behavior.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In
`@src/backend/editor/services/user-service/data/__tests__/arduino-cli-config.test.ts`:
- Around line 122-124: Strengthen the scalar YAML tests for
reconcileArduinoCliConfig: in the board_manager scalar test, assert the result
is null; in the additional_urls scalar test, assert the returned YAML is
rewritten and contains the shipped URLs rather than preserving additional_urls:
7. Update the existing tests around reconcileArduinoCliConfig without changing
production behavior.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e3944d17-d58b-455e-95b2-3985bbd3e3f2

📥 Commits

Reviewing files that changed from the base of the PR and between d301457 and 1f06b1f.

📒 Files selected for processing (7)

• src/backend/editor/services/user-service/data/__tests__/arduino-cli-config.test.ts
• src/backend/editor/services/user-service/data/arduino-cli-config.ts
• src/backend/editor/services/user-service/index.ts
• src/frontend/utils/__tests__/debugger-session.test.ts
• src/frontend/utils/__tests__/terminal-output.test.ts
• src/frontend/utils/debugger-session.ts
• src/frontend/utils/terminal-output.ts

🚧 Files skipped from review as they are similar to previous changes (5)

• src/frontend/utils/tests/debugger-session.test.ts
• src/backend/editor/services/user-service/index.ts
• src/frontend/utils/debugger-session.ts
• src/frontend/utils/tests/terminal-output.test.ts
• src/backend/editor/services/user-service/data/arduino-cli-config.ts