Skip to content

fix(theme): refresh terminal colors via direct OSC probe - #1130

Merged
elucid merged 2 commits into
mainfrom
fix/osc-terminal-color-refresh
Oct 7, 2026
Merged

elucid merged 2 commits into
mainfrom
fix/osc-terminal-color-refresh

Conversation

@elucid

@elucid elucid commented Sep 28, 2026 •

Copy link
Copy Markdown
Member

Problem

#1128 made the terminal theme the default and refreshes it live. When the terminal sends a color-scheme notification (CSI ?997) or Hunk receives SIGWINCH, it calls OpenTUI's clearPaletteCache() + getPalette(). That covers @benvinegar's follow-up on #1128 ("detect the theme switch vs having to restart") and the SIGWINCH hook dhh added in omacom/omarchy#13518. Measured against a scripted PTY terminal that answers OSC 10/11/12 and OSC 4 but not OSC 13–19 (like most terminals and multiplexers), the getPalette() path has three problems:

  • Slow: every refresh costs about 300 ms. OpenTUI also queries OSC 12–19 and waits out its idle timeout for the ones nobody answers. Time from ?997 to repaint was 419–437 ms.

  • Misses slow terminals: when replies take 300 ms or more (SSH, busy multiplexers), getPalette() returns all nulls and Hunk keeps the old colors. This happened in 18/18 runs at 300, 500, and 1000 ms.

  • Mixed palette after rapid switches: Ghostty often sends a stale ?997 first and answers it with the old colors, then switches and sends the real one 0–150 ms later. Hunk ended on the new palette in 0/108 of these races. Two things cause it:

    • clearPaletteCache() does not cancel an in-flight detection, so the follow-up probe joins it.
    • That detection also absorbs OpenTUI's own OSC 10/11 replies to the second ?997.

    The screen is left with the new foreground/background and the old 16-color palette.

What changed

  • One watcher, new probe. watchTerminalColors (packages/hunk/src/ui/theme/terminalColorWatcher.ts) now probes the terminal directly:
    • It writes Hunk's startup query (OSC 10/11 + OSC 4;0–15) plus a primary device attributes query (DA1, ESC[c) through the renderer's frame-ordered output.
    • It collects the replies with OpenTUI's public renderer.subscribeOsc() and parses them with parseTerminalColorReplies.
    • A probe finishes when every color is answered or the DA1 reply arrives (terminals answer in order, so DA1 marks the end), or after 1.5 s.
    • The triggers are unchanged: ?997, SIGWINCH, and one mount probe when startup found no colors. The debounce stays at 100 ms.
    • The getPalette() refresh path is gone.
  • Supersede instead of join. A trigger during an in-flight probe discards that probe's answer and queues one fresh probe. Probes never overlap on the wire:
    • The next probe waits until the previous probe's DA1 reply arrives. A probe that timed out still waits for its DA1, so a late DA1 cannot close a newer probe early.
    • After 5 s the watcher stops waiting for a missing DA1.
    • The watcher swallows only the DA1 replies it is waiting for. OpenTUI treats every DA1 as a capability reply and forces a full repaint for it; passing ours through repainted about 12.8 KB on every no-op SIGWINCH.
  • OpenTUI's startup capability window. In 0.5.6, OpenTUI writes its own DA1 only from setupTerminal(), plus a DCS-wrapped retry inside tmux once XTVERSION answers. It stops listening for capability replies 5 s later. So for 5 s after mount the watcher can't tell whether a DA1 reply is OpenTUI's. During that window, probes skip DA1 and finish once every color is answered or on the timeout. OpenTUI's DA1 replies are never swallowed.
  • theme_mode trigger. OpenTUI's theme_mode event is now also a trigger, for when Hunk's startup probe came up empty and OpenTUI later learns the colors. It only requests a probe and never discards one in flight.
  • Private writeOut, with a fallback. OpenTUI 0.5.6 declares writeOut private, and there is no public raw-write API.
    • It is the path OpenTUI's own palette detector uses. On the threaded renderer it waits for an in-progress frame (BufferedBackend.writeOut in renderer-output.zig), which a direct stdout.write does not.
    • The cast is isolated in rendererRawWriter(). If a future OpenTUI removes writeOut, the watcher falls back to clearPaletteCache() + getPalette(), so an upgrade makes refresh slower instead of breaking it.
  • Shared query. core/theme/detection.ts now exports the color query (TERMINAL_COLOR_QUERY), DEVICE_ATTRIBUTES_QUERY, isDeviceAttributesReply, and hasTerminalColors. The startup probe and the watcher share them (ui → core only; deps:check is clean).
  • Docs and changeset. Updated docs/themes.md and the website themes page, and added a patch changeset.

This makes the existing SIGWINCH/?997 refresh faster and more reliable, in both regular and piped (patch -, pager) modes. It adds no new way for a terminal to signal a switch.

Results

The scripted fake PTY terminal is /tmp/theme-exp-osc (OpenTUI 0.5.6). It answers OSC 10/11/12 and OSC 4 but not OSC 13–19. Modes: diff with TTY stdin, patch - piped, pager piped. Branch runs were taken both inside OpenTUI's 5 s startup window (probes without DA1) and after it (probes with DA1).

Scenario Before (getPalette) After (this branch)
?997 → repaint, instant replies (12 switches) 419–437 ms 116–127 ms (the probe itself takes 1–5 ms; the rest is the 100 ms debounce)
Replies delayed 100 / 300 / 500 / 1000 ms (24 switches) 100 ms: ~620 ms; 300 ms and slower: missed, 18/18 All picked up: ~225 / ~425 / ~625 / ~1125 ms (both phases)
Stale-first ?997 race (d1 50–150 ms × d2 0–150 ms × 3 transitions × 3 modes) ends on new palette 0/108 (mixed palette) 108/108 in both phases, new colors 101–107 ms after the second ?997
Second ?997 while a probe is in flight (250 ms replies) 12/12 (~910 ms) 12/12 (~605–610 ms)
Terminal answers DA1 but not OSC colors 302 ms probe, colors kept After the window: 1–2 ms probe, colors kept. Inside the window: waits the 1.5 s timeout, colors kept.
Replies leaking as keys or screen text; OpenTUI themeMode / theme_mode none; correct none; identical to before

Real Ghostty (macOS, light/dark GitHub Light/Dark Default, system appearance toggled via System Events, two Dark→Light→Dark cycles per mode):

Mode Result osascript → Hunk color update
diff (TTY stdin) 4/4 switches repainted with the correct background and palette 377–613 ms
patch - piped 4/4 switches repainted with the correct background and palette 378–415 ms
  • The probe itself took 1–3 ms each time, and no replies leaked as keypresses.
  • Before this change the same setup took 590–690 ms, of which about 300 ms was getPalette().
  • Ghostty's stale-first ?997 pattern appeared (for example 997;1 then 997;2), and the watcher still ended on the right colors every time.

Tests

  • Unit (terminalColorWatcher.test.ts):

    • ends a probe on its DA1 reply, or once every color is answered while still waiting for the DA1
    • supersedes a probe and keeps its late replies out of the re-probe
    • DA1-only terminal
    • leaves foreign DA1 replies alone during the startup window
    • a timed-out probe still waits for its DA1; stops waiting after 5 s
    • suspended renderer
    • getPalette() fallback when writeOut is missing
    • theme_mode trigger
    • mount probe
    • rendererRawWriter
  • PTY (test/pty/terminal-theme-integration.test.ts; the harness's launchShellCommand can now answer terminal queries):

    • dark→dark switch in diff mode
    • dark→dark switch in piped patch - mode
    • the stale-first ?997 race ends on the new palette
    • a 400 ms-slow terminal after the startup window, with no DA1 or color replies on screen

    On origin/main the race and slow-terminal tests fail; they pass on this branch.

Verification

Command Result
bun run typecheck ✅
bun run lint ✅
bun run deps:check ✅
bun run check:docs ✅
bun run format:check ✅ for every tracked file; only fails on two untracked local notes in my checkout
bun run test 4511 pass, 1 fail: Jujutsu source reading > logs unexpected source failures… — fails the same way on origin/main (6c95708) in this environment
bun run test:integration 5 fail: PTY extensions trust prompt ×2 and PTY lifecycle SIGHUP/SIGQUIT/SIGPIPE — all fail the same way on origin/main here (bun test test/pty/extensions-integration.test.ts test/pty/lifecycle.test.ts); all theme PTY tests pass
bun run test:tty-smoke not run: all tests skip on macOS because they need util-linux script, which isn't available here

Not tested

  • tmux older than 3.6. Hunk's queries are not wrapped for tmux passthrough, same as the startup probe. There, OSC 4 goes unanswered, so the watcher gets foreground/background after its 1.5 s timeout.
  • tmux 3.6a was checked with a probe script, not with Hunk itself. tmux answers OSC 10/11/4 itself, in order, before DA1, and forwards ?997 only after refreshing its colors.

Follow-ups (out of scope)

@vercel

vercel Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
hunk-web Ignored Ignored Preview Oct 6, 2026 7:57pm UTC

Request Review

elucid added 2 commits October 6, 2026 15:56
Live terminal-color refresh (?997 notifications, SIGWINCH, and the mount probe) now writes Hunk's startup OSC 10/11/4 query plus a DA1 fence through the renderer and collects replies with OpenTUI's subscribeOsc, instead of clearPaletteCache() + getPalette(). A probe ends when the terminal has answered rather than after OpenTUI's 300 ms idle timeout, still picks up terminals answering after 300 ms, and a notification during an in-flight probe discards it and re-probes, so rapid switches no longer settle on a mix of old and new colors. OpenTUI's theme_mode event is an extra trigger. Probes skip the DA1 fence during OpenTUI's startup capability window, keep an owed fence after a timeout, and fall back to getPalette() if the renderer's private writeOut disappears.
@elucid
elucid force-pushed the fix/osc-terminal-color-refresh branch from 7f193a9 to 90fc0e0 Compare October 6, 2026 19:57
@elucid
elucid marked this pull request as ready for review October 7, 2026 13:36
@elucid
elucid merged commit 9566e33 into main Oct 7, 2026
15 checks passed
@greptile-apps

greptile-apps Bot commented Oct 7, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 4/5

[Medium risk] Speeds up terminal color detection after theme switches.

Fix the startup timeout path before merging; late replies can leave Hunk on the old palette.

Findings

  1. P1 Late replies restore old colors ▶
  2. P2 Queued refresh skips the delay ▶
  3. P2 Failed checks leave sessions open ▶
Fix with agent prompt
### Issue 1
packages/hunk/src/ui/theme/terminalColorWatcher.ts:239-243
During the first five seconds, probes have no DA1 fence. If a probe times out while another refresh is queued, `finish()` immediately starts the next probe. Late colors from the old probe then reach the new subscriber. A complete old reply set closes that probe before its own replies arrive, leaving Hunk on the old palette.

Keep timed-out startup probes from feeding later probes, and add a test with a queued refresh and late startup replies.

### Issue 2
packages/hunk/src/ui/theme/terminalColorWatcher.ts:264-268
`schedule()` restarts the debounce but leaves an earlier `probeRequested` set. If a queued refresh is waiting for DA1 and another notification arrives, that DA1 can call `drain()` before the new delay ends. Hunk sends one query immediately and another when the timer fires. This adds extra queries and can briefly apply colors before the intended settling delay ends.

Clear the pending request when restarting the debounce, or make `drain()` wait while `timer` is set.

### Issue 3
test/pty/terminal-theme-integration.test.ts:173-176
`launchTestDiffSession()` now waits for text and colors before returning the session to the caller's `try/finally`. If any wait fails, the caller never receives the session and cannot close it. Previously these checks were inside the test's `try`, so cleanup still ran. The new path can leave the Hunk process and terminal session running after a failed check.

Close the session in this helper's failure path before rethrowing.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

Replaces live getPalette() refreshes with direct terminal color queries and a DA1 reply boundary.

  • Shares the startup query with the watcher and retains the palette fallback.
  • Adds tests for slow replies, rapid switches, and piped patches.
  • Startup timeout isolation still needs a fix. Two smaller issues affect debounce behavior and test cleanup.

Acknowledged behavior:

  • elucid explicitly describes ignoring theme_mode during an active probe as intentional: the event requests a probe without discarding one already running.
  • elucid explicitly acknowledges stopping the DA1 wait after five seconds. The code documents the remaining risk from a very late reply.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["Notification or SIGWINCH"] --> B["Debounce"]
  C["theme_mode when idle"] --> B
  B --> D["Queue refresh"]
  D --> E{"Probe or DA1 still pending?"}
  E -->|Yes| F["Wait"]
  F --> E
  E -->|No| G{"Raw writer available?"}
  G -->|No| H["clearPaletteCache + getPalette"]
  G -->|Yes| I["Write OSC color queries"]
  I --> J{"Startup window ended?"}
  J -->|Yes| K["Add DA1 boundary"]
  J -->|No| L["No DA1 boundary"]
  K --> M["Collect colors"]
  L --> M
  M --> N["Finish on all colors, owned DA1, or timeout"]
  H --> O["Apply changed colors unless superseded"]
  N --> O
Loading

Reviews (1) · Last reviewed commit: "fix(theme): settle the palette fallback ..." · Reviewed by Greptile

Comment on lines +239 to +243
// During OpenTUI's startup capability window a DA1 reply may be OpenTUI's own, so probes there
// skip the fence and finish once every color is answered or on the timeout.
const fenced = fencesActive;
try {
write(fenced ? `${TERMINAL_COLOR_QUERY}${DEVICE_ATTRIBUTES_QUERY}` : TERMINAL_COLOR_QUERY);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Late replies restore old colors

During the first five seconds, probes have no DA1 fence. If a probe times out while another refresh is queued, finish() immediately starts the next probe. Late colors from the old probe then reach the new subscriber. A complete old reply set closes that probe before its own replies arrive, leaving Hunk on the old palette.

Keep timed-out startup probes from feeding later probes, and add a test with a queued refresh and late startup replies.

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/hunk/src/ui/theme/terminalColorWatcher.ts
Line: 239-243

Comment:
**Late replies restore old colors**

During the first five seconds, probes have no DA1 fence. If a probe times out while another refresh is queued, `finish()` immediately starts the next probe. Late colors from the old probe then reach the new subscriber. A complete old reply set closes that probe before its own replies arrive, leaving Hunk on the old palette.

Keep timed-out startup probes from feeding later probes, and add a test with a queued refresh and late startup replies.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment on lines 264 to +268
const schedule = () => {
if (disposed) return;
if (probe) probe.stale = true;
if (timer) clearTimeout(timer);
timer = setTimeout(() => void refresh(), REFRESH_DEBOUNCE_MS);
timer = setTimeout(requestProbe, debounceMs);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Queued refresh skips the delay

schedule() restarts the debounce but leaves an earlier probeRequested set. If a queued refresh is waiting for DA1 and another notification arrives, that DA1 can call drain() before the new delay ends. Hunk sends one query immediately and another when the timer fires. This adds extra queries and can briefly apply colors before the intended settling delay ends.

Clear the pending request when restarting the debounce, or make drain() wait while timer is set.

Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/hunk/src/ui/theme/terminalColorWatcher.ts
Line: 264-268

Comment:
**Queued refresh skips the delay**

`schedule()` restarts the debounce but leaves an earlier `probeRequested` set. If a queued refresh is waiting for DA1 and another notification arrives, that DA1 can call `drain()` before the new delay ends. Hunk sends one query immediately and another when the timer fires. This adds extra queries and can briefly apply colors before the intended settling delay ends.

Clear the pending request when restarting the debounce, or make `drain()` wait while `timer` is set.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Comment on lines +173 to +176
await session.waitForText("export const answer = 42;");
await waitForForeground(session, PALETTE_A.ansi[5]!, "export");
await waitForForeground(session, PALETTE_A.foreground, "answer");
return session;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Failed checks leave sessions open

launchTestDiffSession() now waits for text and colors before returning the session to the caller's try/finally. If any wait fails, the caller never receives the session and cannot close it. Previously these checks were inside the test's try, so cleanup still ran. The new path can leave the Hunk process and terminal session running after a failed check.

Close the session in this helper's failure path before rethrowing.

Prompt To Fix With AI
This is a comment left during a code review.
Path: test/pty/terminal-theme-integration.test.ts
Line: 173-176

Comment:
**Failed checks leave sessions open**

`launchTestDiffSession()` now waits for text and colors before returning the session to the caller's `try/finally`. If any wait fails, the caller never receives the session and cannot close it. Previously these checks were inside the test's `try`, so cleanup still ran. The new path can leave the Hunk process and terminal session running after a failed check.

Close the session in this helper's failure path before rethrowing.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant