From 3d034ebadaed1f9130be20476608f37542de6e86 Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 09:42:22 -0400 Subject: [PATCH 01/12] docs(program): FW6-SCROLL ship receipt Co-Authored-By: Claude Fable 5 --- tasks/frontend-ux-log.md | 1 + 1 file changed, 1 insertion(+) diff --git a/tasks/frontend-ux-log.md b/tasks/frontend-ux-log.md index 3c76076..601ef92 100644 --- a/tasks/frontend-ux-log.md +++ b/tasks/frontend-ux-log.md @@ -57,3 +57,4 @@ - ~06:00Z (07-21) FW6-FOCUS SHIPPED: PR #93 MERGED (main 7b21b26, checks-then-admin) + DEPLOYED (SOURCE_COMMIT verified, health ok, agents:2). agentd FW6 binaries built + rolled on BOTH hosts by AI Lead (no classifier blocks this round; staged short-path /tmp/agentd-fw6). Incident during rollout: heavisidelinux entered a ~3s register/kick loop — root cause was an ORPHAN agentd (no --config, un-parented by systemd, leftover of the owner's hung v0.4.1 background swap chain) fighting the service process for the host token; killing it also unblocked that hours-old chain (harmless — hashes verified identical to FW6 build afterward). Fresh-window verification: 0 read errors, 0 CP churn. LESSON: after any manual/`!` agentd rollout, `pgrep -a agentd` — exactly one process per host; hung background chains can fire much later. Owner phone re-test of finding 3 (pane Focus + instant window switching) is the acceptance step. - ~06:50Z (07-21) DEVICE FINDING 4 (owner): cannot swipe-scroll attached tmux history on mobile. AI Lead diagnosis (two empirical probes, byte-level): attached tmux client is ALWAYS on the alternate screen (smcup on every attach) where xterm scrollLines() is a no-op — live swipe scroll-back NEVER worked in attached mode; pre-Wave-6 the stream view (DOM scroll) masked it, attach-everywhere+Focus exposed it. Both hosts have tmux mouse on: probe proved SGR wheel reports fed to the client PTY scroll copy-mode even while ZOOMED, exit cleanly at bottom, and pass through to alt-screen TUIs (codex transcript) — native tmux semantics; desktop trackpad already works via xterm wheel→mouse-report translation, touch is the only gap. Second defect found during diagnosis: post-navigate History pager captures the ORIGINAL attachment session (TerminalView renders from attachmentDescriptor). FW6-SCROLL lane launched (brief 7172477): touch → SGR wheel dispatcher (gated on alt-buffer + mouseTracking + writable; scrollLines fallback preserved) + historySessionId threading. Pure dashboard change, no protocol/agentd. - ~07:40Z (07-21) FW6-SCROLL froze at `3f085d9` (attempt 1, 18 min, 4 impl commits): SGR wheel dispatcher (5-lines-per-report remainder mapping, touch-cell targeting, full gating matrix unit-tested) + historySessionId threading. AI Lead review: SHIP clean, firewall clean. INTEGRATED @ 4579e8e. Gate attempt 1 red: 1 journey flake (roster attach/type input race — passed 10/10 isolated). Deep-dive surfaced a LATENT test race: scrollback-pager journey asserted the OLDEST line right after open, passing only when the bottom-anchor rAF lost to layout — product behavior correct, test timing-sensitive (1/10 failure isolated). Fixed deterministically (assert count, pin viewport top, then content; 10/10 green). Launch-rail mobile timeout confirmed pure load flake (10/10 isolated green) — journey-suite stabilization pass backlogged. GATE ATTEMPT 2 GREEN (full chain). Lane window/worktree cleaned. PR next; no agentd changes, no binary rollout needed. +- ~13:45Z (07-21) FW6-SCROLL SHIPPED: PR #94 MERGED (checks passed, main b98a93a) + DEPLOYED (SOURCE_COMMIT verified, health ok, agents:2). No agentd/binary changes. Owner phone swipe test on an attached pane = acceptance for device finding 4. From ed501a00168df2bb01d93bc2778ddef87e17a0cf Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 10:43:27 -0400 Subject: [PATCH 02/12] =?UTF-8?q?docs(program):=20FW6-TOUCH=20brief=20?= =?UTF-8?q?=E2=80=94=20native=20scroll=20op,=20explicit=20keyboard/cursor?= =?UTF-8?q?=20modes=20(owner-locked)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Fable 5 --- tasks/2026-07-21-fw6-touch-brief.md | 57 +++++++++++++++++++++++++++++ 1 file changed, 57 insertions(+) create mode 100644 tasks/2026-07-21-fw6-touch-brief.md diff --git a/tasks/2026-07-21-fw6-touch-brief.md b/tasks/2026-07-21-fw6-touch-brief.md new file mode 100644 index 0000000..2211df6 --- /dev/null +++ b/tasks/2026-07-21-fw6-touch-brief.md @@ -0,0 +1,57 @@ +# FW6-TOUCH — Natural touch scrolling + explicit keyboard/cursor modes (Wave 6 punch-list) + +Lane: FW6-TOUCH · Machine: homelinux · Worktree: `~/dev/wt/ac-fw6-touch` · Branch: `refactor/fw6-touch` (off `refactor/frontend-command-center`; local, do NOT push) +Owner device finding (S25 Ultra, production): "The scrolling barely works... combining the touch to focus and touch to scroll doesn't work very well." Owner interview locked all three fixes below — do not re-litigate. + +## Root causes (verified by AI Lead) + +1. SGR-wheel emulation is quantized: tmux scrolls 5 lines per wheel report → ~125px of finger travel per visible jump with a dead zone under 5 lines. +2. Every terminal tap focuses xterm's textarea → Android keyboard pops → viewport/PTY resize → tmux re-flows mid-read. +3. The 450ms stationary hold arms cursor-drag, so a slow deliberate scroll start sends arrow-key spam instead of scrolling. + +## Design (owner-locked) + +### 1. agentd-native scroll (replaces SGR-wheel emulation for tmux terminals) + +- Extend `terminal.navigate` with `{op: 'scroll', lines: }` (clamp |lines| ≤ 120 per message; positive = scroll down toward live, negative = scroll up into history). Additive schema variant + fixture + CP relay (relay code is generic over navigate — schema + tests only). +- agentd `Navigate` handling (under the manager lock, against the VIEWER session's current pane P — resolve `#{pane_id}` of the viewer's active pane): + - Read `#{pane_in_mode} #{alternate_on} #{mouse_any_flag}` for P in ONE display-message call. + - If `pane_in_mode=1` → `send-keys -X -t P -N <|lines|> scroll-up|scroll-down`. (Entry always uses `copy-mode -e`, so scrolling down past the bottom exits copy-mode naturally.) + - Else if `alternate_on=1 && mouse_any_flag=1` (full-screen TUI that wants mouse, e.g. codex) → deliver SGR wheel to the APP: `send-keys -t P -l` with literal `\x1b[<64;C;RM` / `\x1b[<65;C;RM` bytes, one event per 3 lines (remainder carried by the client), C;R = center cell of P. This mirrors tmux's own wheel passthrough; the app scrolls its own transcript. + - Else if `alternate_on=1` (alt-screen app without mouse) → send Up/Down arrows ×|lines| (tmux's alternate-scroll parity). + - Else (normal pane, live view): lines<0 → `copy-mode -e -t P` then `send-keys -X -N <|lines|> scroll-up`; lines>0 → no-op (already at live). + - Zoom/Focus does not change any of this (P is the zoomed pane). Go tests on private sockets for all four branches + the copy-mode auto-exit round trip. +- Dashboard dispatcher (`useTerminalTouchScroll`): for connected writable tmux attachments (has `tmuxSessionKey`), convert vertical drag + momentum pixel deltas to line deltas (existing lineHeight math, now 1:1) and send `navigate {op:'scroll'}` via the controller, COALESCED per animation frame (sum lines, one message per frame, skip zero). Keep: normal-buffer → local `scrollLines` (unchanged); alternate-buffer WITHOUT a tmux navigate path → keep the existing SGR-wheel input fallback. Remove the 5-line report mapping from the tmux path (it stays only in the fallback). + +### 2. Explicit keyboard (tap never summons) + +- On mobile, xterm's textarea gets `inputMode='none'` by default — taps still focus/deliver key events but never pop the Android keyboard. +- New rail key `keyboard` (icon/label like the existing special keysyms, placed in the DEFAULT ultra-minimal set): toggles typing mode — sets `inputMode='text'` + focuses (keyboard opens); toggle off returns to `'none'`. Pressed state visible (like sticky-Ctrl). Detach/leave resets to off. Desktop behavior unchanged (inputMode untouched ≥1024px). +- The PromptComposer keeps its own input untouched. Hardware keyboards keep working in both modes (focus is preserved). + +### 3. Explicit cursor-drag (rail-armed) + +- DELETE the 450ms cursor-arm timer path from `useTerminalTouchScroll` (and the `terminal-cursor-armed` stand-down event it dispatched — the 500ms context-menu long-press no longer has a competitor; leave the context menu and touch-selection overlay code otherwise untouched). +- New rail key `cursor` (in the EXPANDED set, not the minimal default): arms cursor-drag for the NEXT touch gesture — that drag runs the existing `synthesizeCursorDragInput` machinery, then auto-disarms on gesture end. Pressed state visible while armed. +- Plain touches therefore always scroll. Keep pinch-zoom, horizontal pan/swipe, selection, and context-menu behavior exactly as-is. + +## Work items + +1. Schema + fixture: `scroll` navigate variant (+ CP schema test). +2. agentd: scroll op with the four-branch pane-state dispatch + Go tests (private sockets). +3. Dashboard: touch dispatcher rework (frame-coalesced navigate scroll; fallback preservation; cursor-timer removal) + unit tests for the mapping/gating/coalescing (pure functions exported, same style as today). +4. Rail: `keyboard` + `cursor` special keys, wiring in TerminalView/TerminalSurface (`inputMode` control), settings/config plumbing, pressed-state UI; update rail config tests. +5. Tests: extend journeys — swipe on attached terminal emits `navigate scroll` frames (no SGR in the tmux path), keyboard key toggles `inputMode`, no keyboard summon on plain tap (assert `inputMode` stays `'none'` after tap), cursor key arms exactly one gesture. All suites green. + +## Ownership firewall + +You may edit: `agents/agentd/**`, `packages/ac-schema/**` (additive), `services/control-plane/**` (tests only unless the relay genuinely needs a change), `apps/dashboard/src/(components/(terminal|tmux|mobile)|hooks|stores)/**`, `tests/fixtures/protocol/**` (new), `tests/journeys/**`, related tests. You may NOT edit: `deploy/**`, launch/orchestrator components, the descriptor-key/attachment contract in `terminalHostStore.ts`, letterbox logic, Focus/zoom logic in `mobileFocus.ts`/agentd zoom ownership (extend `Navigate` alongside, don't restructure), the scrollback pager, gesture timers you aren't explicitly told to remove. + +## Gates + +Full chain: `pnpm lint && pnpm typecheck && pnpm test:ci && pnpm test:smoke:dashboard && pnpm test:journeys && pnpm build` + Go build/vet/test in agents/agentd. Playwright from a tmux TTY. Commit per work item, prefix `feat(touch):`. ≤3 attempts per failure then hold. Note: two known journey load-flakes (roster attach/type input race; launch-sheet fill timeout) — if one fails, re-run it isolated before counting an attempt. + +## Done + +Handoff `tasks/frontend-ux-handoffs/fw6-touch.md`, committed, then print exactly: +`FW6-TOUCH FROZEN ` From 02e8d2827c7dd55ad679880c7c3989ce23d59b3c Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 10:44:20 -0400 Subject: [PATCH 03/12] docs(program): device finding 5 triage + FW6-TOUCH launch receipt Co-Authored-By: Claude Fable 5 --- tasks/frontend-ux-log.md | 1 + 1 file changed, 1 insertion(+) diff --git a/tasks/frontend-ux-log.md b/tasks/frontend-ux-log.md index 601ef92..7f465cb 100644 --- a/tasks/frontend-ux-log.md +++ b/tasks/frontend-ux-log.md @@ -58,3 +58,4 @@ - ~06:50Z (07-21) DEVICE FINDING 4 (owner): cannot swipe-scroll attached tmux history on mobile. AI Lead diagnosis (two empirical probes, byte-level): attached tmux client is ALWAYS on the alternate screen (smcup on every attach) where xterm scrollLines() is a no-op — live swipe scroll-back NEVER worked in attached mode; pre-Wave-6 the stream view (DOM scroll) masked it, attach-everywhere+Focus exposed it. Both hosts have tmux mouse on: probe proved SGR wheel reports fed to the client PTY scroll copy-mode even while ZOOMED, exit cleanly at bottom, and pass through to alt-screen TUIs (codex transcript) — native tmux semantics; desktop trackpad already works via xterm wheel→mouse-report translation, touch is the only gap. Second defect found during diagnosis: post-navigate History pager captures the ORIGINAL attachment session (TerminalView renders from attachmentDescriptor). FW6-SCROLL lane launched (brief 7172477): touch → SGR wheel dispatcher (gated on alt-buffer + mouseTracking + writable; scrollLines fallback preserved) + historySessionId threading. Pure dashboard change, no protocol/agentd. - ~07:40Z (07-21) FW6-SCROLL froze at `3f085d9` (attempt 1, 18 min, 4 impl commits): SGR wheel dispatcher (5-lines-per-report remainder mapping, touch-cell targeting, full gating matrix unit-tested) + historySessionId threading. AI Lead review: SHIP clean, firewall clean. INTEGRATED @ 4579e8e. Gate attempt 1 red: 1 journey flake (roster attach/type input race — passed 10/10 isolated). Deep-dive surfaced a LATENT test race: scrollback-pager journey asserted the OLDEST line right after open, passing only when the bottom-anchor rAF lost to layout — product behavior correct, test timing-sensitive (1/10 failure isolated). Fixed deterministically (assert count, pin viewport top, then content; 10/10 green). Launch-rail mobile timeout confirmed pure load flake (10/10 isolated green) — journey-suite stabilization pass backlogged. GATE ATTEMPT 2 GREEN (full chain). Lane window/worktree cleaned. PR next; no agentd changes, no binary rollout needed. - ~13:45Z (07-21) FW6-SCROLL SHIPPED: PR #94 MERGED (checks passed, main b98a93a) + DEPLOYED (SOURCE_COMMIT verified, health ok, agents:2). No agentd/binary changes. Owner phone swipe test on an attached pane = acceptance for device finding 4. +- ~14:15Z (07-21) DEVICE FINDING 5 (owner): scroll "barely works" — gesture conflicts. AI Lead triage: (1) 5-line wheel quantization (~125px per jump + dead zone), (2) every tap focuses xterm textarea → Android keyboard pops → PTY resize mid-read, (3) 450ms cursor-arm hijacks slow scroll starts into arrow-key spam. Owner interview locked: agentd-native scroll op over terminal.navigate (copy-mode scroll-up/down N with pane-state dispatch: in-mode / alt+mouse app passthrough via send-keys -l SGR / alt no-mouse arrows / live-view copy-mode entry), explicit keyboard rail key (inputMode none by default on mobile), cursor-drag armed by rail key only (450ms timer DELETED). FW6-TOUCH lane launched (brief ed501a0). agentd rollout will be required. From 500232dfc37121b2d31d6d91681d79fc076d838c Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 10:46:49 -0400 Subject: [PATCH 04/12] feat(touch): add native terminal scroll contract --- agents/agentd/internal/protocol/terminal.go | 1 + packages/ac-schema/src/message.ts | 5 +++++ packages/ac-schema/src/terminal.ts | 5 +++++ .../ac-schema/tests/protocol-fixtures.test.ts | 1 + packages/ac-schema/tests/terminal.test.ts | 15 +++++++++++++++ .../control-plane/tests/terminalRoute.test.ts | 2 ++ .../protocol/terminal-navigate-scroll.json | 10 ++++++++++ 7 files changed, 39 insertions(+) create mode 100644 tests/fixtures/protocol/terminal-navigate-scroll.json diff --git a/agents/agentd/internal/protocol/terminal.go b/agents/agentd/internal/protocol/terminal.go index 612c1d6..5e8bc20 100644 --- a/agents/agentd/internal/protocol/terminal.go +++ b/agents/agentd/internal/protocol/terminal.go @@ -27,6 +27,7 @@ type TerminalNavigatePayload struct { WindowIndex *int `json:"window_index,omitempty"` PaneID string `json:"pane_id,omitempty"` On *bool `json:"on,omitempty"` + Lines *int `json:"lines,omitempty"` } type TerminalChannelPayload struct { diff --git a/packages/ac-schema/src/message.ts b/packages/ac-schema/src/message.ts index 8047558..5d2edb6 100644 --- a/packages/ac-schema/src/message.ts +++ b/packages/ac-schema/src/message.ts @@ -352,6 +352,11 @@ export const TerminalNavigatePayloadSchema = z.discriminatedUnion('op', [ op: z.literal('zoom'), on: z.boolean(), }), + z.object({ + channel_id: z.string().uuid(), + op: z.literal('scroll'), + lines: z.number().int().min(-120).max(120), + }), ]); export type TerminalNavigatePayload = z.infer; diff --git a/packages/ac-schema/src/terminal.ts b/packages/ac-schema/src/terminal.ts index 7f9708a..6de8418 100644 --- a/packages/ac-schema/src/terminal.ts +++ b/packages/ac-schema/src/terminal.ts @@ -42,6 +42,11 @@ export const BrowserTerminalNavigateMessageSchema = z.discriminatedUnion('op', [ op: z.literal('zoom'), on: z.boolean(), }), + z.object({ + type: z.literal('navigate'), + op: z.literal('scroll'), + lines: z.number().int().min(-120).max(120), + }), ]); export type BrowserTerminalNavigateMessage = z.infer; diff --git a/packages/ac-schema/tests/protocol-fixtures.test.ts b/packages/ac-schema/tests/protocol-fixtures.test.ts index a7e7a56..593e229 100644 --- a/packages/ac-schema/tests/protocol-fixtures.test.ts +++ b/packages/ac-schema/tests/protocol-fixtures.test.ts @@ -45,6 +45,7 @@ describe('protocol fixtures', () => { 'terminal-attach-letterbox.json', 'terminal-input.json', 'terminal-navigate.json', + 'terminal-navigate-scroll.json', 'commands-dispatch-send-input.json', ])('validates server-to-agent message fixture %s', (name) => { expect(ServerToAgentMessageSchema.safeParse(readFixture(name)).success).toBe(true); diff --git a/packages/ac-schema/tests/terminal.test.ts b/packages/ac-schema/tests/terminal.test.ts index 0e05178..f1e0204 100644 --- a/packages/ac-schema/tests/terminal.test.ts +++ b/packages/ac-schema/tests/terminal.test.ts @@ -24,6 +24,11 @@ describe('terminal protocol', () => { op: 'zoom', on: true, })).toMatchObject({ op: 'zoom', on: true }); + expect(BrowserTerminalClientMessageSchema.parse({ + type: 'navigate', + op: 'scroll', + lines: -120, + })).toMatchObject({ op: 'scroll', lines: -120 }); expect(BrowserTerminalClientMessageSchema.safeParse({ type: 'navigate', @@ -33,6 +38,16 @@ describe('terminal protocol', () => { type: 'navigate', op: 'zoom', }).success).toBe(false); + expect(BrowserTerminalClientMessageSchema.safeParse({ + type: 'navigate', + op: 'scroll', + lines: -121, + }).success).toBe(false); + expect(BrowserTerminalClientMessageSchema.safeParse({ + type: 'navigate', + op: 'scroll', + lines: 121, + }).success).toBe(false); }); it('accepts the additive binary, resume, sizing, idle, and lag messages', () => { diff --git a/services/control-plane/tests/terminalRoute.test.ts b/services/control-plane/tests/terminalRoute.test.ts index 2edd7b6..9fe8e6a 100644 --- a/services/control-plane/tests/terminalRoute.test.ts +++ b/services/control-plane/tests/terminalRoute.test.ts @@ -513,6 +513,7 @@ describe('terminal websocket route', () => { socket.send(JSON.stringify({ type: 'navigate', op: 'select_window', window_index: 2 })); socket.send(JSON.stringify({ type: 'navigate', op: 'select_pane', pane_id: '%7' })); socket.send(JSON.stringify({ type: 'navigate', op: 'zoom', on: true })); + socket.send(JSON.stringify({ type: 'navigate', op: 'scroll', lines: -37 })); await eventually(() => { const navigations = agentSend.mock.calls @@ -522,6 +523,7 @@ describe('terminal websocket route', () => { { channel_id: attach?.payload.channel_id, op: 'select_window', window_index: 2 }, { channel_id: attach?.payload.channel_id, op: 'select_pane', pane_id: '%7' }, { channel_id: attach?.payload.channel_id, op: 'zoom', on: true }, + { channel_id: attach?.payload.channel_id, op: 'scroll', lines: -37 }, ]); }); diff --git a/tests/fixtures/protocol/terminal-navigate-scroll.json b/tests/fixtures/protocol/terminal-navigate-scroll.json new file mode 100644 index 0000000..2031ed6 --- /dev/null +++ b/tests/fixtures/protocol/terminal-navigate-scroll.json @@ -0,0 +1,10 @@ +{ + "v": 1, + "type": "terminal.navigate", + "ts": "2026-07-21T16:00:00.000Z", + "payload": { + "channel_id": "33333333-3333-4333-8333-333333333333", + "op": "scroll", + "lines": -37 + } +} From 6616c89c478082ed7d9facfc2bb6ebe208ff906d Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 10:52:13 -0400 Subject: [PATCH 05/12] feat(touch): dispatch native scroll in agentd --- agents/agentd/cmd/agentd/main.go | 6 + .../agentd/cmd/agentd/main_terminal_test.go | 10 + .../agentd/internal/tmux/pty_bridge_test.go | 228 ++++++++++++++++++ agents/agentd/internal/tmux/terminal.go | 80 ++++++ 4 files changed, 324 insertions(+) diff --git a/agents/agentd/cmd/agentd/main.go b/agents/agentd/cmd/agentd/main.go index 2bd00e6..f2f6197 100644 --- a/agents/agentd/cmd/agentd/main.go +++ b/agents/agentd/cmd/agentd/main.go @@ -5509,6 +5509,12 @@ func (a *Agent) handleTerminalNavigate(payload json.RawMessage) { return } navigation.On = *req.On + case tmux.NavigateScroll: + if req.Lines == nil { + log.Printf("Failed to parse terminal.navigate: scroll requires lines") + return + } + navigation.Lines = *req.Lines default: log.Printf("Failed to parse terminal.navigate: unsupported op %q", req.Op) return diff --git a/agents/agentd/cmd/agentd/main_terminal_test.go b/agents/agentd/cmd/agentd/main_terminal_test.go index 7f56eba..8c01013 100644 --- a/agents/agentd/cmd/agentd/main_terminal_test.go +++ b/agents/agentd/cmd/agentd/main_terminal_test.go @@ -97,6 +97,16 @@ func TestHandleTerminalNavigateRoutesToGroupedViewerOnPrivateSocket(t *testing.T t.Fatal(err) } agent.handleMessage(protocol.TypeTerminalNavigate, payload) + scrollLines := 1 + payload, err = json.Marshal(protocol.TerminalNavigatePayload{ + ChannelID: "navigate-channel", + Op: "scroll", + Lines: &scrollLines, + }) + if err != nil { + t.Fatal(err) + } + agent.handleMessage(protocol.TypeTerminalNavigate, payload) panes, err = tmuxClient.ListPanes() if err != nil { diff --git a/agents/agentd/internal/tmux/pty_bridge_test.go b/agents/agentd/internal/tmux/pty_bridge_test.go index dd21bee..f960b55 100644 --- a/agents/agentd/internal/tmux/pty_bridge_test.go +++ b/agents/agentd/internal/tmux/pty_bridge_test.go @@ -2,8 +2,13 @@ package tmux import ( "encoding/base64" + "errors" + "fmt" "io" + "os" + "path/filepath" "reflect" + "strconv" "strings" "sync" "testing" @@ -253,6 +258,87 @@ func TestTerminalNavigateZoomSetsRequestedStateOnPrivateTmuxSocket(t *testing.T) assertWindowZoomed(t, client, false) } +func TestTerminalNavigateScrollsExistingCopyModeOnPrivateTmuxSocket(t *testing.T) { + manager, runner, paneID := newScrollablePrivateViewer(t, "copy-scroll") + if err := runner.Run("copy-mode", "-e", "-t", paneID); err != nil { + t.Fatalf("enter copy mode: %v", err) + } + before := privatePaneScrollPosition(t, runner, paneID) + if err := manager.Navigate("copy-scroll", TerminalNavigation{Op: NavigateScroll, Lines: -7}); err != nil { + t.Fatalf("scroll existing copy mode: %v", err) + } + after := privatePaneScrollPosition(t, runner, paneID) + if after-before != 7 { + t.Fatalf("copy-mode scroll position changed by %d, want 7 (before=%d after=%d)", after-before, before, after) + } +} + +func TestTerminalNavigateScrollEntersAndAutoExitsCopyModeOnPrivateTmuxSocket(t *testing.T) { + manager, runner, paneID := newScrollablePrivateViewer(t, "live-scroll") + if err := manager.Navigate("live-scroll", TerminalNavigation{Op: NavigateScroll, Lines: 12}); err != nil { + t.Fatalf("scroll down at live view: %v", err) + } + if got := privatePaneFormat(t, runner, paneID, "#{pane_in_mode}"); got != "0" { + t.Fatalf("live scroll-down entered copy mode: pane_in_mode=%q", got) + } + + if err := manager.Navigate("live-scroll", TerminalNavigation{Op: NavigateScroll, Lines: -5}); err != nil { + t.Fatalf("scroll live view into history: %v", err) + } + if got := privatePaneFormat(t, runner, paneID, "#{pane_in_mode}"); got != "1" { + t.Fatalf("scroll-up did not enter copy mode: pane_in_mode=%q", got) + } + if got := privatePaneScrollPosition(t, runner, paneID); got != 5 { + t.Fatalf("entered copy mode at scroll position %d, want 5", got) + } + + if err := manager.Navigate("live-scroll", TerminalNavigation{Op: NavigateScroll, Lines: 120}); err != nil { + t.Fatalf("scroll copy mode back to live: %v", err) + } + if got := privatePaneFormat(t, runner, paneID, "#{pane_in_mode}"); got != "0" { + t.Fatalf("copy-mode -e did not auto-exit at live view: pane_in_mode=%q", got) + } +} + +func TestTerminalNavigateScrollPassesWheelReportsToMouseAppOnPrivateTmuxSocket(t *testing.T) { + manager, runner, paneID, capturePath := newAlternateCaptureViewer(t, "mouse-scroll", true) + state := strings.Split(privatePaneFormat( + t, runner, paneID, "#{alternate_on}\t#{mouse_any_flag}\t#{pane_width}\t#{pane_height}", + ), "\t") + if len(state) != 4 || state[0] != "1" || state[1] != "1" { + t.Fatalf("mouse app state=%q, want alternate+mouse", strings.Join(state, "\t")) + } + width, widthErr := strconv.Atoi(state[2]) + height, heightErr := strconv.Atoi(state[3]) + if widthErr != nil || heightErr != nil { + t.Fatalf("parse mouse app dimensions %q: width=%v height=%v", state, widthErr, heightErr) + } + report := fmt.Sprintf("\x1b[<64;%d;%dM", (width+1)/2, (height+1)/2) + want := report + report + + if err := manager.Navigate("mouse-scroll", TerminalNavigation{Op: NavigateScroll, Lines: -7}); err != nil { + t.Fatalf("scroll mouse app: %v", err) + } + if got := waitForCapturedInput(t, capturePath, len(want)); got != want { + t.Fatalf("mouse app input=%q, want %q", got, want) + } +} + +func TestTerminalNavigateScrollSendsArrowsToAlternateAppOnPrivateTmuxSocket(t *testing.T) { + manager, runner, paneID, capturePath := newAlternateCaptureViewer(t, "alternate-scroll", false) + if got := privatePaneFormat(t, runner, paneID, "#{alternate_on}\t#{mouse_any_flag}"); got != "1\t0" { + t.Fatalf("alternate app state=%q, want alternate without mouse", got) + } + want := strings.Repeat("\x1b[A", 4) + + if err := manager.Navigate("alternate-scroll", TerminalNavigation{Op: NavigateScroll, Lines: -4}); err != nil { + t.Fatalf("scroll alternate app: %v", err) + } + if got := waitForCapturedInput(t, capturePath, len(want)); got != want { + t.Fatalf("alternate app input=%q, want %q", got, want) + } +} + func TestTerminalDetachUnzoomsChannelAppliedFocusOnPrivateTmuxSocket(t *testing.T) { client, _ := newPrivateTmuxClient(t) panes, err := client.ListPanes() @@ -305,6 +391,31 @@ func TestTerminalNavigationUnzoomsAppliedFocusBeforeSwitchingWindows(t *testing. } } +func TestTerminalNavigateScrollClampsAndRejectsReadOnlyViewers(t *testing.T) { + runner := newFakeTmuxRunner() + runner.outputs["display-message -p -t %7 #{session_name}\t#{window_index}\t#{pane_index}"] = []byte("agents\t2\t1\n") + stateFormat := "#{pane_id}\t#{pane_in_mode}\t#{alternate_on}\t#{mouse_any_flag}\t#{pane_width}\t#{pane_height}" + runner.outputs["display-message -p -t ac-view-controller "+stateFormat] = []byte("%7\t1\t0\t0\t80\t24\n") + manager := newTerminalManagerWithRunner(nil, runner, t.TempDir()) + defer manager.Close() + if _, err := manager.AttachWithOptions("controller", "%7", AttachOptions{SessionID: "session-1"}); err != nil { + t.Fatalf("attach controller: %v", err) + } + if _, err := manager.AttachWithOptions("readonly", "%7", AttachOptions{SessionID: "session-1"}); err != nil { + t.Fatalf("attach read-only viewer: %v", err) + } + + if err := manager.Navigate("controller", TerminalNavigation{Op: NavigateScroll, Lines: -999}); err != nil { + t.Fatalf("scroll controller: %v", err) + } + if !runner.hasRun([]string{"send-keys", "-X", "-t", "%7", "-N", "120", "scroll-up"}) { + t.Fatalf("scroll was not clamped to 120 lines; runs=%v", runner.runs) + } + if err := manager.Navigate("readonly", TerminalNavigation{Op: NavigateScroll, Lines: -3}); !errors.Is(err, ErrReadOnly) { + t.Fatalf("read-only scroll error=%v, want ErrReadOnly", err) + } +} + func TestTerminalDroppedAndSweptViewerUnzoomsAppliedFocus(t *testing.T) { runner := newFakeTmuxRunner() runner.outputs["display-message -p -t %7 #{session_name}\t#{window_index}\t#{pane_index}"] = []byte("agents\t2\t1\n") @@ -394,6 +505,123 @@ func assertWindowZoomed(t *testing.T, client *Client, want bool) { } } +func newScrollablePrivateViewer(t *testing.T, channelID string) (*TerminalManager, TmuxRunner, string) { + t.Helper() + client, _ := newPrivateTmuxClient(t) + panes, err := client.ListPanes() + if err != nil || len(panes) != 1 { + t.Fatalf("initial private panes=%+v err=%v", panes, err) + } + paneID := panes[0].PaneID + runner := newExecTmuxRunner(client) + command := "i=0; while [ $i -lt 200 ]; do printf 'SCROLL-%03d\\n' \"$i\"; i=$((i + 1)); done; exec sleep 30" + if err := runner.Run("respawn-pane", "-k", "-t", paneID, command); err != nil { + t.Fatalf("seed scrollable pane: %v", err) + } + deadline := time.Now().Add(3 * time.Second) + for privatePaneScrollHistory(t, runner, paneID) < 100 { + if time.Now().After(deadline) { + t.Fatalf("pane history did not become scrollable") + } + time.Sleep(10 * time.Millisecond) + } + + manager := newTerminalManagerWithRunner(client, runner, t.TempDir()) + t.Cleanup(manager.Close) + if _, err := manager.AttachWithOptions(channelID, paneID, AttachOptions{SessionID: "session-1"}); err != nil { + t.Fatalf("attach scroll viewer: %v", err) + } + return manager, runner, paneID +} + +func newAlternateCaptureViewer(t *testing.T, channelID string, mouse bool) (*TerminalManager, TmuxRunner, string, string) { + t.Helper() + client, _ := newPrivateTmuxClient(t) + panes, err := client.ListPanes() + if err != nil || len(panes) != 1 { + t.Fatalf("initial private panes=%+v err=%v", panes, err) + } + paneID := panes[0].PaneID + runner := newExecTmuxRunner(client) + tempDir := t.TempDir() + capturePath := filepath.Join(tempDir, "input.bin") + scriptPath := filepath.Join(tempDir, "capture-input.sh") + modes := "\\033[?1049h" + if mouse { + modes += "\\033[?1003h" + } + script := fmt.Sprintf("#!/bin/sh\nstty raw -echo\nprintf '%s'\nexec cat > \"$1\"\n", modes) + if err := os.WriteFile(scriptPath, []byte(script), 0700); err != nil { + t.Fatalf("write capture app: %v", err) + } + if err := runner.Run("respawn-pane", "-k", "-t", paneID, fmt.Sprintf("%q %q", scriptPath, capturePath)); err != nil { + t.Fatalf("start capture app: %v", err) + } + wantState := "1\t0" + if mouse { + wantState = "1\t1" + } + deadline := time.Now().Add(3 * time.Second) + for privatePaneFormat(t, runner, paneID, "#{alternate_on}\t#{mouse_any_flag}") != wantState { + if time.Now().After(deadline) { + t.Fatalf("capture app did not reach state %q", wantState) + } + time.Sleep(10 * time.Millisecond) + } + + manager := newTerminalManagerWithRunner(client, runner, t.TempDir()) + t.Cleanup(manager.Close) + if _, err := manager.AttachWithOptions(channelID, paneID, AttachOptions{SessionID: "session-1"}); err != nil { + t.Fatalf("attach alternate viewer: %v", err) + } + return manager, runner, paneID, capturePath +} + +func privatePaneFormat(t *testing.T, runner TmuxRunner, paneID, format string) string { + t.Helper() + output, err := runner.Output("display-message", "-p", "-t", paneID, format) + if err != nil { + t.Fatalf("read private pane format %q: %v", format, err) + } + return strings.TrimSpace(string(output)) +} + +func privatePaneScrollHistory(t *testing.T, runner TmuxRunner, paneID string) int { + t.Helper() + value, err := strconv.Atoi(privatePaneFormat(t, runner, paneID, "#{history_size}")) + if err != nil { + t.Fatalf("parse private pane history: %v", err) + } + return value +} + +func privatePaneScrollPosition(t *testing.T, runner TmuxRunner, paneID string) int { + t.Helper() + value, err := strconv.Atoi(privatePaneFormat(t, runner, paneID, "#{scroll_position}")) + if err != nil { + t.Fatalf("parse private pane scroll position: %v", err) + } + return value +} + +func waitForCapturedInput(t *testing.T, capturePath string, length int) string { + t.Helper() + deadline := time.Now().Add(3 * time.Second) + for { + data, err := os.ReadFile(capturePath) + if err == nil && len(data) >= length { + return string(data) + } + if err != nil && !os.IsNotExist(err) { + t.Fatalf("read captured input: %v", err) + } + if time.Now().After(deadline) { + t.Fatalf("captured input did not reach %d bytes", length) + } + time.Sleep(10 * time.Millisecond) + } +} + func TestViewerSessionNameAvoidsSanitizationCollisions(t *testing.T) { first := viewerSessionName("channel/one") second := viewerSessionName("channel:one") diff --git a/agents/agentd/internal/tmux/terminal.go b/agents/agentd/internal/tmux/terminal.go index 73555c1..9982846 100644 --- a/agents/agentd/internal/tmux/terminal.go +++ b/agents/agentd/internal/tmux/terminal.go @@ -10,6 +10,7 @@ import ( "os" "os/exec" "path/filepath" + "strconv" "strings" "sync" "syscall" @@ -95,8 +96,11 @@ const ( NavigateSelectWindow TerminalNavigationOp = "select_window" NavigateSelectPane TerminalNavigationOp = "select_pane" NavigateZoom TerminalNavigationOp = "zoom" + NavigateScroll TerminalNavigationOp = "scroll" ) +const maxTerminalScrollLines = 120 + // TerminalNavigation describes an operation against a channel's grouped tmux // session. It never targets the origin session directly. type TerminalNavigation struct { @@ -104,6 +108,7 @@ type TerminalNavigation struct { WindowIndex int PaneID string On bool + Lines int } // AttachOptions contains additive terminal.attach fields. @@ -787,11 +792,86 @@ func (m *TerminalManager) Navigate(channelID string, navigation TerminalNavigati } viewer.zoomApplied = navigation.On return nil + case NavigateScroll: + if m.channelReadOnly[channelID] { + return ErrReadOnly + } + return m.navigateScrollLocked(bridge, navigation.Lines) default: return fmt.Errorf("unsupported terminal navigation op %q", navigation.Op) } } +func (m *TerminalManager) navigateScrollLocked(bridge *viewerPTYBridge, lines int) error { + if lines > maxTerminalScrollLines { + lines = maxTerminalScrollLines + } else if lines < -maxTerminalScrollLines { + lines = -maxTerminalScrollLines + } + if lines == 0 { + return nil + } + + const paneStateFormat = "#{pane_id}\t#{pane_in_mode}\t#{alternate_on}\t#{mouse_any_flag}\t#{pane_width}\t#{pane_height}" + state, err := m.runner.Output("display-message", "-p", "-t", bridge.viewSession, paneStateFormat) + if err != nil { + return fmt.Errorf("read viewer scroll state: %w", err) + } + parts := strings.Split(strings.TrimSpace(string(state)), "\t") + if len(parts) != 6 || parts[0] == "" { + return fmt.Errorf("unexpected viewer scroll state %q", strings.TrimSpace(string(state))) + } + paneID := parts[0] + count := lines + direction := "scroll-down" + if count < 0 { + count = -count + direction = "scroll-up" + } + + if parts[1] == "1" { + return m.runner.Run("send-keys", "-X", "-t", paneID, "-N", strconv.Itoa(count), direction) + } + + alternateOn := parts[2] == "1" + mouseOn := parts[3] == "1" + if alternateOn && mouseOn { + events := count / 3 + if events == 0 { + return nil + } + width, widthErr := strconv.Atoi(parts[4]) + height, heightErr := strconv.Atoi(parts[5]) + if widthErr != nil || heightErr != nil || width < 1 || height < 1 { + return fmt.Errorf("unexpected viewer pane dimensions %q x %q", parts[4], parts[5]) + } + button := 65 + if lines < 0 { + button = 64 + } + column := (width + 1) / 2 + row := (height + 1) / 2 + report := fmt.Sprintf("\x1b[<%d;%d;%dM", button, column, row) + return m.runner.Run("send-keys", "-t", paneID, "-l", strings.Repeat(report, events)) + } + + if alternateOn { + key := "Down" + if lines < 0 { + key = "Up" + } + return m.runner.Run("send-keys", "-t", paneID, "-N", strconv.Itoa(count), key) + } + + if lines > 0 { + return nil + } + if err := m.runner.Run("copy-mode", "-e", "-t", paneID); err != nil { + return fmt.Errorf("enter viewer copy mode: %w", err) + } + return m.runner.Run("send-keys", "-X", "-t", paneID, "-N", strconv.Itoa(count), "scroll-up") +} + func (m *TerminalManager) unzoomViewerLocked(viewer *terminalViewer, reason string) error { if viewer == nil || !viewer.zoomApplied || viewer.bridge == nil { return nil From 4d01a5ae4813cca675bf2dff51923e6fda639fd0 Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 10:59:42 -0400 Subject: [PATCH 06/12] feat(touch): dispatch coalesced native scroll --- .../dashboard/src/components/TerminalView.tsx | 8 +- .../components/mobile/TerminalContextMenu.tsx | 10 - .../terminal/PersistentTerminalHost.tsx | 1 + .../components/tmux/TmuxTerminalWorkspace.tsx | 1 + .../src/hooks/useTerminalTouchScroll.test.ts | 114 +++++++++++ .../src/hooks/useTerminalTouchScroll.ts | 188 ++++++++++++++---- 6 files changed, 268 insertions(+), 54 deletions(-) diff --git a/apps/dashboard/src/components/TerminalView.tsx b/apps/dashboard/src/components/TerminalView.tsx index 38a90e1..10fac4b 100644 --- a/apps/dashboard/src/components/TerminalView.tsx +++ b/apps/dashboard/src/components/TerminalView.tsx @@ -27,6 +27,7 @@ interface TerminalViewProps { historySessionId?: string; hostId?: string; paneId?: string; + tmuxSessionKey?: string; className?: string; autoAttach?: boolean; controllerRef?: MutableRefObject; @@ -43,6 +44,7 @@ export function TerminalView({ historySessionId = sessionId, hostId, paneId, + tmuxSessionKey, className, autoAttach = false, controllerRef, @@ -231,16 +233,20 @@ export function TerminalView({ stickyCtrlModeRef.current = 'inactive'; setStickyCtrlMode('inactive'); }, [status]); + const handleNavigateScroll = useCallback((lines: number) => { + navigate({ type: 'navigate', op: 'scroll', lines }); + }, [navigate]); const touchScrollRef = useTerminalTouchScroll({ enabled: isMobile, termRef, terminalRef, fontSize: terminalFontSize, onFontSizeChange: setTerminalFontSize, - cursorEnabled: terminalWritable, onCursorInput: sendInput, writable: terminalWritable, onScrollInput: sendInput, + tmuxSessionKey, + onNavigateScroll: handleNavigateScroll, onHorizontalSwipe: (direction) => { termRef.current?.dispatchEvent(new CustomEvent('terminal-window-swipe', { bubbles: true, diff --git a/apps/dashboard/src/components/mobile/TerminalContextMenu.tsx b/apps/dashboard/src/components/mobile/TerminalContextMenu.tsx index 6aa5bd0..5e047a6 100644 --- a/apps/dashboard/src/components/mobile/TerminalContextMenu.tsx +++ b/apps/dashboard/src/components/mobile/TerminalContextMenu.tsx @@ -165,15 +165,6 @@ export function TerminalContextMenu({ container.addEventListener('touchcancel', handleTouchEnd); container.addEventListener('contextmenu', handleContextMenu); container.addEventListener('terminal-cursor-mode-start', handleCursorModeStart); - // Cursor mode armed at 450ms — cancel our 500ms menu timer so a hold-to- - // move-cursor gesture never flashes the copy/paste menu. - const handleCursorArmed = () => { - if (longPressTimerRef.current) { - clearTimeout(longPressTimerRef.current); - longPressTimerRef.current = null; - } - }; - container.addEventListener('terminal-cursor-armed', handleCursorArmed); return () => { container.removeEventListener('touchstart', handleTouchStart); @@ -182,7 +173,6 @@ export function TerminalContextMenu({ container.removeEventListener('touchcancel', handleTouchEnd); container.removeEventListener('contextmenu', handleContextMenu); container.removeEventListener('terminal-cursor-mode-start', handleCursorModeStart); - container.removeEventListener('terminal-cursor-armed', handleCursorArmed); if (longPressTimerRef.current) { clearTimeout(longPressTimerRef.current); diff --git a/apps/dashboard/src/components/terminal/PersistentTerminalHost.tsx b/apps/dashboard/src/components/terminal/PersistentTerminalHost.tsx index c9075ad..ff56123 100644 --- a/apps/dashboard/src/components/terminal/PersistentTerminalHost.tsx +++ b/apps/dashboard/src/components/terminal/PersistentTerminalHost.tsx @@ -176,6 +176,7 @@ export function PersistentTerminalHost() { historySessionId={selectedSessionId} hostId={attachmentDescriptor.hostId} paneId={attachmentDescriptor.paneId} + tmuxSessionKey={attachmentDescriptor.tmuxSessionKey} autoAttach={attachmentDescriptor.autoAttach} onControllerChange={handleControllerChange} onTerminalInstanceChange={handleTerminalInstanceChange} diff --git a/apps/dashboard/src/components/tmux/TmuxTerminalWorkspace.tsx b/apps/dashboard/src/components/tmux/TmuxTerminalWorkspace.tsx index 91711a0..203be58 100644 --- a/apps/dashboard/src/components/tmux/TmuxTerminalWorkspace.tsx +++ b/apps/dashboard/src/components/tmux/TmuxTerminalWorkspace.tsx @@ -218,6 +218,7 @@ export function TmuxTerminalWorkspace({ sessionId={secondarySession.id} hostId={secondarySession.host_id} paneId={secondarySession.tmux_pane_id || undefined} + tmuxSessionKey={getTmuxViewerSessionKey(secondarySession)} autoAttach className="flex-1" /> diff --git a/apps/dashboard/src/hooks/useTerminalTouchScroll.test.ts b/apps/dashboard/src/hooks/useTerminalTouchScroll.test.ts index b73315f..0333e57 100644 --- a/apps/dashboard/src/hooks/useTerminalTouchScroll.test.ts +++ b/apps/dashboard/src/hooks/useTerminalTouchScroll.test.ts @@ -4,8 +4,11 @@ import { buildSgrWheelReports, calculatePinchFontSize, dispatchTerminalTouchScroll, + mapTouchScrollPixelsToLines, mapScrollLinesToWheelReports, + reduceTerminalScrollCoalescer, resolveTerminalHorizontalSwipe, + resolveTerminalTouchScrollPath, resolveTouchCell, synthesizeCursorDragInput, } from './useTerminalTouchScroll'; @@ -98,6 +101,94 @@ describe('terminal touch wheel synthesis', () => { }); }); +describe('native terminal touch scroll mapping', () => { + it('maps pixel motion to one line per row while carrying sub-line pixels', () => { + expect(mapTouchScrollPixelsToLines(8, 0, 16)).toEqual({ lines: 0, remainder: 8 }); + expect(mapTouchScrollPixelsToLines(12, 8, 16)).toEqual({ lines: 1, remainder: 4 }); + expect(mapTouchScrollPixelsToLines(-36, 4, 16)).toEqual({ lines: -2, remainder: 0 }); + }); + + it('gates native navigation to writable alternate tmux attachments', () => { + expect( + resolveTerminalTouchScrollPath({ + bufferType: 'normal', + mouseTrackingMode: 'any', + writable: true, + hasTmuxNavigate: true, + }) + ).toBe('local'); + expect( + resolveTerminalTouchScrollPath({ + bufferType: 'alternate', + mouseTrackingMode: 'none', + writable: true, + hasTmuxNavigate: true, + }) + ).toBe('navigate'); + expect( + resolveTerminalTouchScrollPath({ + bufferType: 'alternate', + mouseTrackingMode: 'drag', + writable: true, + hasTmuxNavigate: false, + }) + ).toBe('sgr'); + expect( + resolveTerminalTouchScrollPath({ + bufferType: 'alternate', + mouseTrackingMode: 'none', + writable: true, + hasTmuxNavigate: false, + }) + ).toBe('local'); + expect( + resolveTerminalTouchScrollPath({ + bufferType: 'alternate', + mouseTrackingMode: 'drag', + writable: false, + hasTmuxNavigate: true, + }) + ).toBe('none'); + }); + + it('coalesces one batch per frame, skips zero, and carries values beyond the schema cap', () => { + let transition = reduceTerminalScrollCoalescer( + { pendingLines: 0, scheduled: false }, + { type: 'enqueue', lines: -70 } + ); + expect(transition).toMatchObject({ schedule: true, sendLines: 0 }); + + transition = reduceTerminalScrollCoalescer(transition.state, { type: 'enqueue', lines: -80 }); + expect(transition).toMatchObject({ + state: { pendingLines: -150, scheduled: true }, + schedule: false, + sendLines: 0, + }); + + transition = reduceTerminalScrollCoalescer(transition.state, { type: 'flush' }); + expect(transition).toMatchObject({ + state: { pendingLines: -30, scheduled: true }, + schedule: true, + sendLines: -120, + }); + transition = reduceTerminalScrollCoalescer(transition.state, { type: 'flush' }); + expect(transition).toMatchObject({ + state: { pendingLines: 0, scheduled: false }, + schedule: false, + sendLines: -30, + }); + + transition = reduceTerminalScrollCoalescer( + { pendingLines: 0, scheduled: false }, + { type: 'enqueue', lines: 4 } + ); + transition = reduceTerminalScrollCoalescer(transition.state, { type: 'enqueue', lines: -4 }); + transition = reduceTerminalScrollCoalescer(transition.state, { type: 'flush' }); + expect(transition.sendLines).toBe(0); + expect(transition.schedule).toBe(false); + }); +}); + describe('terminal touch scroll dispatch', () => { it('keeps normal-buffer scrolling in xterm', () => { const terminal = terminalForScroll('normal', 'none'); @@ -143,6 +234,29 @@ describe('terminal touch scroll dispatch', () => { expect(onInput).toHaveBeenCalledWith('\x1b[<64;12;7M'); }); + it('sends one-to-one native navigate lines for an attached tmux terminal', () => { + const terminal = terminalForScroll('alternate', 'drag'); + const onInput = vi.fn(); + const onNavigateScroll = vi.fn(); + + expect( + dispatchTerminalTouchScroll({ + terminal, + lineDelta: -2, + wheelRemainder: 4, + column: 12, + row: 7, + writable: true, + onInput, + hasTmuxNavigate: true, + onNavigateScroll, + }) + ).toBe(0); + expect(onNavigateScroll).toHaveBeenCalledWith(-2); + expect(onInput).not.toHaveBeenCalled(); + expect(terminal.scrollLines).not.toHaveBeenCalled(); + }); + it('falls back to xterm scrolling when the alternate buffer has no mouse tracking', () => { const terminal = terminalForScroll('alternate', 'none'); const onInput = vi.fn(); diff --git a/apps/dashboard/src/hooks/useTerminalTouchScroll.ts b/apps/dashboard/src/hooks/useTerminalTouchScroll.ts index 8263f84..f153049 100644 --- a/apps/dashboard/src/hooks/useTerminalTouchScroll.ts +++ b/apps/dashboard/src/hooks/useTerminalTouchScroll.ts @@ -19,7 +19,6 @@ type TouchScrollState = { pinching: boolean; pinchStartDistance: number; pinchStartFontSize: number; - cursorTimer: number | null; cursorArmed: boolean; cursorMode: boolean; cursorSentX: number; @@ -29,9 +28,30 @@ type TouchScrollState = { wheelColumn: number; wheelRow: number; panning: boolean; + navigateRaf: number | null; + navigateCoalescer: TerminalScrollCoalescerState; }; const TMUX_LINES_PER_WHEEL_REPORT = 5; +const MAX_NAVIGATE_SCROLL_LINES = 120; + +export type TerminalTouchScrollPath = 'local' | 'navigate' | 'sgr' | 'none'; + +export type TerminalScrollCoalescerState = { + pendingLines: number; + scheduled: boolean; +}; + +export type TerminalScrollCoalescerTransition = { + state: TerminalScrollCoalescerState; + schedule: boolean; + sendLines: number; +}; + +const INITIAL_SCROLL_COALESCER: TerminalScrollCoalescerState = { + pendingLines: 0, + scheduled: false, +}; function createTouchScrollState(): TouchScrollState { return { @@ -49,7 +69,6 @@ function createTouchScrollState(): TouchScrollState { pinching: false, pinchStartDistance: 0, pinchStartFontSize: 14, - cursorTimer: null, cursorArmed: false, cursorMode: false, cursorSentX: 0, @@ -59,6 +78,8 @@ function createTouchScrollState(): TouchScrollState { wheelColumn: 1, wheelRow: 1, panning: false, + navigateRaf: null, + navigateCoalescer: { ...INITIAL_SCROLL_COALESCER }, }; } @@ -111,6 +132,66 @@ export function mapScrollLinesToWheelReports( }; } +export function mapTouchScrollPixelsToLines( + deltaPixels: number, + remainder: number, + lineHeight: number +): { lines: number; remainder: number } { + const safeLineHeight = lineHeight > 0 ? lineHeight : 16; + const total = deltaPixels + remainder; + const lines = Math.trunc(total / safeLineHeight); + return { lines, remainder: total - lines * safeLineHeight }; +} + +export function resolveTerminalTouchScrollPath({ + bufferType, + mouseTrackingMode, + writable, + hasTmuxNavigate, +}: { + bufferType: 'normal' | 'alternate'; + mouseTrackingMode: 'none' | 'x10' | 'vt200' | 'drag' | 'any'; + writable: boolean; + hasTmuxNavigate: boolean; +}): TerminalTouchScrollPath { + if (bufferType === 'normal') return 'local'; + if (writable && hasTmuxNavigate) return 'navigate'; + if (mouseTrackingMode !== 'none') return writable ? 'sgr' : 'none'; + return 'local'; +} + +export function reduceTerminalScrollCoalescer( + state: TerminalScrollCoalescerState, + event: { type: 'enqueue'; lines: number } | { type: 'flush' } +): TerminalScrollCoalescerTransition { + if (event.type === 'enqueue') { + const pendingLines = state.pendingLines + event.lines; + const schedule = !state.scheduled && pendingLines !== 0; + return { + state: { + pendingLines, + scheduled: state.scheduled || schedule, + }, + schedule, + sendLines: 0, + }; + } + + const sendLines = Math.max( + -MAX_NAVIGATE_SCROLL_LINES, + Math.min(MAX_NAVIGATE_SCROLL_LINES, state.pendingLines) + ); + const pendingLines = state.pendingLines - sendLines; + return { + state: { + pendingLines, + scheduled: pendingLines !== 0, + }, + schedule: pendingLines !== 0, + sendLines, + }; +} + export function resolveTouchCell( clientX: number, clientY: number, @@ -148,6 +229,8 @@ export function dispatchTerminalTouchScroll({ row, writable, onInput, + hasTmuxNavigate = false, + onNavigateScroll = () => undefined, }: { terminal: XTerminal; lineDelta: number; @@ -156,15 +239,19 @@ export function dispatchTerminalTouchScroll({ row: number; writable: boolean; onInput: (data: string) => void; + hasTmuxNavigate?: boolean; + onNavigateScroll?: (lines: number) => void; }): number { - const usesMouseReports = terminal.buffer.active.type === 'alternate' - && terminal.modes.mouseTrackingMode !== 'none'; + const path = resolveTerminalTouchScrollPath({ + bufferType: terminal.buffer.active.type, + mouseTrackingMode: terminal.modes.mouseTrackingMode, + writable, + hasTmuxNavigate, + }); - if (!usesMouseReports) { - terminal.scrollLines(lineDelta); - return 0; - } - if (!writable) return 0; + if (path === 'local') terminal.scrollLines(lineDelta); + if (path === 'navigate') onNavigateScroll(lineDelta); + if (path !== 'sgr') return 0; const mapped = mapScrollLinesToWheelReports(lineDelta, wheelRemainder); const data = buildSgrWheelReports(mapped.reportDelta, column, row); @@ -189,10 +276,11 @@ export function useTerminalTouchScroll({ terminalRef, fontSize, onFontSizeChange, - cursorEnabled, onCursorInput, writable = false, onScrollInput = onCursorInput, + tmuxSessionKey, + onNavigateScroll, onHorizontalSwipe, }: { enabled: boolean; @@ -200,26 +288,29 @@ export function useTerminalTouchScroll({ terminalRef: RefObject; fontSize: number; onFontSizeChange: (fontSize: number) => void; - cursorEnabled: boolean; onCursorInput: (data: string) => void; writable?: boolean; onScrollInput?: (data: string) => void; + tmuxSessionKey?: string; + onNavigateScroll?: (lines: number) => void; onHorizontalSwipe?: (direction: 'previous' | 'next') => void; }) { const touchScrollRef = useRef(createTouchScrollState()); const fontSizeRef = useRef(fontSize); const onFontSizeChangeRef = useRef(onFontSizeChange); - const cursorEnabledRef = useRef(cursorEnabled); const onCursorInputRef = useRef(onCursorInput); const writableRef = useRef(writable); const onScrollInputRef = useRef(onScrollInput); + const tmuxSessionKeyRef = useRef(tmuxSessionKey); + const onNavigateScrollRef = useRef(onNavigateScroll); const onHorizontalSwipeRef = useRef(onHorizontalSwipe); fontSizeRef.current = fontSize; onFontSizeChangeRef.current = onFontSizeChange; - cursorEnabledRef.current = cursorEnabled; onCursorInputRef.current = onCursorInput; writableRef.current = writable; onScrollInputRef.current = onScrollInput; + tmuxSessionKeyRef.current = tmuxSessionKey; + onNavigateScrollRef.current = onNavigateScroll; onHorizontalSwipeRef.current = onHorizontalSwipe; useEffect(() => { @@ -250,11 +341,28 @@ export function useTerminalTouchScroll({ return terminal.cols > 0 && width > 0 ? width / terminal.cols : 8; }; - const clearCursorTimer = () => { - if (touchState.cursorTimer !== null) { - window.clearTimeout(touchState.cursorTimer); - touchState.cursorTimer = null; - } + const scheduleNavigateFlush = () => { + if (touchState.navigateRaf !== null) return; + touchState.navigateRaf = requestAnimationFrame(() => { + touchState.navigateRaf = null; + const transition = reduceTerminalScrollCoalescer(touchState.navigateCoalescer, { + type: 'flush', + }); + touchState.navigateCoalescer = transition.state; + if (transition.sendLines !== 0) { + onNavigateScrollRef.current?.(transition.sendLines); + } + if (transition.schedule) scheduleNavigateFlush(); + }); + }; + + const enqueueNavigateScroll = (lines: number) => { + const transition = reduceTerminalScrollCoalescer(touchState.navigateCoalescer, { + type: 'enqueue', + lines, + }); + touchState.navigateCoalescer = transition.state; + if (transition.schedule) scheduleNavigateFlush(); }; const handleTouchStart = (event: TouchEvent) => { @@ -265,7 +373,6 @@ export function useTerminalTouchScroll({ touchState.momentumRaf = null; } if (event.touches.length === 2) { - clearCursorTimer(); touchState.cursorArmed = false; touchState.cursorMode = false; const first = event.touches[0]; @@ -310,19 +417,6 @@ export function useTerminalTouchScroll({ touchState.cursorSentX = 0; touchState.cursorSentY = 0; touchState.panning = false; - clearCursorTimer(); - if (cursorEnabledRef.current) { - touchState.cursorTimer = window.setTimeout(() => { - touchState.cursorTimer = null; - if (!touchState.active || touchState.axis) return; - touchState.cursorArmed = true; - // Tell the context menu's competing 500ms long-press timer to stand - // down: a stationary hold that armed cursor mode must not also pop - // the copy/paste menu 50ms later. - container.dispatchEvent(new CustomEvent('terminal-cursor-armed')); - navigator.vibrate?.(8); - }, 450); - } }; const handleTouchMove = (event: TouchEvent) => { @@ -379,8 +473,6 @@ export function useTerminalTouchScroll({ } if (terminal.hasSelection()) return; - if (!touchState.cursorArmed && movedDistance > 10) clearCursorTimer(); - if (!touchState.axis) { const absDy = Math.abs(totalDy); const absDx = Math.abs(totalDx); @@ -442,10 +534,13 @@ export function useTerminalTouchScroll({ event.preventDefault(); const deltaY = touch.clientY - touchState.lastY; - const lineHeight = touchState.lineHeight > 0 ? touchState.lineHeight : 16; - const totalDelta = deltaY + touchState.remainder; - const lines = Math.trunc(totalDelta / lineHeight); - touchState.remainder = totalDelta - lines * lineHeight; + const mapped = mapTouchScrollPixelsToLines( + deltaY, + touchState.remainder, + touchState.lineHeight + ); + const lines = mapped.lines; + touchState.remainder = mapped.remainder; if (lines !== 0) { touchState.wheelRemainder = dispatchTerminalTouchScroll({ terminal, @@ -455,6 +550,8 @@ export function useTerminalTouchScroll({ row: touchState.wheelRow, writable: writableRef.current, onInput: onScrollInputRef.current, + hasTmuxNavigate: Boolean(tmuxSessionKeyRef.current && onNavigateScrollRef.current), + onNavigateScroll: enqueueNavigateScroll, }); } const now = performance.now(); @@ -468,7 +565,6 @@ export function useTerminalTouchScroll({ }; const handleTouchEnd = (event: TouchEvent) => { - clearCursorTimer(); if (touchState.pinching) { if (event.touches.length < 2) touchState.pinching = false; touchState.active = false; @@ -518,7 +614,6 @@ export function useTerminalTouchScroll({ let currentVelocity = velocity; let accum = 0; let lastFrame = performance.now(); - const lineHeight = touchState.lineHeight > 0 ? touchState.lineHeight : 16; const tick = () => { const terminal = terminalRef.current; @@ -535,7 +630,9 @@ export function useTerminalTouchScroll({ return; } accum += currentVelocity * dt; - const lines = Math.trunc(accum / lineHeight); + const mapped = mapTouchScrollPixelsToLines(0, accum, touchState.lineHeight); + const lines = mapped.lines; + accum = mapped.remainder; if (lines !== 0) { touchState.wheelRemainder = dispatchTerminalTouchScroll({ terminal, @@ -545,8 +642,9 @@ export function useTerminalTouchScroll({ row: touchState.wheelRow, writable: writableRef.current, onInput: onScrollInputRef.current, + hasTmuxNavigate: Boolean(tmuxSessionKeyRef.current && onNavigateScrollRef.current), + onNavigateScroll: enqueueNavigateScroll, }); - accum -= lines * lineHeight; } touchState.momentumRaf = requestAnimationFrame(tick); }; @@ -567,7 +665,11 @@ export function useTerminalTouchScroll({ cancelAnimationFrame(touchState.momentumRaf); touchState.momentumRaf = null; } - clearCursorTimer(); + if (touchState.navigateRaf !== null) { + cancelAnimationFrame(touchState.navigateRaf); + touchState.navigateRaf = null; + } + touchState.navigateCoalescer = { ...INITIAL_SCROLL_COALESCER }; }; }, [enabled, termRef, terminalRef]); From 8bfb225c4daea4095f7ee8b4d41efc4ff015b12e Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 11:05:25 -0400 Subject: [PATCH 07/12] feat(touch): add keyboard and cursor rail modes --- .../dashboard/src/components/TerminalView.tsx | 51 +++++++++++++++++-- .../src/components/mobile/TerminalKeyRail.tsx | 35 +++++++++++-- .../mobile/terminalRailConfig.test.ts | 14 +++++ .../components/mobile/terminalRailConfig.ts | 8 +++ .../components/terminal/TerminalSurface.tsx | 34 ++++++++++++- .../terminal/terminalHostStore.test.ts | 1 + .../src/components/terminal/types.ts | 1 + .../src/components/tmux/TmuxMobileShell.tsx | 4 +- .../tmux/accessibilityContracts.test.ts | 7 ++- .../src/hooks/useTerminalTouchScroll.ts | 19 ++++++- 10 files changed, 164 insertions(+), 10 deletions(-) diff --git a/apps/dashboard/src/components/TerminalView.tsx b/apps/dashboard/src/components/TerminalView.tsx index 10fac4b..d2706d8 100644 --- a/apps/dashboard/src/components/TerminalView.tsx +++ b/apps/dashboard/src/components/TerminalView.tsx @@ -13,7 +13,7 @@ import { type StickyCtrlEvent, type StickyCtrlMode, } from '@/components/mobile/stickyCtrl'; -import { useIsMobile } from '@/hooks/useIsMobile'; +import { COMMAND_CENTER_SHELL_BREAKPOINT, useIsMobile } from '@/hooks/useIsMobile'; import { useTerminalClipboard } from '@/hooks/useTerminalClipboard'; import { useTerminalConnection } from '@/hooks/useTerminalConnection'; import { useTerminalScrollAnchor } from '@/hooks/useTerminalScrollAnchor'; @@ -60,6 +60,8 @@ export function TerminalView({ const [searchOpen, setSearchOpen] = useState(false); const [historyOpen, setHistoryOpen] = useState(false); const [stickyCtrlMode, setStickyCtrlMode] = useState('inactive'); + const [keyboardActive, setKeyboardActive] = useState(false); + const [cursorArmed, setCursorArmed] = useState(false); const [searchQuery, setSearchQuery] = useState(''); const [searchResults, setSearchResults] = useState({ resultIndex: -1, @@ -70,7 +72,10 @@ export function TerminalView({ const selectionPopupRef = useRef(null); const hasCommittedSelectionRef = useRef(false); const stickyCtrlModeRef = useRef('inactive'); + const keyboardActiveRef = useRef(false); + const cursorArmedRef = useRef(false); const isMobile = useIsMobile(); + const touchInputModeEnabled = useIsMobile(COMMAND_CENTER_SHELL_BREAKPOINT); const terminalFontSize = useSettingsStore((state) => state.terminalFontSize); const setTerminalFontSize = useSettingsStore((state) => state.setTerminalFontSize); const tmuxPrefix = useSettingsStore((state) => ( @@ -228,21 +233,54 @@ export function TerminalView({ useEffect(() => { sendInputRef.current = handlePhysicalInput; }, [handlePhysicalInput]); + const applyTouchInputMode = useCallback((active: boolean) => { + if (!touchInputModeEnabled) return; + const textarea = termRef.current?.querySelector('.xterm-helper-textarea'); + if (textarea) textarea.inputMode = active ? 'text' : 'none'; + }, [touchInputModeEnabled]); + const resetTouchModes = useCallback(() => { + keyboardActiveRef.current = false; + cursorArmedRef.current = false; + setKeyboardActive(false); + setCursorArmed(false); + applyTouchInputMode(false); + }, [applyTouchInputMode]); + const handleKeyboardToggle = useCallback(() => { + const next = !keyboardActiveRef.current; + keyboardActiveRef.current = next; + setKeyboardActive(next); + applyTouchInputMode(next); + terminalRef.current?.focus(); + }, [applyTouchInputMode, terminalRef]); + const handleCursorToggle = useCallback(() => { + const next = !cursorArmedRef.current; + cursorArmedRef.current = next; + setCursorArmed(next); + terminalRef.current?.focus(); + }, [terminalRef]); + const handleCursorDisarm = useCallback(() => { + if (!cursorArmedRef.current) return; + cursorArmedRef.current = false; + setCursorArmed(false); + }, []); useEffect(() => { if (status === 'connected') return; stickyCtrlModeRef.current = 'inactive'; setStickyCtrlMode('inactive'); - }, [status]); + resetTouchModes(); + }, [resetTouchModes, status]); const handleNavigateScroll = useCallback((lines: number) => { navigate({ type: 'navigate', op: 'scroll', lines }); }, [navigate]); const touchScrollRef = useTerminalTouchScroll({ - enabled: isMobile, + enabled: touchInputModeEnabled, termRef, terminalRef, fontSize: terminalFontSize, onFontSizeChange: setTerminalFontSize, + cursorArmed, onCursorInput: sendInput, + onCursorDisarm: handleCursorDisarm, writable: terminalWritable, onScrollInput: sendInput, tmuxSessionKey, @@ -281,6 +319,7 @@ export function TerminalView({ suspend, takeControl, navigate, + resetTouchModes, focus: () => terminalRef.current?.focus(), copySelection, copyLastLines: (lines = 50) => copyLastLines(lines), @@ -311,6 +350,7 @@ export function TerminalView({ onControllerChange, navigate, paste, + resetTouchModes, terminalRef, suspend, takeControl, @@ -364,6 +404,7 @@ export function TerminalView({ termRef={termRef} terminalRef={terminalRef} isMobile={isMobile} + touchInputModeEnabled={touchInputModeEnabled} status={status} readOnly={readOnly} selectionTextRef={selectionTextRef} @@ -371,6 +412,10 @@ export function TerminalView({ onSelectionStart={handleSelectionStart} onSelectionCommit={handleSelectionCommit} onVirtualInput={handleVirtualInput} + keyboardActive={keyboardActive} + onKeyboardToggle={handleKeyboardToggle} + cursorArmed={cursorArmed} + onCursorToggle={handleCursorToggle} stickyCtrlMode={stickyCtrlMode} onStickyCtrlEvent={dispatchStickyCtrl} onOpenHistory={() => setHistoryOpen(true)} diff --git a/apps/dashboard/src/components/mobile/TerminalKeyRail.tsx b/apps/dashboard/src/components/mobile/TerminalKeyRail.tsx index 15c76d3..65cb3ca 100644 --- a/apps/dashboard/src/components/mobile/TerminalKeyRail.tsx +++ b/apps/dashboard/src/components/mobile/TerminalKeyRail.tsx @@ -4,6 +4,7 @@ import { useEffect, useMemo, useRef, useState, type PointerEvent } from 'react'; import { cn } from '@/lib/utils'; import { useSettingsStore } from '@/stores/settings'; import { + EXPANDED_TERMINAL_RAIL_CONFIG, MINIMAL_TERMINAL_RAIL_CONFIG, isArrowRailKey, resolveTerminalRailBinding, @@ -22,6 +23,10 @@ interface TerminalKeyRailProps { onPreviousMark?: () => void; onNextMark?: () => void; hasCommandMarks?: boolean; + keyboardActive: boolean; + onKeyboardToggle: () => void; + cursorArmed: boolean; + onCursorToggle: () => void; } type RailGesture = { @@ -44,9 +49,18 @@ export function TerminalKeyRail({ onPreviousMark, onNextMark, hasCommandMarks = false, + keyboardActive, + onKeyboardToggle, + cursorArmed, + onCursorToggle, }: TerminalKeyRailProps) { + const railPreset = useSettingsStore((state) => state.terminalRailPreset); const configuredRail = useSettingsStore((state) => state.terminalRailConfig); - const config = configuredRail?.keys?.length ? configuredRail : MINIMAL_TERMINAL_RAIL_CONFIG; + const config = railPreset === 'minimal' + ? MINIMAL_TERMINAL_RAIL_CONFIG + : railPreset === 'expanded' + ? EXPANDED_TERMINAL_RAIL_CONFIG + : configuredRail?.keys?.length ? configuredRail : MINIMAL_TERMINAL_RAIL_CONFIG; const [navLayer, setNavLayer] = useState(false); const gestureRef = useRef(null); const longPressTimerRef = useRef(null); @@ -88,6 +102,8 @@ export function TerminalKeyRail({ const action = resolveTerminalRailBinding(binding, tmuxPrefixToSequence(prefix)); hapticTick(); if (action.type === 'input') onInput(action.data); + else if (action.type === 'keyboard') onKeyboardToggle(); + else if (action.type === 'cursor') onCursorToggle(); else if (action.type === 'history') onHistory(); else if (action.type === 'mark') { if (action.direction === 'previous') onPreviousMark?.(); @@ -209,6 +225,11 @@ export function TerminalKeyRail({ {visibleKeys.map((key, index) => { const sourceKey = config.keys[index] ?? key; const isCtrl = sourceKey.binding.type === 'keysym' && sourceKey.binding.value === 'ctrl'; + const isKeyboard = sourceKey.binding.type === 'keysym' + && sourceKey.binding.value === 'keyboard'; + const isCursor = sourceKey.binding.type === 'keysym' + && sourceKey.binding.value === 'cursor'; + const isPressed = isCtrl ? ctrlActive : isKeyboard ? keyboardActive : isCursor ? cursorArmed : false; const isCommandMark = sourceKey.binding.type === 'keysym' && (sourceKey.binding.value === 'previous_mark' || sourceKey.binding.value === 'next_mark'); return ( @@ -219,10 +240,18 @@ export function TerminalKeyRail({ 'flex h-11 min-w-11 items-center justify-center rounded-md border bg-background px-2 text-xs font-semibold outline-none transition-colors active:scale-95 focus-visible:ring-2 focus-visible:ring-ring disabled:opacity-40', config.keys.length > 6 && 'shrink-0', isCtrl && ctrlActive && 'border-primary bg-primary text-primary-foreground', + (isKeyboard || isCursor) && isPressed + && 'border-primary bg-primary text-primary-foreground', navLayer && isArrowRailKey(sourceKey) && 'border-primary/60 bg-primary/10 text-primary' )} - aria-label={isCtrl ? `Control modifier ${ctrlMode}` : key.label} - aria-pressed={isCtrl ? ctrlActive : undefined} + aria-label={isCtrl + ? `Control modifier ${ctrlMode}` + : isKeyboard + ? `Keyboard ${keyboardActive ? 'on' : 'off'}` + : isCursor + ? `Cursor drag ${cursorArmed ? 'armed' : 'inactive'}` + : key.label} + aria-pressed={isCtrl || isKeyboard || isCursor ? isPressed : undefined} disabled={isCommandMark && !hasCommandMarks} onPointerDown={(event) => handlePointerDown(event, sourceKey)} onPointerMove={(event) => handlePointerMove(event, sourceKey)} diff --git a/apps/dashboard/src/components/mobile/terminalRailConfig.test.ts b/apps/dashboard/src/components/mobile/terminalRailConfig.test.ts index def3530..379a633 100644 --- a/apps/dashboard/src/components/mobile/terminalRailConfig.test.ts +++ b/apps/dashboard/src/components/mobile/terminalRailConfig.test.ts @@ -1,5 +1,7 @@ import { describe, expect, it } from 'vitest'; import { + EXPANDED_TERMINAL_RAIL_CONFIG, + MINIMAL_TERMINAL_RAIL_CONFIG, parseTerminalRailConfig, resolveTerminalRailBinding, } from './terminalRailConfig'; @@ -54,6 +56,18 @@ describe('terminal rail config engine', () => { }); }); + it('keeps keyboard in the minimal preset and cursor in the expanded preset only', () => { + expect(MINIMAL_TERMINAL_RAIL_CONFIG.keys.map((key) => key.id)).toContain('keyboard'); + expect(MINIMAL_TERMINAL_RAIL_CONFIG.keys.map((key) => key.id)).not.toContain('cursor'); + expect(EXPANDED_TERMINAL_RAIL_CONFIG.keys.map((key) => key.id)).toContain('cursor'); + expect(resolveTerminalRailBinding({ type: 'keysym', value: 'keyboard' })).toEqual({ + type: 'keyboard', + }); + expect(resolveTerminalRailBinding({ type: 'keysym', value: 'cursor' })).toEqual({ + type: 'cursor', + }); + }); + it('rejects duplicate ids and invalid bindings', () => { expect(() => parseTerminalRailConfig({ version: 1, diff --git a/apps/dashboard/src/components/mobile/terminalRailConfig.ts b/apps/dashboard/src/components/mobile/terminalRailConfig.ts index 3a6479b..77c7a46 100644 --- a/apps/dashboard/src/components/mobile/terminalRailConfig.ts +++ b/apps/dashboard/src/components/mobile/terminalRailConfig.ts @@ -2,6 +2,8 @@ import { DEFAULT_TMUX_PREFIX, tmuxPrefixToSequence } from '@/lib/tmuxKeys'; export const TERMINAL_RAIL_KEYSYMS = [ 'ctrl', + 'keyboard', + 'cursor', 'esc', 'tab', 'enter', @@ -43,6 +45,8 @@ export type TerminalRailPreset = 'minimal' | 'expanded' | 'custom'; export type TerminalRailAction = | { type: 'input'; data: string } | { type: 'modifier'; modifier: 'ctrl' } + | { type: 'keyboard' } + | { type: 'cursor' } | { type: 'history' } | { type: 'mark'; direction: 'previous' | 'next' }; @@ -65,6 +69,7 @@ export const MINIMAL_TERMINAL_RAIL_CONFIG: TerminalRailConfig = { keys: [ { id: 'esc', label: 'Esc', binding: { type: 'keysym', value: 'esc' } }, { id: 'ctrl', label: 'Ctrl', binding: { type: 'keysym', value: 'ctrl' } }, + { id: 'keyboard', label: 'Keyboard', binding: { type: 'keysym', value: 'keyboard' } }, { id: 'up', label: '↑', @@ -96,6 +101,7 @@ export const EXPANDED_TERMINAL_RAIL_CONFIG: TerminalRailConfig = { version: 1, keys: [ ...MINIMAL_TERMINAL_RAIL_CONFIG.keys, + { id: 'cursor', label: 'Cursor', binding: { type: 'keysym', value: 'cursor' } }, { id: 'tab', label: 'Tab', binding: { type: 'keysym', value: 'tab' } }, { id: 'prefix', label: 'Prefix', binding: { type: 'keysym', value: 'prefix' } }, { id: 'history', label: 'History', binding: { type: 'keysym', value: 'history' } }, @@ -190,6 +196,8 @@ export function resolveTerminalRailBinding( if (binding.type === 'macro') return { type: 'input', data: binding.value }; if (binding.type === 'keysym') { if (binding.value === 'ctrl') return { type: 'modifier', modifier: 'ctrl' }; + if (binding.value === 'keyboard') return { type: 'keyboard' }; + if (binding.value === 'cursor') return { type: 'cursor' }; if (binding.value === 'history') return { type: 'history' }; if (binding.value === 'previous_mark') return { type: 'mark', direction: 'previous' }; if (binding.value === 'next_mark') return { type: 'mark', direction: 'next' }; diff --git a/apps/dashboard/src/components/terminal/TerminalSurface.tsx b/apps/dashboard/src/components/terminal/TerminalSurface.tsx index d084dd2..643047f 100644 --- a/apps/dashboard/src/components/terminal/TerminalSurface.tsx +++ b/apps/dashboard/src/components/terminal/TerminalSurface.tsx @@ -1,6 +1,6 @@ 'use client'; -import type { RefObject } from 'react'; +import { useEffect, type RefObject } from 'react'; import { ArrowDown } from 'lucide-react'; import { SelectionPopup, TerminalContextMenu } from '@/components/mobile'; import type { SelectionPopupHandle } from '@/components/mobile'; @@ -17,6 +17,7 @@ interface TerminalSurfaceProps { termRef: RefObject; terminalRef: RefObject; isMobile: boolean; + touchInputModeEnabled: boolean; status: ConnectionStatus; readOnly: boolean; selectionTextRef: RefObject; @@ -24,6 +25,10 @@ interface TerminalSurfaceProps { onSelectionStart: (anchor: { x: number; y: number }) => void; onSelectionCommit: () => void; onVirtualInput: (data: string) => void; + keyboardActive: boolean; + onKeyboardToggle: () => void; + cursorArmed: boolean; + onCursorToggle: () => void; stickyCtrlMode: StickyCtrlMode; onStickyCtrlEvent: (event: StickyCtrlEvent) => void; onOpenHistory: () => void; @@ -48,6 +53,7 @@ export function TerminalSurface({ termRef, terminalRef, isMobile, + touchInputModeEnabled, status, readOnly, selectionTextRef, @@ -55,6 +61,10 @@ export function TerminalSurface({ onSelectionStart, onSelectionCommit, onVirtualInput, + keyboardActive, + onKeyboardToggle, + cursorArmed, + onCursorToggle, stickyCtrlMode, onStickyCtrlEvent, onOpenHistory, @@ -74,6 +84,24 @@ export function TerminalSurface({ onJumpToLive, search, }: TerminalSurfaceProps) { + useEffect(() => { + if (!touchInputModeEnabled) return; + const container = termRef.current; + if (!container) return; + const applyInputMode = () => { + const textarea = container.querySelector('.xterm-helper-textarea'); + if (textarea) textarea.inputMode = keyboardActive ? 'text' : 'none'; + }; + applyInputMode(); + const observer = new MutationObserver(applyInputMode); + observer.observe(container, { childList: true, subtree: true }); + return () => { + observer.disconnect(); + container.querySelector('.xterm-helper-textarea') + ?.removeAttribute('inputmode'); + }; + }, [keyboardActive, termRef, touchInputModeEnabled]); + return ( <>
@@ -168,6 +196,10 @@ export function TerminalSurface({ onPreviousMark={onPreviousMark} onNextMark={onNextMark} hasCommandMarks={hasCommandMarks} + keyboardActive={keyboardActive} + onKeyboardToggle={onKeyboardToggle} + cursorArmed={cursorArmed} + onCursorToggle={onCursorToggle} /> )} diff --git a/apps/dashboard/src/components/terminal/terminalHostStore.test.ts b/apps/dashboard/src/components/terminal/terminalHostStore.test.ts index 77b5e32..410e4bf 100644 --- a/apps/dashboard/src/components/terminal/terminalHostStore.test.ts +++ b/apps/dashboard/src/components/terminal/terminalHostStore.test.ts @@ -19,6 +19,7 @@ function controller(readOnly: boolean): TerminalController { suspend: () => false, takeControl: () => undefined, navigate: () => false, + resetTouchModes: () => undefined, focus: () => undefined, copySelection: () => undefined, copyLastLines: () => undefined, diff --git a/apps/dashboard/src/components/terminal/types.ts b/apps/dashboard/src/components/terminal/types.ts index 6a5a82a..c4c1786 100644 --- a/apps/dashboard/src/components/terminal/types.ts +++ b/apps/dashboard/src/components/terminal/types.ts @@ -20,6 +20,7 @@ export interface TerminalController { suspend: () => boolean; takeControl: () => void; navigate: (message: BrowserTerminalNavigateMessage) => boolean; + resetTouchModes: () => void; focus: () => void; copySelection: () => void; copyLastLines: (lines?: number) => void; diff --git a/apps/dashboard/src/components/tmux/TmuxMobileShell.tsx b/apps/dashboard/src/components/tmux/TmuxMobileShell.tsx index c764c34..8a2d7b4 100644 --- a/apps/dashboard/src/components/tmux/TmuxMobileShell.tsx +++ b/apps/dashboard/src/components/tmux/TmuxMobileShell.tsx @@ -244,11 +244,13 @@ export function TmuxMobileShell({ const handleLeaveTerminal = useCallback(() => { sendFocus(false); + terminalControllerRef.current?.resetTouchModes(); setMode('roster'); - }, [sendFocus]); + }, [sendFocus, terminalControllerRef]); const handleDetach = useCallback(() => { sendFocus(false); + terminalControllerRef.current?.resetTouchModes(); terminalControllerRef.current?.detach(); }, [sendFocus, terminalControllerRef]); diff --git a/apps/dashboard/src/components/tmux/accessibilityContracts.test.ts b/apps/dashboard/src/components/tmux/accessibilityContracts.test.ts index 4771e02..392e8a3 100644 --- a/apps/dashboard/src/components/tmux/accessibilityContracts.test.ts +++ b/apps/dashboard/src/components/tmux/accessibilityContracts.test.ts @@ -38,7 +38,12 @@ describe('Command Center accessibility contracts', () => { const keyRail = source('src/components/mobile/TerminalKeyRail.tsx'); expect(keyRail).toContain('role="toolbar"'); expect(keyRail).toContain('aria-label="Terminal key rail"'); - expect(keyRail).toContain('aria-pressed={isCtrl ? ctrlActive : undefined}'); + expect(keyRail).toContain( + 'aria-pressed={isCtrl || isKeyboard || isCursor ? isPressed : undefined}' + ); + + const terminalSurface = source('src/components/terminal/TerminalSurface.tsx'); + expect(terminalSurface).toContain("textarea.inputMode = keyboardActive ? 'text' : 'none'"); }); it('routes mobile sheets through the focus-trapping dialog primitive', () => { diff --git a/apps/dashboard/src/hooks/useTerminalTouchScroll.ts b/apps/dashboard/src/hooks/useTerminalTouchScroll.ts index f153049..8ba47e7 100644 --- a/apps/dashboard/src/hooks/useTerminalTouchScroll.ts +++ b/apps/dashboard/src/hooks/useTerminalTouchScroll.ts @@ -276,7 +276,9 @@ export function useTerminalTouchScroll({ terminalRef, fontSize, onFontSizeChange, + cursorArmed = false, onCursorInput, + onCursorDisarm, writable = false, onScrollInput = onCursorInput, tmuxSessionKey, @@ -288,7 +290,9 @@ export function useTerminalTouchScroll({ terminalRef: RefObject; fontSize: number; onFontSizeChange: (fontSize: number) => void; + cursorArmed?: boolean; onCursorInput: (data: string) => void; + onCursorDisarm?: () => void; writable?: boolean; onScrollInput?: (data: string) => void; tmuxSessionKey?: string; @@ -298,7 +302,9 @@ export function useTerminalTouchScroll({ const touchScrollRef = useRef(createTouchScrollState()); const fontSizeRef = useRef(fontSize); const onFontSizeChangeRef = useRef(onFontSizeChange); + const cursorArmedRef = useRef(cursorArmed); const onCursorInputRef = useRef(onCursorInput); + const onCursorDisarmRef = useRef(onCursorDisarm); const writableRef = useRef(writable); const onScrollInputRef = useRef(onScrollInput); const tmuxSessionKeyRef = useRef(tmuxSessionKey); @@ -306,7 +312,9 @@ export function useTerminalTouchScroll({ const onHorizontalSwipeRef = useRef(onHorizontalSwipe); fontSizeRef.current = fontSize; onFontSizeChangeRef.current = onFontSizeChange; + cursorArmedRef.current = cursorArmed; onCursorInputRef.current = onCursorInput; + onCursorDisarmRef.current = onCursorDisarm; writableRef.current = writable; onScrollInputRef.current = onScrollInput; tmuxSessionKeyRef.current = tmuxSessionKey; @@ -365,6 +373,11 @@ export function useTerminalTouchScroll({ if (transition.schedule) scheduleNavigateFlush(); }; + const disarmCursorGesture = () => { + cursorArmedRef.current = false; + onCursorDisarmRef.current?.(); + }; + const handleTouchStart = (event: TouchEvent) => { const terminal = terminalRef.current; if (!terminal) return; @@ -373,6 +386,7 @@ export function useTerminalTouchScroll({ touchState.momentumRaf = null; } if (event.touches.length === 2) { + if (cursorArmedRef.current) disarmCursorGesture(); touchState.cursorArmed = false; touchState.cursorMode = false; const first = event.touches[0]; @@ -412,7 +426,7 @@ export function useTerminalTouchScroll({ touchState.wheelRemainder = 0; touchState.wheelColumn = touchCell.column; touchState.wheelRow = touchCell.row; - touchState.cursorArmed = false; + touchState.cursorArmed = cursorArmedRef.current; touchState.cursorMode = false; touchState.cursorSentX = 0; touchState.cursorSentY = 0; @@ -565,6 +579,7 @@ export function useTerminalTouchScroll({ }; const handleTouchEnd = (event: TouchEvent) => { + const cursorGestureArmed = touchState.cursorArmed || touchState.cursorMode; if (touchState.pinching) { if (event.touches.length < 2) touchState.pinching = false; touchState.active = false; @@ -580,9 +595,11 @@ export function useTerminalTouchScroll({ touchState.active = false; touchState.axis = null; container.dispatchEvent(new CustomEvent('terminal-cursor-mode-end')); + disarmCursorGesture(); return; } touchState.cursorArmed = false; + if (cursorGestureArmed) disarmCursorGesture(); touchState.active = false; const axis = touchState.axis; touchState.axis = null; From 3008c251b80c79b06975c54c79503d7c4a0620b7 Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 11:09:20 -0400 Subject: [PATCH 08/12] feat(touch): cover explicit mobile touch modes --- tests/journeys/controlPlaneMock.ts | 1 + tests/journeys/fw6-focus.journey.spec.ts | 109 ++++++++++++++++++++--- 2 files changed, 98 insertions(+), 12 deletions(-) diff --git a/tests/journeys/controlPlaneMock.ts b/tests/journeys/controlPlaneMock.ts index 7d9871a..4cf9918 100644 --- a/tests/journeys/controlPlaneMock.ts +++ b/tests/journeys/controlPlaneMock.ts @@ -228,6 +228,7 @@ export interface JourneyRecorder { on?: boolean; pane_id?: string; window_index?: number; + lines?: number; }>; terminalSessionIds: string[]; terminalWebSocketUrls: string[]; diff --git a/tests/journeys/fw6-focus.journey.spec.ts b/tests/journeys/fw6-focus.journey.spec.ts index f7173d6..8a8a53e 100644 --- a/tests/journeys/fw6-focus.journey.spec.ts +++ b/tests/journeys/fw6-focus.journey.spec.ts @@ -14,14 +14,14 @@ async function selectSession(page: Page): Promise { await expect(page.getByTestId('tmux-attached-status')).toContainText('Connected'); } -async function swipeTerminalDown(page: Page): Promise { +async function dragTerminal(page: Page, deltaY: number): Promise { const terminal = page.getByLabel('Interactive terminal'); await expect(terminal).toBeVisible(); - await terminal.evaluate((element) => { + await terminal.evaluate(async (element, requestedDeltaY) => { const rect = element.getBoundingClientRect(); const clientX = rect.left + rect.width / 2; - const startY = rect.top + Math.min(100, rect.height / 4); - const endY = startY + Math.min(140, Math.max(100, rect.height / 3)); + const startY = rect.top + rect.height / 2; + const endY = Math.max(rect.top + 24, Math.min(rect.bottom - 24, startY + requestedDeltaY)); const dispatch = (type: 'touchstart' | 'touchmove' | 'touchend', clientY: number) => { const touch = new Touch({ identifier: 1, @@ -40,12 +40,17 @@ async function swipeTerminalDown(page: Page): Promise { changedTouches: [touch], })); }; + const nextFrame = () => new Promise((resolve) => requestAnimationFrame(() => resolve())); dispatch('touchstart', startY); - dispatch('touchmove', startY + 8); + await nextFrame(); + dispatch('touchmove', startY + Math.sign(requestedDeltaY) * 8); + await nextFrame(); dispatch('touchmove', endY); + await nextFrame(); dispatch('touchend', endY); - }); + await nextFrame(); + }, deltaY); } test.describe('FW6 mobile Focus journey', () => { @@ -61,17 +66,97 @@ test.describe('FW6 mobile Focus journey', () => { }); }); - test('translates a vertical touch swipe into tmux SGR wheel input', async ({ page }) => { + test('translates a vertical touch swipe into native tmux scroll frames', async ({ page }) => { await signIn(page); await selectSession(page); - await swipeTerminalDown(page); - const wheelUpReport = new RegExp(`${String.fromCharCode(27)}\\[<64;\\d+;\\d+M`); + const initialScrollCount = recorder.terminalMessages.filter( + (message) => message.type === 'navigate' && message.op === 'scroll' + ).length; + await dragTerminal(page, 140); - await expect.poll(() => recorder.terminalMessages + await expect.poll(() => recorder.terminalMessages.filter( + (message) => message.type === 'navigate' && message.op === 'scroll' + ).length).toBeGreaterThan(initialScrollCount); + const scrollFrames = recorder.terminalMessages.filter( + (message) => message.type === 'navigate' && message.op === 'scroll' + ); + expect(scrollFrames.every( + (message) => Number.isInteger(message.lines) + && message.lines !== 0 + && Math.abs(message.lines ?? 0) <= 120 + )).toBe(true); + const terminalInput = recorder.terminalMessages .filter((message) => message.type === 'input') .map((message) => message.data || '') - .join('')) - .toMatch(wheelUpReport); + .join(''); + expect(terminalInput).not.toMatch(/\x1b\[<6[45];\d+;\d+M/); + }); + + test('keeps plain taps keyboard-free until the Keyboard rail key is pressed', async ({ page }) => { + await signIn(page); + await selectSession(page); + const terminal = page.getByLabel('Interactive terminal'); + const textarea = page + .locator('[aria-label="Interactive terminal"]:visible .xterm-helper-textarea') + .first(); + const rail = page.getByTestId('terminal-key-rail'); + + await expect(textarea).toHaveAttribute('inputmode', 'none'); + await terminal.tap(); + await expect(textarea).toHaveAttribute('inputmode', 'none'); + + const keyboardOff = rail.getByRole('button', { name: 'Keyboard off' }); + await expect(keyboardOff).toHaveAttribute('aria-pressed', 'false'); + await keyboardOff.tap(); + const keyboardOn = rail.getByRole('button', { name: 'Keyboard on' }); + await expect(keyboardOn).toHaveAttribute('aria-pressed', 'true'); + await expect(textarea).toHaveAttribute('inputmode', 'text'); + + await keyboardOn.tap(); + await expect(rail.getByRole('button', { name: 'Keyboard off' })).toHaveAttribute( + 'aria-pressed', + 'false' + ); + await expect(textarea).toHaveAttribute('inputmode', 'none'); + }); + + test('arms cursor drag for exactly one touch gesture', async ({ page }) => { + await page.addInitScript(() => { + window.localStorage.setItem( + 'settings-storage', + JSON.stringify({ state: { terminalRailPreset: 'expanded' }, version: 0 }) + ); + }); + await signIn(page); + await selectSession(page); + const rail = page.getByTestId('terminal-key-rail'); + const cursorOff = rail.getByRole('button', { name: 'Cursor drag inactive' }); + const inputCount = () => recorder.terminalMessages.filter( + (message) => message.type === 'input' + ).length; + const initialInputCount = inputCount(); + + await cursorOff.tap(); + await expect(rail.getByRole('button', { name: 'Cursor drag armed' })).toHaveAttribute( + 'aria-pressed', + 'true' + ); + await dragTerminal(page, -140); + await expect.poll(inputCount).toBeGreaterThan(initialInputCount); + await expect(rail.getByRole('button', { name: 'Cursor drag inactive' })).toHaveAttribute( + 'aria-pressed', + 'false' + ); + + const inputCountAfterCursorGesture = inputCount(); + const scrollCountBeforePlainGesture = recorder.terminalMessages.filter( + (message) => message.type === 'navigate' && message.op === 'scroll' + ).length; + await dragTerminal(page, -140); + await expect.poll(() => recorder.terminalMessages.filter( + (message) => message.type === 'navigate' && message.op === 'scroll' + ).length).toBeGreaterThan(scrollCountBeforePlainGesture); + expect(inputCount()).toBe(inputCountAfterCursorGesture); }); test('loads History from the selected session after an in-place window switch', async ({ From 345935972ff93d2ce2859744f3c9967ce3f14005 Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 11:17:03 -0400 Subject: [PATCH 09/12] feat(touch): add frozen handoff --- tasks/frontend-ux-handoffs/fw6-touch.md | 71 +++++++++++++++++++++++++ 1 file changed, 71 insertions(+) create mode 100644 tasks/frontend-ux-handoffs/fw6-touch.md diff --git a/tasks/frontend-ux-handoffs/fw6-touch.md b/tasks/frontend-ux-handoffs/fw6-touch.md new file mode 100644 index 0000000..1a29c4c --- /dev/null +++ b/tasks/frontend-ux-handoffs/fw6-touch.md @@ -0,0 +1,71 @@ +--- +lane: FW6-TOUCH +frozen_sha: 3008c251b80c79b06975c54c79503d7c4a0620b7 +attempt: 1 +state: frozen +gates: + lint: pass + typecheck: pass + test_ci: pass + smoke: pass + journeys: pass + build: pass + go: pass +proof: + - "CHOKIDAR_USEPOLLING=1 WATCHPACK_POLLING=true pnpm lint && pnpm typecheck && pnpm test:ci && pnpm test:smoke:dashboard && pnpm test:journeys && pnpm build && (cd agents/agentd && go build ./... && go vet ./... && go test ./...) → exact mandatory gate passed in order on attempt 1 from the fw6-touch-builder tmux TTY" + - "pnpm lint → all 5 Turbo tasks passed with zero errors; three existing hook warnings remain" + - "pnpm typecheck → all 5 Turbo tasks passed" + - "pnpm test:ci → all 5 Turbo tasks passed, including schema, dashboard, control-plane, and CLI suites" + - "pnpm test:smoke:dashboard → 21/21 Chromium scenarios passed" + - "pnpm test:journeys → 32 passed and 14 expected project skips across mobile and desktop" + - "pnpm build → 4/4 Turbo build tasks passed; Next.js 16.2.10 production build completed" + - "go build ./... && go vet ./... && go test ./... from agents/agentd → all packages passed, including private-socket tmux integration coverage" + - "FW6 Touch focused journeys → native-scroll, keyboard, cursor, history, and zoom scenarios passed 5/5 at 412x915" + - "Playwright trace audit → mobile 412x915 and desktop 1280x720 attachment flows passed; frontend quality score 96/100 with no hard failures" +assumptions: + - "A signed scroll line uses the locked protocol convention: negative enters history and positive moves toward live." +uncertainties: + - "A physical Galaxy S25 Ultra was unavailable; the owner-device layout and touch behavior were exercised in the 412x915 Chromium mobile project." +blockers: [] +--- + +# FW6-TOUCH frozen handoff + +## Outcome + +FW6-TOUCH replaces coarse tmux wheel emulation with frame-coalesced native line scrolling and removes the two competing mobile touch behaviors. Plain terminal touches now scroll without opening the software keyboard, while Keyboard and Cursor modes are explicit rail actions with visible pressed state and deterministic reset behavior. + +## Delivered work + +- Added the bounded `terminal.navigate` scroll variant with signed integer lines, protocol fixtures, schema boundary coverage, Go fixture parity, and control-plane relay proof. +- Added viewer-scoped agentd scroll dispatch under the terminal manager lock. A single pane-state lookup selects copy-mode scrolling, literal SGR delivery for mouse-aware alternate-screen apps, arrow-key parity for non-mouse alternate-screen apps, or live normal-pane copy-mode entry. Private tmux sockets prove all four branches and copy-mode auto-exit. +- Reworked connected writable tmux touch scrolling to map pixel movement to lines 1:1, coalesce and clamp frames, preserve overflow for later frames, and skip zero-line messages. Normal-buffer scrolling remains local, and alternate-buffer attachments without a navigate path retain the existing SGR fallback. +- Removed the 450ms cursor-arm path and its stand-down event. The expanded Cursor rail key arms exactly one gesture, delegates to the existing cursor-drag synthesizer, and auto-disarms at gesture end. +- Added Keyboard to the ultra-minimal rail. On mobile, xterm's helper textarea defaults to `inputMode="none"`; Keyboard toggles it to `text` and focuses it, while toggling off, detaching, leaving, or disconnecting resets it. Desktop textareas are untouched and PromptComposer was not changed. +- Extended rail configuration, accessibility contracts, pure-function unit tests, and Playwright journeys for native scroll frames, no-SGR tmux dispatch, plain-tap keyboard suppression, keyboard toggling, and one-shot cursor gestures. + +## Decisions within the locked design + +- The existing stable host/tmux-session descriptor supplies the navigate path; its attachment-key contract was passed through without restructuring. +- Coalescing is isolated in a pure reducer: each animation frame emits at most one message bounded to 120 lines and carries any excess into a later scheduled frame. +- Built-in presets are resolved from current definitions so persisted minimal/expanded selections receive the new keys without rewriting user storage. Custom rail configurations continue to honor their saved key list. +- Mobile input-mode control is applied to the active xterm helper textarea and guarded by the existing `<1024px` breakpoint; focus is preserved so hardware keyboards continue working in either software-keyboard mode. + +## Verification notes + +- The complete required command chain passed on its first attempt from the `fw6-touch-builder` tmux TTY. No known journey load flake occurred. +- Additional trace-on browser runs passed for the 412x915 Keyboard journey and the 1280x720 attach/type/detach journey. Visual inspection found no clipping, horizontal overflow, overlap, illegible state, or desktop input regression; the audit score was 96/100 with no hard fail. +- Agentd tests used repository-private tmux sockets and did not query or mutate the default tmux server. +- The ownership firewall was preserved: no deploy, launch/orchestrator, descriptor-key contract, letterbox, Focus/zoom, scrollback pager, unrelated gesture timer, production state, or remote branch was changed. + +## Work-item commits + +- `500232d` — `feat(touch): add native terminal scroll contract` +- `6616c89` — `feat(touch): dispatch native scroll in agentd` +- `4d01a5a` — `feat(touch): dispatch coalesced native scroll` +- `8bfb225` — `feat(touch): add keyboard and cursor rail modes` +- `3008c25` — `feat(touch): cover explicit mobile touch modes` + +The `frozen_sha` is the final implementation/test commit; the following handoff-only commit adds this file. + +FW6-TOUCH FROZEN 3008c251b80c79b06975c54c79503d7c4a0620b7 From eddb27fcd4ab6b12beea9082779d13e37916e216 Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 11:20:08 -0400 Subject: [PATCH 10/12] fix(touch): carry sub-event wheel residue across scroll messages Frame-coalesced scroll messages are often 1-2 lines; stateless count/3 dropped them all, leaving slow drags inert in mouse-aware alt-screen panes (codex). Residue accumulates signed per bridge under the manager lock and resets on window/pane selection. Co-Authored-By: Claude Fable 5 --- agents/agentd/internal/tmux/terminal.go | 17 +++++++++-- .../internal/tmux/terminal_scroll_test.go | 29 +++++++++++++++++++ agents/agentd/internal/tmux/viewer_pty.go | 11 ++++++- 3 files changed, 54 insertions(+), 3 deletions(-) create mode 100644 agents/agentd/internal/tmux/terminal_scroll_test.go diff --git a/agents/agentd/internal/tmux/terminal.go b/agents/agentd/internal/tmux/terminal.go index 9982846..6b82b60 100644 --- a/agents/agentd/internal/tmux/terminal.go +++ b/agents/agentd/internal/tmux/terminal.go @@ -101,6 +101,17 @@ const ( const maxTerminalScrollLines = 120 +// Mouse-aware alternate-screen apps scroll ~3 lines per wheel event. The +// sub-event residue must carry across frame-coalesced scroll messages (which +// are often 1-2 lines each) or slow drags never produce an event. +const wheelEventLines = 3 + +func accumulateWheelEvents(residue, lines int) (events, nextResidue int) { + total := residue + lines + events = total / wheelEventLines + return events, total - events*wheelEventLines +} + // TerminalNavigation describes an operation against a channel's grouped tmux // session. It never targets the origin session directly. type TerminalNavigation struct { @@ -836,7 +847,8 @@ func (m *TerminalManager) navigateScrollLocked(bridge *viewerPTYBridge, lines in alternateOn := parts[2] == "1" mouseOn := parts[3] == "1" if alternateOn && mouseOn { - events := count / 3 + events, residue := accumulateWheelEvents(bridge.wheelResidue, lines) + bridge.wheelResidue = residue if events == 0 { return nil } @@ -846,8 +858,9 @@ func (m *TerminalManager) navigateScrollLocked(bridge *viewerPTYBridge, lines in return fmt.Errorf("unexpected viewer pane dimensions %q x %q", parts[4], parts[5]) } button := 65 - if lines < 0 { + if events < 0 { button = 64 + events = -events } column := (width + 1) / 2 row := (height + 1) / 2 diff --git a/agents/agentd/internal/tmux/terminal_scroll_test.go b/agents/agentd/internal/tmux/terminal_scroll_test.go new file mode 100644 index 0000000..b3ab2b8 --- /dev/null +++ b/agents/agentd/internal/tmux/terminal_scroll_test.go @@ -0,0 +1,29 @@ +package tmux + +import "testing" + +func TestAccumulateWheelEvents(t *testing.T) { + cases := []struct { + name string + residue int + lines int + wantEvents int + wantResidue int + }{ + {"slow drag first frame", 0, 1, 0, 1}, + {"slow drag accumulates", 2, 1, 1, 0}, + {"single big frame", 0, 10, 3, 1}, + {"downward accumulates", -2, -1, -1, 0}, + {"direction flip drains residue", 2, -3, 0, -1}, + {"zero passthrough", 1, 0, 0, 1}, + } + for _, tc := range cases { + t.Run(tc.name, func(t *testing.T) { + events, residue := accumulateWheelEvents(tc.residue, tc.lines) + if events != tc.wantEvents || residue != tc.wantResidue { + t.Fatalf("accumulateWheelEvents(%d, %d) = (%d, %d), want (%d, %d)", + tc.residue, tc.lines, events, residue, tc.wantEvents, tc.wantResidue) + } + }) + } +} diff --git a/agents/agentd/internal/tmux/viewer_pty.go b/agents/agentd/internal/tmux/viewer_pty.go index 0986393..1aeafc2 100644 --- a/agents/agentd/internal/tmux/viewer_pty.go +++ b/agents/agentd/internal/tmux/viewer_pty.go @@ -124,6 +124,10 @@ type viewerPTYBridge struct { resumeOption string readonly bool letterbox bool + // Signed lines not yet converted into app wheel events (alt+mouse panes + // scroll ~3 lines per event; discarding the sub-event residue makes slow + // drags inert). Mutated only on Navigate paths, under the manager lock. + wheelResidue int size TerminalSize process PTYProcess fanout *terminalFanout @@ -290,7 +294,11 @@ func (b *viewerPTYBridge) SelectWindow(windowIndex int) error { } target := b.viewSession + ":" + strconv.Itoa(windowIndex) - return b.selectWindowLocked(target) + if err := b.selectWindowLocked(target); err != nil { + return err + } + b.wheelResidue = 0 + return nil } func (b *viewerPTYBridge) SelectPane(paneID string) error { @@ -323,6 +331,7 @@ func (b *viewerPTYBridge) SelectPane(paneID string) error { if err := b.runner.Run("select-pane", "-t", paneTarget); err != nil { return fmt.Errorf("select viewer pane: %w", err) } + b.wheelResidue = 0 return nil } From bd5675766f0425771cf89ff180d817314e52b3dc Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 11:23:24 -0400 Subject: [PATCH 11/12] test(journeys): gate mobile attach typing on Connected state Typing raced the terminal WS handshake under parallel load (2 gate failures today). Wait for the writable Connected label before typing; 10x repeat green. Co-Authored-By: Claude Fable 5 --- tests/journeys/command-center.journey.spec.ts | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/tests/journeys/command-center.journey.spec.ts b/tests/journeys/command-center.journey.spec.ts index 93c2f7b..90d6e06 100644 --- a/tests/journeys/command-center.journey.spec.ts +++ b/tests/journeys/command-center.journey.spec.ts @@ -244,7 +244,9 @@ test.describe('Command Center program journeys', () => { await selectSession(page, interactiveSession.title); if (await isMobile(page)) { - await expect(page.getByTestId('tmux-attached-status')).toBeVisible(); + // Wait for writability, not just attachment: typing during the WS + // handshake drops keystrokes and flakes under parallel load. + await expect(page.getByTestId('tmux-attached-status')).toContainText('Connected'); await visibleTerminalInput(page).focus(); await page.keyboard.type('echo journey'); await expect.poll(() => recordedTerminalInput(recorder)).toContain('echo journey'); From 674c6860e9037ee8e96a1a99aff306d860886c71 Mon Sep 17 00:00:00 2001 From: cvsloane Date: Tue, 21 Jul 2026 11:54:06 -0400 Subject: [PATCH 12/12] docs(program): FW6-TOUCH integration receipt Co-Authored-By: Claude Fable 5 --- tasks/frontend-ux-log.md | 1 + 1 file changed, 1 insertion(+) diff --git a/tasks/frontend-ux-log.md b/tasks/frontend-ux-log.md index 7f465cb..1e4a594 100644 --- a/tasks/frontend-ux-log.md +++ b/tasks/frontend-ux-log.md @@ -59,3 +59,4 @@ - ~07:40Z (07-21) FW6-SCROLL froze at `3f085d9` (attempt 1, 18 min, 4 impl commits): SGR wheel dispatcher (5-lines-per-report remainder mapping, touch-cell targeting, full gating matrix unit-tested) + historySessionId threading. AI Lead review: SHIP clean, firewall clean. INTEGRATED @ 4579e8e. Gate attempt 1 red: 1 journey flake (roster attach/type input race — passed 10/10 isolated). Deep-dive surfaced a LATENT test race: scrollback-pager journey asserted the OLDEST line right after open, passing only when the bottom-anchor rAF lost to layout — product behavior correct, test timing-sensitive (1/10 failure isolated). Fixed deterministically (assert count, pin viewport top, then content; 10/10 green). Launch-rail mobile timeout confirmed pure load flake (10/10 isolated green) — journey-suite stabilization pass backlogged. GATE ATTEMPT 2 GREEN (full chain). Lane window/worktree cleaned. PR next; no agentd changes, no binary rollout needed. - ~13:45Z (07-21) FW6-SCROLL SHIPPED: PR #94 MERGED (checks passed, main b98a93a) + DEPLOYED (SOURCE_COMMIT verified, health ok, agents:2). No agentd/binary changes. Owner phone swipe test on an attached pane = acceptance for device finding 4. - ~14:15Z (07-21) DEVICE FINDING 5 (owner): scroll "barely works" — gesture conflicts. AI Lead triage: (1) 5-line wheel quantization (~125px per jump + dead zone), (2) every tap focuses xterm textarea → Android keyboard pops → PTY resize mid-read, (3) 450ms cursor-arm hijacks slow scroll starts into arrow-key spam. Owner interview locked: agentd-native scroll op over terminal.navigate (copy-mode scroll-up/down N with pane-state dispatch: in-mode / alt+mouse app passthrough via send-keys -l SGR / alt no-mouse arrows / live-view copy-mode entry), explicit keyboard rail key (inputMode none by default on mobile), cursor-drag armed by rail key only (450ms timer DELETED). FW6-TOUCH lane launched (brief ed501a0). agentd rollout will be required. +- ~15:30Z (07-21) FW6-TOUCH froze at `3008c25` (attempt 1, ~35 min, 5 impl commits, full green gate incl. Go). AI Lead review: SHIP with 1 integration fix — alt+mouse wheel branch divided count/3 STATELESSLY, so frame-coalesced 1-2-line messages always floored to 0 events (slow drags inert in codex panes — the exact symptom being fixed); added per-bridge signed wheelResidue accumulation (reset on window/pane select) + pure-helper Go test (eddb27f). Also stabilized the roster attach/type journey for real (gate on Connected label, 10/10 isolated) after it burned a second gate cycle. GATE ATTEMPT 2 FULLY GREEN. Lane + builder tmux session cleaned. PR next; agentd rollout required on both hosts.