Skip to content

Manual testing checklist for recent inline-chat / LLM PRs #80

@fentas

Description

@fentas

Tracking issue collecting manual-testing items for the inline-chat / LLM PRs that have shipped recently. Each item is something the test suite can't easily cover end-to-end (real terminal interactions, real shell wiring, real LLM output) — keep this open and tick items off when you've verified them in a live atty session.

After each round of PR merges, run `make link` (or wait for the release-please tag → binary rebuild) and exercise the items below in Ghostty.


#76 — Ctrl+Right stops at `/` boundaries

PR: #76

  • Type `cd ` (so ghost shows a path completion like `cd /home/fentas/github/atty`).
  • Press `Ctrl+Right` repeatedly — accepts segment-by-segment, one slash per press: `/home/`, then `fentas/`, then `github/`, then `atty`.
  • Confirm non-path ghosts still accept one space-word at a time (e.g. `git ` → `checkout ` → `master`).
  • Edge: empty input → no-op. `/foo` (no trailing slash) → accepts all 4 chars at once.

#77 — Ctrl+Up/Down focus jump (panel ↔ shell)

PR: #77

  • Open inline chat with `Alt+C`. Type a few chars in the panel input.
  • `Ctrl+Up` — focus parks on shell. Type at the shell: keystrokes go to bash, panel input stays as-was. Panel input row visually dims to show "parked."
  • `Ctrl+Down` — focus returns to panel. Typed chars again go into panel input (not bash).
  • `Ctrl+Up` when panel is CLOSED — should pass through to the shell as a normal Ctrl+Up (e.g. tmux pane nav).

#78 — Discoverability hint + statusbar AI-dialog tracking

PR: #78

  • Run `atty bash` in a fresh terminal. Press no keys for ~2s.
  • Idle-state hint should appear (compact one-line: Alt+C / Alt+S / Alt+H or similar).
  • Toggle through global shortcuts: `Alt+C`, `Alt+S`, `Alt+Shift+S`, `Alt+H` — confirm all are wired.
  • Statusbar AI-dialog regression check: enter AI mode (`#: ` prefix), send a query, see the dialog. After the dialog completes, the statusbar should still reflect "ai-dialog idle" or the resting state — not get stuck showing the in-flight indicator. (Was flagged as flaky in a long session; not yet repro'd cleanly.)

#79 — Alt+C → Alt+C cursor position

PR: #79 (this PR — the cursor-restore fix)

  • Open a bash prompt mid-screen (e.g. after `ls`). Note the prompt row.
  • Press `Alt+C` to open inline chat. Then `Alt+C` again to close. Press Enter.
  • Cursor should return to the row where the prompt was (or one row below for the new prompt), NOT jump down to where the chat input rendered.
  • Repeat with terminal scrolled (prompt at row 8 of 24, prompt at row 2, prompt at row 22): all should restore to the prompt area, never inside the reservation.
  • Type something while panel is OPEN, then close. The typed chars should appear at the shell prompt (not echoed inside the panel area).

#75 — Security guard design doc

PR: #75

(No manual testing — docs only. Listed here for traceability.)

  • Re-read `docs/security-guard-design.md` once the eBPF guard implementation starts, and check the ASCII diagram + threat-level matrix still match the architecture.

#74 — System context injection (OS / pwd / git)

PR: #74

  • Run `atty bash` outside a git repo, in a known directory. Open chat, ask "what dir am I in?". LLM should answer with the actual pwd (not stale or wrong).
  • In a git repo, ask "what branch am I on?". LLM should answer with the actual branch.
  • In a git WORKTREE (`.git` is a file, not a dir): same. Confirm the `gitdir:` indirection follow works.
  • On Ubuntu / Arch / NixOS: ask "what distro?" — should pick up the PRETTY_NAME from `/etc/os-release`.
  • Privacy: toggle incognito (`Ctrl+Shift+I` or `Alt+I`). Ask the same questions. The sys-context injection should be SUPPRESSED — LLM should not have access to pwd / git / os info.

Backlog (not yet fixed)

These are open issues / regressions noted but not in any of the PRs above:

  • Statusbar fresh-start glitch — on the very first paint after `atty bash` launch, the statusbar can be invisible until the next keystroke. Repro: open a new terminal, run `atty bash`, observe the bottom row. Sometimes shows nothing until you press a key.
  • `alt+c` from a fresh prompt with no scrollback — cursor tracking is uncertain; the panel paints but the cursor might land at the wrong row on close. The clamp in fix(llm): inline-chat paint CUP-restores cursor to shell row #79 should defend against this; verify.

How to add items

When something surprises you in a live session, add a checkbox item with: (1) what you did, (2) what you expected, (3) what actually happened. Then either fix it (PR) or leave it for triage. Close this issue when everything is ticked.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions