Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
152 changes: 152 additions & 0 deletions .agents/skills/verify-ferry/SKILL.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,152 @@
---
name: verify-ferry
description: Build and verify ferry's macOS CLI, window movement and Space following, and Raycast Script Commands using the repository's CLI tests and live desktop harness. Use when proving ferry behavior or checking a change with retained evidence.
---

# Verify ferry

Read [the feature map](features/README.md), then the feature being exercised.
The primary surface is a one-shot CLI that moves macOS windows; Raycast scripts
are a second entry point. There is no daemon, port, auth, database, or seed data.
Never substitute an installed Homebrew binary for the checkout's binary.

## Launch

Run from the repository root in a persistent shell:

```sh
FERRY_VERIFY="$PWD/.agents/skills/verify-ferry/scripts/verify.sh"
FERRY_EVIDENCE="$("$FERRY_VERIFY" launch)"
export FERRY_VERIFY FERRY_EVIDENCE
FERRY_BIN="$FERRY_EVIDENCE/scratch/ferry"
export FERRY_BIN
printf 'Evidence: %s\n' "$FERRY_EVIDENCE"
"$FERRY_VERIFY" doctor "$FERRY_EVIDENCE"
```

Launch builds with the existing Makefile into a fresh scratch directory. It needs
Apple silicon, Xcode Command Line Tools or Xcode, and a macOS 26+ SDK. The default
build is arm64, minimum macOS 26.0, version from `Makefile`. Readiness is a matching
source/binary fingerprint, correct `--version`, and passing architecture, minimum
OS, and AppKit linkage checks. A CLI invocation runs once and exits; launch does
not start a service. A failed build retains its logs and removes its scratch.

The supported runtime is macOS 26. CLI-only checks can run without Accessibility
or a desktop. Results on a newer OS are compatibility observations, not proof on
the supported OS.

No app data isolation is needed for CLI checks, but **the desktop is shared**.
Never run two movement journeys together or move an existing user window.
The `desktop` helper reserves a per-user lock, uses the existing harness's own
two windows, and restores the prior app. It cannot isolate Mission Control from
the user, yabai, hotkeys, or other tools. Do not drive while they are moving
windows. If a lock already exists, stop; do not remove someone else's lock.
For manual journeys, reserve the same lock and release only the lock you created.

## Doctor

```sh
"$FERRY_VERIFY" doctor "$FERRY_EVIDENCE"
```

This is read-only with respect to the app and desktop: it validates this run's
build and source without moving windows or requesting permissions. Run it first
when anything looks off. A source edit invalidates the run; launch a new one.
It does not assert desktop readiness. The live harness checks Accessibility
without prompting and requires two normal Spaces on the active display. A skip
returns `77` from `build/follow-test`; `make test-follow` typically returns `2`
instead, so read `desktop/stderr.txt` for `SKIP:` rather than treating it as pass.
Do not silently grant permissions, create Spaces, or alter Mission Control settings.

## Drive

CLI discovery and argument validation, in a separate real PTY:

```sh
/usr/bin/script -q "$FERRY_EVIDENCE/cli.pty.txt" \
"$FERRY_VERIFY" cli "$FERRY_EVIDENCE"
```

Require exit `0` and the `PASS:` line. The helper executes real `--help`, `-h`,
`--version`, and invalid input, verifies stream routing, and reuses `tests/cli.sh`.
All these paths return before WindowServer calls; inspect `main` in `src/ferry.m`
when changing the argument cases. There is **no dry-run flag**: a valid destination
can really move a window. Do not treat `--no-follow` as a dry run.

Background explicit-window movement, already-there behavior, and no-follow:

```sh
"$FERRY_VERIFY" desktop "$FERRY_EVIDENCE"
```

This executes `make BIN="$FERRY_BIN" test-follow`, reusing `tests/follow.m`.
Require all three `ok:` messages, no `FAIL:` or `SKIP:`, and exit `0`. The test
checks actual WindowServer membership and active Space, including that the other
fixture window stays put. It does not cover focused-window selection or Raycast.
Those journeys require native desktop UI control and the matching feature recipe.
Missing UI control, permission, or suitable Spaces is an unmet precondition;
report it instead of weakening the journey.

For a further command, preserve its action and streams with a fresh label:

```sh
"$FERRY_VERIFY" capture "$FERRY_EVIDENCE" binary-contract \
tests/binary.sh "$FERRY_BIN" arm64 26.0
```

Each label is single-use. This prevents a retry from overwriting failed evidence.

## Evidence

`launch` prints the absolute artifact directory, normally under `$TMPDIR` as
`ferry-verification.XXXXXX`. It contains platform, revision, source diff/hashes,
binary hash, and per-action `command.txt`, `stdout.txt`, `stderr.txt`, and
`exit-code.txt`. The PTY transcript records the CLI session. Preserve the directory
and report its absolute path. The binary itself is scratch, not retained evidence.

Exercise the real CLI or launcher, never replace ferry with a mock or use private
setters to manufacture the result. Capture the action and resulting state.
For movement, a successful exit or `focus requested` is insufficient: observe
the window's destination **and** the active Space independently. The existing
live harness supplies those assertions for its three covered paths. A visual
journey also needs before/after screenshots of the fixture and a short recording
of movement/follow timing using available native UI capture; save them in this
artifact directory. Record the selected window ID and source/destination indexes.
Do not infer animation latency from ferry's verbose internal timing.

For manual CLI launches there should be no app data files, background service,
network activity, or git-ref changes; compare the relevant filesystem/process
state if the change could introduce them. No journey here mocks an external
boundary. Shell-running a Raycast script proves the script path only; the real
Raycast action/hotkey must be exercised separately for launcher claims.

## Cleanup

Run after successful **and failed** iterations:

```sh
"$FERRY_VERIFY" cleanup "$FERRY_EVIDENCE"
test ! -e "$FERRY_EVIDENCE/scratch"
test -s "$FERRY_EVIDENCE/build/command.txt"
test -s "$FERRY_EVIDENCE/source.sha256"
test -s "$FERRY_EVIDENCE/cleanup.txt"
```

The live harness normally closes its own windows and restores the previous app
in `@finally`. Let it finish; do not force-kill it mid-switch. After an abnormal
termination, inspect the exact fixture PID/window IDs from that run and close
only those windows/processes; restore the recorded source Space. Never kill by
process name. `cleanup` removes only the run's scratch, not evidence or the shared
`build/` directory. `make test-follow` may leave its reusable `build/follow-test`.
Manual recipes must stop their recorded fixture PID and release their own lock
before scratch removal. Verify the evidence files for the chosen feature still
exist after cleanup; copy artifacts to durable storage if needed before OS temp
cleanup. Do not commit desktop captures or local evidence by default.

## Helpers

[scripts/verify.sh](scripts/verify.sh) is executable and provides `launch`,
`doctor`, `cli`, `desktop`, `capture`, and `cleanup` as invoked above. It requires
the system Bash and existing Makefile/test tools, no package installation.
Feature-specific manual setup is documented in the map, not hidden in the helper.
Keep the map current with `/maintain-verification-skill` as ferry changes.
76 changes: 76 additions & 0 deletions .agents/skills/verify-ferry/features/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,76 @@
# Ferry verification map

This is the maintained map of user-visible verification paths. Read this index
before choosing a journey. One passing journey is not full application coverage.

## Baseline preconditions

- Launch and doctor using [the skill](../SKILL.md); use its run-owned `FERRY_BIN`.
- CLI-only checks need no desktop. Live journeys need a local logged-in desktop,
Accessibility for the launcher, two normal Spaces, and stable Space numbering.
- Do not rearrange Spaces or change system permissions automatically. Report a
missing precondition as failure to execute the journey, with its reason.
- For the manual recipes below, confirm Mission Control shows the current Space
as global index 1 and destination as global index 2; both must be normal Spaces
on the active display. Do not assume these indexes from screen position on a
multiple-display system. The automated regression discovers indexes itself.
- Only use disposable fixture windows. Reserve the desktop lock specified in the
skill. Do not double-drive a shared desktop or shared Raycast settings.

## Driving conventions

Use the executable helper for command/stream/exit evidence and native UI tools
for visible app focus and launcher actions. Manual desktop recipes share this
setup after `doctor`; the first command reserves the lock:

```sh
mkdir "${TMPDIR:-/tmp}/ferry-verification-desktop.lock"
make build/follow-test
build/follow-test --fixture > "$FERRY_EVIDENCE/fixture-ids.txt" \
2> "$FERRY_EVIDENCE/fixture-stderr.txt" &
FERRY_FIXTURE_PID=$!
printf '%s\n' "$FERRY_FIXTURE_PID" > "$FERRY_EVIDENCE/fixture-pid.txt"
```

Wait for `fixture-ids.txt` to contain two nonzero decimal window IDs and for both
windows to appear on source Space 1. Record the original frontmost app and Space
with native UI control before starting. The titles are `Ferry follow test: target`
and `Ferry follow test: other window`. Read IDs only after readiness:

```sh
read -r FERRY_TARGET FERRY_OTHER < "$FERRY_EVIDENCE/fixture-ids.txt"
export FERRY_TARGET FERRY_OTHER
```

Focus windows by their title and owning fixture process, never by coordinates or
guessing a PID as a window ID. Capture the before state, action, resulting window
location and active Space. After each manual journey, close only this fixture:

```sh
kill -TERM "$FERRY_FIXTURE_PID"
wait "$FERRY_FIXTURE_PID" || true
rmdir "${TMPDIR:-/tmp}/ferry-verification-desktop.lock"
```

Restore the original app and Space with native UI control, then run the skill's
cleanup. If fixture startup fails, still stop the PID you started and release
your lock. Do not launch a second fixture to hide a broken first attempt.

## Proof and skip reporting

- Record feature ID and entry point, command, stdout, stderr, and exit code.
- UI journeys require before/after screenshots and movement recording; retain
evidence through cleanup. Native UI absence is an unmet precondition.
- `focus requested` does not assert that the active Space changed. Inspect the
actual active Space and the selected window's location.
- `make test-follow` covers explicit IDs, already-there, and no-follow only.
- Report Raycast shell, Raycast action, hotkey, and focused CLI paths separately.
- A missing prerequisite must not turn into a different, easier passing journey.

## Features

- [CLI discovery and validation](cli.md): help, version, invalid arguments, streams.
- [Move the focused window](focused-window.md): positional CLI and AX selection.
- [Move an explicit window](explicit-window.md): background selection and already-there.
- [Follow and no-follow](follow.md): active Space behavior and verbose output.
- [Raycast commands](raycast.md): script, action, and hotkey entry points.
38 changes: 38 additions & 0 deletions .agents/skills/verify-ferry/features/cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# CLI discovery and validation

Users discover the syntax and version without moving windows, and receive useful
argument errors with a distinct exit status.

## Sub-features

- `cli-help`: `--help` and `-h` print identical usage to stdout, exit 0.
- `cli-version`: `--version` prints `ferry` and the Makefile version, exit 0.
- `cli-invalid`: absent, nonnumeric, zero, overflowing, duplicate, and unknown
arguments are rejected with exit 2; invalid/missing window IDs are rejected.
- `cli-streams`: success text goes to stdout; argument errors go to stderr.

## How to get to it (user POV)

Run `ferry --help`, `ferry -h`, or `ferry --version` from a terminal. Invalid
invocations use the same executable, not a separate validation interface.

## Driving it with verify.sh

Preconditions: launch and doctor passed; no live desktop or Accessibility needed.

- **Drive all mapped CLI cases.** Run `/usr/bin/script -q
"$FERRY_EVIDENCE/cli.pty.txt" "$FERRY_VERIFY" cli "$FERRY_EVIDENCE"`.
Require exit 0 and the final `PASS:` line.
- **Inspect proof.** `help`, `help-short`, `version`, and `invalid-index` action
folders contain separate streams and statuses. `contract/stdout.txt` contains
ten `ok:` messages from the existing CLI tests; the helper additionally checks
identical help aliases and stream routing.
- **Confirm teardown.** Run the skill cleanup. The PTY transcript and all action
folders remain while `scratch/ferry` is removed.

## Gotchas

- Valid destination arguments would enter real window-management code. The safe
cases above exit during parsing, before WindowServer access; this is not a mock.
- `tests/cli.sh` merges streams internally, so its pass alone cannot prove routing.
- A `make test` pass does not prove a move or a Space focus change.
44 changes: 44 additions & 0 deletions .agents/skills/verify-ferry/features/explicit-window.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Move an explicit window

Users select a particular macOS window ID even when its app is in the background.
The selected window moves and follows; another window from the same app stays.

## Sub-features

- `explicit-background`: select a background app's non-key window by ID.
- `explicit-already-there`: a repeated destination succeeds without another move.
- `explicit-stale-id`: a nonexistent window ID fails with runtime exit 1.

## How to get to it (user POV)

Run `ferry --window 1234 2`, replacing the example ID with a current macOS window
ID. IDs identify windows, not processes, and become stale when windows close.

## Driving it with verify.sh

Preconditions: launch/doctor passed; live local desktop, Accessibility, two normal
Spaces on its active display, and no other desktop journey running.

- **Background selection and repeated destination.** Run `"$FERRY_VERIFY"
desktop "$FERRY_EVIDENCE"`. Require `ok: background --window moves and follows
the selected window` and `ok: --window already on the active destination Space`.
The harness creates two windows, backgrounds them, selects the non-key one,
calls the real CLI, and independently checks selected/sibling Space membership.
- **Inspect action evidence.** `desktop/command.txt` identifies the checkout binary
passed to `make`; `desktop/stdout.txt` includes real IDs, indexes, verbose
results, and assertions. Require overall exit 0 and no `SKIP:`/`FAIL:`.
- **Stale ID.** In a separate manual fixture journey from the index, retain
`FERRY_TARGET`, close that fixture through its recorded PID, and immediately run
`"$FERRY_VERIFY" capture "$FERRY_EVIDENCE" stale-window "$FERRY_BIN" --window
"$FERRY_TARGET" 2`. Require exit 1 and `window N not found` in stderr, with no
other window moved. If the OS reused that ID, fail the setup and use a new run.

## Gotchas

- Harness exit 77 or a `SKIP:` message is missing coverage, not success. `make`
can translate it into exit 2. Look at both streams.
- Resolving an ID on an inactive Space may lack AX support: ferry can move
successfully without following. Verify actual active Space independently.
- The regression checks already-there on the active destination. Already-there
on an inactive destination is a distinct precondition and is not covered.
- Existing harness cleanup runs in `@finally`; force termination can bypass it.
48 changes: 48 additions & 0 deletions .agents/skills/verify-ferry/features/focused-window.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,48 @@
# Move the focused window

The positional command moves the focused normal window to an existing Space and
follows it. Accessibility identifies the focused window; the visible window list
provides a fallback if the app's focused Accessibility window cannot be read.

## Sub-features

- `focused-move`: choose the focused window without passing `--window`.
- `focused-selection`: move the chosen window, leaving its sibling in place.
- `focused-fallback`: use the frontmost app's visible normal window without AX.

## How to get to it (user POV)

Run `ferry 2` in a terminal to move that terminal, or bind the same command to a
launcher/hotkey to move another app's focused window. Raycast has its own recipe.

## Driving it with verify.sh and native UI

Preconditions: manual desktop baseline and fixture from the index; both windows
start on global Space 1. Capture source state and start a short screen recording.

- **Choose a window.** With native UI control, activate the fixture and focus
`Ferry follow test: target`. Run commands through the agent's non-GUI shell so
it does not steal focus. If the tool activates another app, stop and fix setup.
- **Move it.** Run `"$FERRY_VERIFY" capture "$FERRY_EVIDENCE" focused-move
"$FERRY_BIN" --verbose 2`. Require exit 0 and a verbose window ID equal to
`FERRY_TARGET`, with `via accessibility` and `focus requested`.
- **Prove the result.** Observe the active destination Space 2 and target window;
visit Space 1 to confirm `Ferry follow test: other window` stayed. Save before,
after, recording, and the observations in the artifact directory.
- **Fallback entry.** Only in a launcher already lacking Accessibility, repeat
a fresh fixture journey. Require `via window list`, the selected visible normal
window's ID, its actual destination, and actual focus outcome. Do not revoke
the user's permissions to manufacture this state.
- **Terminal entry.** If separately claiming interactive terminal coverage, use
a disposable terminal window, type the same positional command there, and
observe that terminal move and follow. The non-GUI fixture route does not prove
terminal launch behavior; close only the disposable window afterward.

## Gotchas

- The existing `test-follow` always uses `--window`; it does not cover this path.
- Missing AX or terminal UI access leaves the corresponding entry unverified.
- The focused window at invocation matters. Running in an existing terminal
moves a user's terminal and violates the disposable-window baseline.
- AX fallback may select a different normal window when an app has several;
record the observed window ID rather than assuming it is the key window.
Loading
Loading