Skip to content

feat(argo): add ChairLift Wayland behave lane - #1165

Merged
castrojo merged 19 commits into
mainfrom
feat/chairlift-wayland-lane
Oct 8, 2026
Merged

castrojo merged 19 commits into
mainfrom
feat/chairlift-wayland-lane

Conversation

@castrojo

@castrojo castrojo commented Oct 5, 2026 •

Copy link
Copy Markdown
Collaborator

Adds run-chairlift-wayland-tests: ChairLift's behave + dogtail AT-SPI suite (test/e2e/features) in a live GNOME Wayland session, plus an Enhanced Troubleshooting Goose-profile isolation check. No Xvfb.

Shape

  • golang:1.26 init container: clones chairlift-repo@chairlift-branch, make build-e2e schemas, and builds two lane tools inside the module — navjson (the page/shortcut inventory internal/navigation gives the Go gate) and gooseprofile (troubleshoot.ProfileAt(dir).Write, plus a config reader).
  • Runner: run-container-tests' nested systemd/GDM Dakota target — headless GNOME Shell drop-in (--unsafe-mode --headless --virtual-monitor), linger, lastchg, resolver bind, uinput — with /usr/share/chairlift masked and /proc/cmdline blanked like ChairLift's dakota_atspi.sh. behave runs from a hash-pinned venv (requirements-atspi.txt) under qecore-headless --session-type wayland with GDK_BACKEND=wayland, DISPLAY unset, CHAIRLIFT_REQUIRE_ATSPI=1. Fails closed on 0 executed scenarios.
  • Goose check (x86_64): ChairLift installs the packages itself — a lane tool built against the branch runs troubleshoot.Setup(troubleshoot.Detect(), …) (tap, linux-mcp-server, cpio, --cask goose-linux via internal/homebrew) with PATH=/usr/bin:/bin and no brew shellenv, as a direct chairlift launch would — then asserts exit 0, that goose-desktop resolves, and that troubleshoot.Command's env PATH starts with Homebrew's bin. It then writes the profile, runs goose-desktop --ozone-platform=wayland for ~40s with GOOSE_PATH_ROOT/XDG_CONFIG_HOME in the profile, then asserts the SingletonLock is in the profile and not ~/.config/Goose, only linux-tools + bluefin-knowledge are enabled, data/state land under GOOSE_PATH_ROOT, and nothing appears in ~/.config/goose.
  • Holds ghost-container-qa; EXIT/TERM/INT traps remove the target; results persisted to ghost's result store.

Wayland traps found on ghost (documented in patterns.md)

  • qecore's script env has no WAYLAND_DISPLAY → read from systemctl --user show-environment.
  • Mutter focus-stealing prevention leaves new windows unfocused → ChairLift's environment.py activates via Shell.Eval.
  • First ponytail key dropped → one Shift warm-up per run.
  • goose-linux cask preflight's bare rpm2cpio | cpio exits 127 without Homebrew's bin on PATH.

Evidence

  • chairlift-wayland-lane-hnq84 (ChairLift aed3a24, dakota:testing sha256:136c79e0…): Succeeded — 181/181 executed scenarios passed (1 @known_issue.347 skipped); Goose check passed all 11 assertions with the install driven by ChairLift's Setup under PATH=/usr/bin:/bin.
  • chairlift-wayland-lane-c9b7z (ChairLift ad6ebb1): Succeeded, 181/181, with the earlier shellenv-based install.

Checks

argo lint --offline argo/workflow-templates/ and argo/bootstrap/; pytest tests/unit (66 passed in a lab pod, incl. new test_run_chairlift_wayland_tests.py); scripts/check_dakota_variants.py.

Update: end-to-end Enhanced Troubleshooting check

The Goose check now drives ChairLift's own code throughout. In order: troubleshoot.Setup; aistack.Enable on the stock host; the alias path (aistack.PullModel + ConfigureActiveModel); the real troubleshoot.Command launch (llmman launch goose-desktop); then llmman launch goose -- run in the same profile. Tool calls are read back from Goose's session database. ChairLift steps run as systemd-run --user --slice=app.slice units, so polkit sees an active session (pkcheck: auth_admin from podman exec, yes from an app unit).

Latest full run, chairlift-wayland-lane-t54w6 (ChairLift 836f58e): behave 181/181 passed. The Goose check has two failing groups of checks:

  • model_pulled_and_alias_set fails because of a ChairLift defect: aistack.VerifyModelStored misses llmman's hf.co/-qualified stored name. It is fixed upstream on a branch that has not been pushed yet.
  • The environ_* checks fail because the Electron browser process's environment had none of the provider names. The next run reads Goose's goose serve backend instead.

Everything else passes:

  • Agent Mode enables on the stock host. Only the llmman formula is trusted, not the tap.
  • The Goose window comes up on Wayland under the profile lock.
  • The CLI calls both linux-tools__get_service_status and bluefin-knowledge__search_knowledge.

Final run: chairlift-wayland-lane-xjws5, Succeeded

Against ChairLift 0e19f36 (the ported --ask-bluefin branch):

  • make ci: exit 0 ("CI mirror passed").
  • behave in the GNOME Wayland session: 174/174 executed scenarios passed, 1 skipped (@known_issue.347).
  • Engine, every check passed, all through ChairLift's own code:
    • troubleshoot.Setup with no Homebrew on PATH, then aistack.Enable on the stock host. Only the llmman formula is trusted, not the tap.
    • aistack.PullModel succeeded on its own for Qwen3-0.6B (397 MB, 6 s); the alias reads back as hf.co/unsloth/Qwen3-0.6B-GGUF:Q4_K_M.
    • A cold chairlift --ask-bluefin and the distro menu's exact bash -c '…/chairlift-wrapper --ask-bluefin' both exited 0. Both left Goose running under llmman launch goose-desktop with the same backend environment:
      • GOOSE_PATH_ROOT and XDG_CONFIG_HOME in the profile;
      • GOOSE_PROVIDER=openai;
      • OPENAI_HOST=http://127.0.0.1:17434;
      • GOOSE_MODEL=hf.co/….
        Both times the lock was in the profile and nothing was in ~/.config/goose.
    • CLI tool calls: linux-tools__* and bluefin-knowledge__search_knowledge.
  • Screenshots: all 8 pages from ChairLift's own capture.

Answer quality (recorded, not gated): none of 0.6B, 1.7B, or 4B named all six units systemctl --failed reports. 4B got 3 of 6 right and added 3 that aren't failed units.

pi and others added 16 commits October 5, 2026 12:48
run-chairlift-wayland-tests builds a ChairLift branch, boots
run-container-tests' nested systemd/GDM Dakota target with a headless GNOME
Shell, and runs ChairLift's behave AT-SPI suite inside the live Wayland
session through qecore-headless. On x86_64 it then installs Enhanced
Troubleshooting's packages through the target's Homebrew and checks that
Goose desktop launched in ChairLift's profile writes nothing outside it.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Read WAYLAND_DISPLAY from the user manager: qecore hands the script
  gnome-session's environment, which predates the compositor.
- Install the Goose packages under brew shellenv, as chairlift-wrapper
  does; the goose-linux cask's bare rpm2cpio | cpio preflight exits 127
  without Homebrew's bin on PATH.
- Mask Dakota's brew-preinstall unit and brew timers so they cannot hold
  Homebrew's locks; python-uinput is best-effort (input goes through
  gnome-ponytail-daemon).
- Document the lane and its Wayland traps.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The isolation check proved the goose-linux cask only under brew shellenv,
not ChairLift's install path. It now runs troubleshoot.Setup(Detect(), …)
from a lane tool built against the branch, with PATH=/usr/bin:/bin and no
shellenv, the way a direct chairlift launch installs; asserts it exits 0
and goose-desktop resolves; and asserts troubleshoot.Command's PATH starts
with Homebrew's bin. The profile isolation assertions are unchanged.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Goose check no longer satisfies troubleshoot.Command with a stand-in
llmman or launches Goose directly. Through ChairLift's own code it now
enables Agent Mode (aistack.Enable, WaitHealthy), resolves and pulls the
smallest Qwen3 through the alias path (PullModel, ConfigureActiveModel),
launches the real troubleshoot.Command in the Wayland session and reads
the Goose process's environment, then asks goose (llmman launch goose, same
profile) a question that needs linux-tools and search_knowledge and
extracts the tool calls from the session database. A model that will not
call tools is retried once a size up and otherwise fails the check.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Run chairlift-wayland-lane-2nlfh failed every keyboard scenario: the
first-login "Welcome to Bluefin / Take Tour" dialog, a Shell modal, took
the keys while the ChairLift window held focus (Main.modalCount was 1).
Mark it shown in the test user's dconf before any session starts, and
refuse to start the suite while any Shell modal is up.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The session opens in the overview, which counts as a Shell modal, so the
new gate refused every run (chairlift-wayland-lane-25vxb: modalCount 1,
screenshot shows the overview). Hide it first.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
On stock Dakota aistack.Enable fails: Homebrew refuses llmmanorg/tap
(chairlift-wayland-lane-jhcs8), and ChairLift offers no way to trust a tap
nothing is installed from yet. Record that as a failing check
(agent_mode_enabled_on_stock_host), then apply the remedy brew's message
gives the user and enable Agent Mode again through ChairLift so the launch
and tool-call checks still produce evidence.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…strator

ChairLift now trusts only the llmman formula when Agent Mode installs it
(chairlift 836f58e), so the lane's manual brew trust is gone: Agent Mode
must enable on the stock host, and the check records that the formula is
trusted while llmmanorg/tap is not.

The test user joins wheel, as Bluefin's first user does. Without it,
Agent Mode's Jan Flatpak failed with "Flatpak system operation Deploy not
allowed for user" (chairlift-wayland-lane-q5xpf).

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The runner script sits in podman's exec cgroup, outside any logind
session, so polkit treated ChairLift's Agent Mode install of Jan (a system
Flatpak) as coming from an inactive subject and refused it even for a
wheel member (chairlift-wayland-lane-w7wtx, -nh4wz). Run every ChairLift
step the way GNOME runs an app: a transient unit in the user manager's
app.slice. Record logind and pkcheck evidence for both contexts.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Running Setup as an app unit hit the user slice's pids limit: brew's
download threads failed with "can't create Thread: Resource temporarily
unavailable" (chairlift-wayland-lane-dsjsw). Lift TasksMax on the user
slice, app.slice, and each unit, and record the limits.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…heck

aistack.PullModel reported unsloth/Qwen3-0.6B-GGUF:Q4_K_M "not found in
daemon stored models" after the pull (chairlift-wayland-lane-rx4k2).
Record /llmman/node, llmman ls and the installed Flatpaks; when only that
check disagrees, set the alias through ChairLift's ConfigureActiveModel so
the launch and tool checks still run, while model_pulled_and_alias_set
stays false. Fix the pkcheck pid inside systemd-run, which expands
${...} itself.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Its /proc environ read back empty as the test user
(chairlift-wayland-lane-jhxkh), so every environ check failed on no data.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The Electron browser process's environ came back without any of the
provider names (chairlift-wayland-lane-t54w6). Record its raw contents and
assert on the environment Goose hands its own backend. Document the
end-to-end Goose check and the session/polkit and TasksMax traps.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
- Build: run ChairLift's `make ci` (golangci-lint v2.12.2 by checksum) on
  the untouched checkout and record it; stage chairlift and
  chairlift-wrapper as the Homebrew cask lays them out.
- Behave: export the session's Wayland socket as an absolute path (the
  suite's MutterInputBackend and per-scenario runtime dirs need it); stop
  starting gnome-ponytail-daemon.
- Engine: aistack.PullModel must succeed on its own (no alias fallback);
  launch Goose through a cold `chairlift --ask-bluefin` and through the
  distro menu's exact command (chairlift-wrapper via bash -c) with GNOME
  Shell's environment and a PATH without Homebrew, asserting the Goose
  backend's environment, lock, and the same outcome for both; check the CLI
  answer against systemctl --failed for 0.6B, 1.7B, then 4B.
- Screenshots: ChairLift's own capture_walkthrough.sh under its
  wayland_session.sh (headless Mutter, 900x700), byproducts stripped.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
ReadActiveModel answers with the name llmman stored the model under,
hf.co/<ref> (chairlift-wayland-lane-r5vxf), after a PullModel that
succeeded on its own.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
make ci gate, ask-bluefin and menu-command launches, PullModel without a
fallback, answer-quality recording, and ChairLift's own Wayland capture.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
pi and others added 3 commits October 5, 2026 19:16
…al answer-quality runs

With a Goose session from a cold --ask-bluefin still running (its window
minimized), run the distro menu command again and assert exit 0, the same
single Goose main process, and a Goose window afterwards; record whether
it came back unminimized and focused. answer-quality=false keeps only the
smallest model's tool-call check.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…s raised

Shell.Eval answers a JSON.stringify result as a JS string inside a GVariant
string, so the relaunch window state never parsed and
relaunch_goose_window_exists read false although one Goose window existed
(chairlift-wayland-lane-n7fsg). Also assert the existing window comes back
unminimized and focused, which that run shows it did not.

Assisted-by: Claude Opus 5.5 via GitHub Copilot
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@castrojo
castrojo merged commit cef6861 into main Oct 8, 2026
3 checks passed
@castrojo
castrojo deleted the feat/chairlift-wayland-lane branch October 8, 2026 02:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant