Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
22 commits
Select commit Hold shift + click to select a range
a3d8f40
test(browser): add X11 Playwright E2E harness with xdotool input inje…
naoNao89 Sep 11, 2026
c8afb2f
ci: add isolated browser E2E workflow on X11/Xvfb
naoNao89 Sep 11, 2026
e540db1
fix(browser): own the X11 display and make harness waits deterministic
naoNao89 Sep 11, 2026
bc179c3
fix(ci): install x11-utils for xdpyinfo display readiness check
naoNao89 Sep 11, 2026
09614d2
fix(browser): force real focus transition in ensureActive
naoNao89 Sep 11, 2026
c917f29
fix(browser): record fcitx5 PID only after readiness succeeds
naoNao89 Sep 11, 2026
cf936aa
fix(browser): load openbox rc.xml and harden process teardown
naoNao89 Sep 13, 2026
65ed1f2
fix(ci): pin least-privilege token and make Playwright's CI branch real
naoNao89 Sep 13, 2026
7b10208
test(browser): close the stale-focus poll and make the control test d…
naoNao89 Sep 13, 2026
b63f91f
test(browser): keep failure diagnostics when the page is dead; harden…
naoNao89 Sep 13, 2026
93a119c
test(browser): trim fixture plumbing gaps
naoNao89 Sep 13, 2026
64140ed
fix(browser): webServer takes port OR url, not both
naoNao89 Sep 13, 2026
f5282e3
fix(browser): don't let a blank PID-file line abort --stop under set -e
naoNao89 Sep 13, 2026
254fd62
test(browser): clamp watermark against mid-poll log replacement
naoNao89 Sep 14, 2026
73cdb36
fix(browser): fail loudly on missing xdpyinfo; wait for openbox to cl…
naoNao89 Sep 14, 2026
9bf512d
test(browser): make red runs and future flakes attributable
naoNao89 Sep 14, 2026
db9f29e
ci(browser): gate on harness changes only; nightly carries the drift …
naoNao89 Sep 14, 2026
6090c98
test(browser): give the canary a spine; document the harness contract
naoNao89 Sep 14, 2026
ae09790
ci(browser): fix the timing artifact that silently uploaded only Firefox
naoNao89 Sep 14, 2026
d7be50a
test(browser): fail up front when run locally off Linux
naoNao89 Sep 14, 2026
6e711d9
test(browser): widen the canary's wrong-char coverage
naoNao89 Sep 14, 2026
966ae37
docs(browser): record the fidelity caveats the cross-model audit surf…
naoNao89 Sep 14, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,8 @@ updates:
schedule:
interval: "weekly"
target-branch: "dev"
- package-ecosystem: "npm" # Playwright pins its browser builds; keep that pin from aging silently
directory: "/test/browser"
schedule:
interval: "weekly"
target-branch: "dev"
175 changes: 175 additions & 0 deletions .github/workflows/browser-e2e.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
# Browser-level E2E for the X11/Preedit key-to-DOM plumbing (canary), not a
# general regression net: engine logic is guarded by the headless ctest suite.
# Gate rule: PRs run only when the harness itself changes; engine PRs see
# this lane via nightly + dev pushes. Promotion to a blocking gate on src/**
# requires >=20 consecutive clean nightlies with >=5 of them unbroken by any
# src/**/bamboo/** change (so the streak can't accrue against 20 different
# engines); demotion back to workflow_dispatch-only at 3 consecutive reds
# without a linked triage issue OR >1 failure per rolling 10 — 20-clean
# alone would still promote a 10%-flake suite ~12% of the time.
name: Browser E2E

on:
workflow_dispatch:
schedule:
- cron: '23 3 * * *'
push:
paths:
- 'test/browser/**'
- '.github/workflows/browser-e2e.yml'
- 'src/**'
- 'bamboo/**'
- 'data/**'
- 'server/**'
- '**/CMakeLists.txt'
pull_request:
paths:
- 'test/browser/**'
- '.github/workflows/browser-e2e.yml'

# INVARIANT: this job executes PR-authored code (scripts, lockfile, and this
# file itself) and must stay secret-free — no repository/environment secrets,
# ever; that plus `contents: read` and ephemeral hosted runners (the X11 -ac /
# XTEST / --no-sandbox trust model) is what makes fork-PR execution safe.
permissions:
contents: read

concurrency:
group: browser-e2e-${{ github.ref }}
cancel-in-progress: ${{ github.ref != 'refs/heads/main' && github.ref != 'refs/heads/dev' && !startsWith(github.ref, 'refs/tags/') }}

jobs:
browser-e2e:
name: Browser E2E (X11 / Xvfb)
runs-on: ubuntu-24.04
env:
GTK_IM_MODULE: fcitx
QT_IM_MODULE: fcitx
XMODIFIERS: '@im=fcitx'
CI: "true"
PLAYWRIGHT_BROWSERS_PATH: /home/runner/.cache/ms-playwright
steps:
- name: Setup test environment
run: echo "TEST_HOME=$(mktemp -d -t fcitx5-browser-e2e-XXXXXX)" >> "$GITHUB_ENV"

- name: Checkout fcitx5-lotus
uses: actions/checkout@v7
with:
submodules: recursive

- name: Setup Go
uses: actions/setup-go@v7
with:
go-version: '1.18'
cache: false

- name: Setup Node.js
uses: actions/setup-node@v7
with:
node-version: 22
- name: Install system dependencies
run: |
sudo apt update
sudo apt install -y --no-install-recommends \
xvfb \
x11-utils \
openbox \
xdotool \
dbus-x11 \
fcitx5 \
fcitx5-frontend-all \
fcitx5-frontend-gtk3 \
fcitx5-frontend-gtk4 \
libfcitx5core-dev \
libfcitx5config-dev \
libfcitx5utils-dev \
fcitx5-modules-dev \
extra-cmake-modules \
cmake \
ninja-build \
gettext \
libfmt-dev \
librsvg2-bin \
libinput-dev \
libudev-dev

- name: Build and install fcitx5-lotus
run: |
cmake -B build -S . -GNinja -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr
sudo ninja -C build install

- name: Install test dependencies and Playwright browsers
working-directory: test/browser
run: |
npm ci
npx playwright install --with-deps chromium firefox

- name: Typecheck browser test suite
working-directory: test/browser
run: npm run typecheck
- name: Setup Fcitx configuration
run: |
chmod +x test/browser/scripts/setup-fcitx.sh test/browser/scripts/run-xvfb.sh
./test/browser/scripts/setup-fcitx.sh

- name: Start X11 Desktop environment and Fcitx5
run: |
# Record which distro fcitx5 this run validated — green suites
# must be attributable when noble SRUs move the 5.1.x series.
fcitx5 --version
./test/browser/scripts/run-xvfb.sh

- name: Run Chromium E2E tests
working-directory: test/browser
env:
E2E_JSON_OUTPUT: reports/results-chromium.json
run: npm run test:chromium

- name: Run Firefox E2E tests
working-directory: test/browser
env:
E2E_JSON_OUTPUT: reports/results-firefox.json
run: npm run test:firefox

# Timing data is the flake-rate experiment's raw material; the dot
# reporter alone can't tell a 50ms poll from a 1.9s one.
- name: Upload timing results
if: always()
uses: actions/upload-artifact@v7
with:
name: browser-e2e-timing
path: test/browser/reports/*.json
if-no-files-found: error

- name: Collect diagnostics on failure
if: failure()
run: |
fcitx5-diagnose > /tmp/fcitx5-diagnose.log 2>&1 || true
# Runner stall vs engine bug, decided from the artifact: a red run
# next to high cpu pressure is a runner-statistics event, not a
# Lotus regression.
cat /proc/pressure/* > /tmp/pressure.txt 2>/dev/null || true
# Copy logs from isolated TEST_HOME to /tmp for artifact upload
cp "${TEST_HOME}/fcitx5.log" /tmp/fcitx5.log 2>/dev/null || true
cp "${TEST_HOME}/xvfb.log" /tmp/xvfb.log 2>/dev/null || true
cp "${TEST_HOME}/openbox.log" /tmp/openbox.log 2>/dev/null || true

- name: Upload diagnostics on failure
if: failure()
uses: actions/upload-artifact@v7
with:
name: browser-e2e-failure-artifacts
path: |
test/browser/playwright-report/
test/browser/test-results/
/tmp/fcitx5.log
/tmp/fcitx5-diagnose.log
/tmp/xvfb.log
/tmp/openbox.log
/tmp/pressure.txt
if-no-files-found: ignore

- name: Stop X11 and Fcitx5
if: always()
run: |
./test/browser/scripts/run-xvfb.sh --stop || true
6 changes: 6 additions & 0 deletions test/browser/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
node_modules/
playwright-report/
test-results/
reports/
.playwright/
bun.lock
55 changes: 55 additions & 0 deletions test/browser/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,55 @@
# Browser E2E harness

Real `fcitx5` session under Xvfb; keys injected via `xdotool` (XTEST); results
asserted in Chromium/Firefox DOM via Playwright. What this lane guards is the
key-to-DOM plumbing in `Mode=Preedit` (XTEST → fcitx5 → XIM/GTK path → browser
→ committed text). Engine logic (Telex/VNI rules, macro, per-app mode rules,
surrounding text) is guarded by the headless ctest suite in `test/` —
deliberately NOT here, and the workflow header records the same scope.

## Invariants (violating any of these produces green-but-meaningless runs)

- The harness owns its X display: `DISPLAY` is never inherited, the run aborts
if the display is occupied (live server or stale socket), and readiness polls
`xdpyinfo` (hard dependency — no silent socket-only fallback).
- `HOME`/`XDG_*` are exported BEFORE openbox starts, and openbox readiness
waits for `_NET_SUPPORTING_WM_CHECK` — the focus/raise policy in `rc.xml` is
load-bearing for `ensureActive()`; without the ordering, tests pass on
stock-default luck.
- D-Bus is a private bus from `dbus-session.conf` (no
`standard_session_servicedirs`) — fcitx5 name ownership (`-r`) is scoped to
it, so the harness can coexist with a developer's running desktop.
- `workers=1`/`fullyParallel=false`: one display, one focused window. Parallel
keystroke injection into the same X server is undefined behavior, not speed.
- `retries=0`: red runs are data. Do not add retries; open a triage issue.
- `fcitx5` comes from apt and its version is printed per run — a green run
only attests to the (addon SHA × fcitx5 version × runner image) triple it
actually executed.

Known fidelity caveats: `ShareInputState=All` + `resetStateWhenFocusIn=No`
are set so `fcitx5-remote` switching works without focus games — real users
mostly run per-window state, so cross-field isolation bugs are NOT covered
here by design; and the corpus pins one Telex configuration (hats/DD/tones),
not VNI or spellcheck edges. Widen both in the phase-2 issue, not ad hoc.

## CI gate contract

Lives in `.github/workflows/browser-e2e.yml` header; summary: PRs are gated on
harness changes only; nightlies + dev pushes carry the engine-facing signal.
Promotion to a blocking `src/**` gate needs ≥20 consecutive clean nightlies
with ≥5 uncontaminated by `src/**`/`bamboo/**` changes; demotion on 3 reds
without a linked triage issue, or >1 failure per rolling 10.

## Local use (Linux/X11 only)

```bash
scripts/run-browser-e2e.sh # creates TEST_HOME, starts stack, runs both browsers
BROWSER_E2E_DISPLAY=:98 scripts/run-browser-e2e.sh # if :99 is taken
```

Triage on red: the failure artifact carries `input-events.json` (DOM-side),
`fcitx5.log` at `*=4` (inter-event timing), xvfb/openbox logs, and
`/proc/pressure` — high cpu pressure + failed poll ⇒ runner statistics event,
not a Lotus regression. The json reporter timing artifact (`*.json`) is
uploaded on green runs too; compare p95 poll margins against the 2 s
`expect.poll` deadlines before blaming the engine.
Loading
Loading