Skip to content

test(e2e): add editor smoke tests for typing, formulas and slides - #197

Open
juliusknorr wants to merge 2 commits into
mainfrom
test/e2e-editor-smoke-tests
Open

test(e2e): add editor smoke tests for typing, formulas and slides#197
juliusknorr wants to merge 2 commits into
mainfrom
test/e2e-editor-smoke-tests

Conversation

@juliusknorr

Copy link
Copy Markdown
Member

Summary

Adds Playwright smoke tests exercising basic editing in each editor, run against the example app:

  • Document — type text, apply bold/italic, verify the text round-trips via the automation API
  • Spreadsheet — enter numbers and a =A1+A2 formula, verify the formula is accepted in the cell
  • Presentation — build a three-slide deck and verify the slide count

A shared helpers.ts opens a new document from the example page, waits for the editor iframe to finish loading, and exposes editorApi() to call window.Asc.editor inside the iframe.

Notes

  • Element ids are renamed in this build, so tests drive the editors via keyboard shortcuts and the asc_* automation API rather than clicking toolbar buttons.
  • Spreadsheet test asserts the formula is accepted, not that it computes 15: asc_getCellInfo().asc_getText() returns the formula string and GetSelectedText() is empty for cell selections. A TODO documents the clipboard-based computed-value check (needs clipboard permissions in the project config).
  • Presentation slides are added through AddSlide() because Ctrl+M only fires when the thumbnail panel is focused.

Testing

Verified locally against a running example server:

3 passed (11.7s)

AI assistance

Tests authored with AI assistance (Claude Code, claude-opus-4-8); reviewed and verified by the contributor.

Add Playwright smoke tests covering basic editing in each editor:
- document: type text, apply bold/italic, verify text round-trips
- spreadsheet: enter numbers and a formula, verify it is accepted
- presentation: build a three-slide deck

Shared helpers open a new document from the example app and reach the
editor automation API inside the iframe.

Assisted-by: ClaudeCode:claude-opus-4-8
Signed-off-by: Julius Knorr <jus@bitgrid.net>
@juliusknorr
juliusknorr requested review from a team, chrip and moodyjmz and removed request for a team June 16, 2026 20:36
@juliusknorr juliusknorr moved this from 📄 To do to 🏗️ In progress in 📄 Euro-Office team Jun 16, 2026
@juliusknorr juliusknorr self-assigned this Jun 16, 2026
@juliusknorr juliusknorr moved this from 🏗️ In progress to 👀 In review in 📄 Euro-Office team Jun 24, 2026
@juliusknorr
juliusknorr requested review from a team and j-base64 and removed request for a team June 26, 2026 10:00
@Alex-Arsys

Copy link
Copy Markdown
Contributor

LGTM

@moodyjmz
moodyjmz requested a review from a team as a code owner July 15, 2026 22:30

@j-base64 j-base64 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good,

Just one tiny comment inline about bold/italic test.

also, not sure of importance or root cause, noticed that running with --workers 1 all tests pass clean.
with parallel workers, tests intermittently open the wrong editor/extension

Comment on lines +10 to +11
await editorPage.keyboard.press('Control+b');
await editorPage.keyboard.press('Control+i');

@j-base64 j-base64 Jul 16, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

commented these lines and the test still passed.
maybe we can test these two bold/italic actions with

const isBold = await editorApi(editorPage, (api) => api.get_TextProps().get_TextPr().get_Bold());
const isItalic = await editorApi(editorPage, (api) => api.get_TextProps().get_TextPr().get_Italic());
expect(isBold).toBe(true);
expect(isItalic).toBe(true);

Tested it locally, fails commenting line 10-11 and passes with them uncommented.

@moodyjmz

Copy link
Copy Markdown
Member

TL;DR: The spreadsheet cell reads (spreadsheet-formula.spec.ts:32-40) race the selection change and can flake. nameBox.press('Enter') resolves when the key is dispatched, not when the editor finishes moving the selection and asc_getCellInfo() reflects the new cell — and expect(await cellText(...)).toBe(...) is a one-shot with no retry. Wrap the reads in expect.poll (the presentation test already does this for the analogous async-count case):

await select('A1');
await expect.poll(() => cellText(editorPage)).toBe('5');
await select('A2');
await expect.poll(() => cellText(editorPage)).toBe('10');
await select('A3');
await expect.poll(() => cellText(editorPage)).toBe('=A1+A2');

retries: 1 will mostly hide the flake rather than fix it, which is worse — hence the poll.

Detail

The spreadsheet runs calc off the UI thread, so there's a genuine async gap between the name-box navigation and asc_getCellInfo() returning the new cell's info. The tell is that presentation-slides.spec.ts already uses await expect.poll(() => slideCount(...)).toBe(3) for exactly this class of problem (a value that updates asynchronously after a mutation) — the same treatment just wasn't applied to the cell reads.

Reviewed with AI assistance (Claude Code, claude-opus-4-8); grounded against the PR diff and the existing example-page.spec.ts / playwright.config.ts on main.

@chrip chrip left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verification

Claim / Item Reality Status
Document test verifies bold/italic (PR desc: "apply bold/italic") Only assertions are #slot-btn-undo button not disabled and asc_GetSelectedText() contains 'Hello world'. Plain typing enables undo and produces the text, so the Ctrl+b/Ctrl+i keystrokes (lines 10-11) are effectively unverified — matches @j-base64's local finding.
Spreadsheet reads are deterministic select() does nameBox.press('Enter') (resolves on key dispatch), then a one-shot expect(await cellText(...)).toBe(...). asc_getCellInfo() can lag the selection move → flake. Presentation test uses expect.poll for the same async-value class; spreadsheet does not.
Presentation slide count uses proper retry wait Uses await expect.poll(() => slideCount(editorPage)).toBe(3) — correct, deterministic.
Spreadsheet asserts formula accepted, not computed (=15) Correct and honestly documented in the TODO; asc_getText() returns the formula string, computed-value path needs clipboard perms.
No arbitrary sleep/waitForTimeout Confirmed — waits are expect-with-timeout, waitForURL, toBeHidden on #loading-mask, expect.poll. Good.
e2e CI job runs these specs and is green build.yml has an e2e job (needs: [build, manifest], self-hosted) that runs npx playwright test in e2e/ — wiring exists. But on this PR's run the build jobs FAILED, so e2e was SKIPPED. The tests were never exercised in CI.
Build failure is caused by this PR No. Root cause is a 3rd-party core build step: gclient_paths.patch failed … patch does not apply / Common/3dParty/build_3rdparty.py failed! — unrelated to adding .ts test files. ✓ (unrelated)
Commit hygiene (conventional, DCO, AI disclosure) test(e2e): …, Signed-off-by: present (DCO check green), Assisted-by: ClaudeCode:claude-opus-4-8 trailer, AI assistance disclosed in body.
Diff scope Exactly 4 new files under e2e/tests/; package.json/playwright.config.ts/constants.ts/example-page.spec.ts are pre-existing scaffold, not touched here.

Issues & Suggestions

🔴 Blocking

  • Document test does not verify the formatting it advertisese2e/tests/document-editing.spec.ts:10-11. The Ctrl+b/Ctrl+i presses have no corresponding assertion; @j-base64 verified locally that commenting them out still passes. A "bold/italic" smoke test that stays green when formatting is entirely removed gives false confidence. Adopt @j-base64's concrete fix:
    const isBold = await editorApi(editorPage, (api) => api.get_TextProps().get_TextPr().get_Bold());
    const isItalic = await editorApi(editorPage, (api) => api.get_TextProps().get_TextPr().get_Italic());
    expect(isBold).toBe(true);
    expect(isItalic).toBe(true);
    (Or rename the test/PR to "type text" and drop the formatting claim — but verifying is clearly the intent.)

⚠️ Major

  • Spreadsheet cell reads can flakee2e/tests/spreadsheet-formula.spec.ts (the selectexpect(await cellText(...)) block, ~lines 28-37). The name-box Enter resolves before asc_getCellInfo() reflects the new cell. Wrap each read in expect.poll(() => cellText(editorPage)).toBe(...), mirroring the presentation test. Note retries: 1 in playwright.config.ts will mask rather than fix this — @moodyjmz's point is correct.
  • Parallel-worker cross-talk@j-base64 reports tests intermittently open the wrong editor with multiple workers (clean with --workers 1). playwright.config.ts sets fullyParallel: false but does not pin workers: 1, so spec files still run concurrently and appear to collide on the shared example app / new-tab targeting. Worth pinning workers: 1 (the workflow comment already notes the e2e job assumes a single runner) or root-causing the tab mismatch before this lands, otherwise CI will be flaky.
  • No CI validation yet — the e2e job was SKIPPED because build failed on an unrelated 3rd-party patch error. There is currently no green run proving these specs pass in CI (only the author's local "3 passed"). Recommend rebasing onto current main to get a clean build + an actual e2e run before merging.

ℹ️ Minor / 💡 Suggestions

  • Presentation test drives the API, not the UIpresentation-slides.spec.ts adds slides via api.AddSlide() because Ctrl+M needs thumbnail focus (documented). Reasonable for a smoke test, but it exercises the automation API more than user interaction; a follow-up focusing the thumbnail panel and using the shortcut would be closer to a real user path. 💡
  • Selector robustness — tests rely on internal ONLYOFFICE ids (#slot-btn-undo, #ce-cell-name, #editor_sdk, #loading-mask). These can shift on upstream bumps, but this matches the existing example-page.spec.ts convention, so it's consistent, not new debt. ℹ️
  • editorApi serializationhelpers.ts rebuilds the passed fn via new Function inside frame.evaluate. The "must be self-contained (no closures)" caveat is documented; fine for tests. ℹ️
  • SPDX headers — the new .ts files carry no SPDX header, but the pre-existing example-page.spec.ts also has none, so the e2e dir convention is header-less. Non-blocking; flag only if the repo later standardizes. ℹ️

Verdict

Request changes — Solid, well-scoped test scaffolding, but the headline document test provably does not verify bold/italic (already flagged inline by @j-base64), the spreadsheet reads have a genuine race (@moodyjmz), and no CI run has actually executed these specs (build failed on an unrelated 3rd-party patch, skipping the e2e job). Fix the bold/italic assertion, poll the spreadsheet reads, and get a green e2e run — after that this is an easy approve.

Assisted-by: ClaudeCode:claude-opus-4-8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: 🏗️ In progress

Development

Successfully merging this pull request may close these issues.

5 participants