test(e2e): add editor smoke tests for typing, formulas and slides - #197
test(e2e): add editor smoke tests for typing, formulas and slides#197juliusknorr wants to merge 2 commits into
Conversation
Add Playwright smoke tests covering basic editing in each editor: - document: type text, apply bold/italic, verify text round-trips - spreadsheet: enter numbers and a formula, verify it is accepted - presentation: build a three-slide deck Shared helpers open a new document from the example app and reach the editor automation API inside the iframe. Assisted-by: ClaudeCode:claude-opus-4-8 Signed-off-by: Julius Knorr <jus@bitgrid.net>
|
LGTM |
| await editorPage.keyboard.press('Control+b'); | ||
| await editorPage.keyboard.press('Control+i'); |
There was a problem hiding this comment.
commented these lines and the test still passed.
maybe we can test these two bold/italic actions with
const isBold = await editorApi(editorPage, (api) => api.get_TextProps().get_TextPr().get_Bold());
const isItalic = await editorApi(editorPage, (api) => api.get_TextProps().get_TextPr().get_Italic());
expect(isBold).toBe(true);
expect(isItalic).toBe(true);
Tested it locally, fails commenting line 10-11 and passes with them uncommented.
|
TL;DR: The spreadsheet cell reads ( await select('A1');
await expect.poll(() => cellText(editorPage)).toBe('5');
await select('A2');
await expect.poll(() => cellText(editorPage)).toBe('10');
await select('A3');
await expect.poll(() => cellText(editorPage)).toBe('=A1+A2');
DetailThe spreadsheet runs calc off the UI thread, so there's a genuine async gap between the name-box navigation and Reviewed with AI assistance (Claude Code, claude-opus-4-8); grounded against the PR diff and the existing |
chrip
left a comment
There was a problem hiding this comment.
Verification
| Claim / Item | Reality | Status |
|---|---|---|
| Document test verifies bold/italic (PR desc: "apply bold/italic") | Only assertions are #slot-btn-undo button not disabled and asc_GetSelectedText() contains 'Hello world'. Plain typing enables undo and produces the text, so the Ctrl+b/Ctrl+i keystrokes (lines 10-11) are effectively unverified — matches @j-base64's local finding. |
❌ |
| Spreadsheet reads are deterministic | select() does nameBox.press('Enter') (resolves on key dispatch), then a one-shot expect(await cellText(...)).toBe(...). asc_getCellInfo() can lag the selection move → flake. Presentation test uses expect.poll for the same async-value class; spreadsheet does not. |
❌ |
| Presentation slide count uses proper retry wait | Uses await expect.poll(() => slideCount(editorPage)).toBe(3) — correct, deterministic. |
✓ |
| Spreadsheet asserts formula accepted, not computed (=15) | Correct and honestly documented in the TODO; asc_getText() returns the formula string, computed-value path needs clipboard perms. |
✓ |
No arbitrary sleep/waitForTimeout |
Confirmed — waits are expect-with-timeout, waitForURL, toBeHidden on #loading-mask, expect.poll. Good. |
✓ |
| e2e CI job runs these specs and is green | build.yml has an e2e job (needs: [build, manifest], self-hosted) that runs npx playwright test in e2e/ — wiring exists. But on this PR's run the build jobs FAILED, so e2e was SKIPPED. The tests were never exercised in CI. |
❌ |
| Build failure is caused by this PR | No. Root cause is a 3rd-party core build step: gclient_paths.patch failed … patch does not apply / Common/3dParty/build_3rdparty.py failed! — unrelated to adding .ts test files. |
✓ (unrelated) |
| Commit hygiene (conventional, DCO, AI disclosure) | test(e2e): …, Signed-off-by: present (DCO check green), Assisted-by: ClaudeCode:claude-opus-4-8 trailer, AI assistance disclosed in body. |
✓ |
| Diff scope | Exactly 4 new files under e2e/tests/; package.json/playwright.config.ts/constants.ts/example-page.spec.ts are pre-existing scaffold, not touched here. |
✓ |
Issues & Suggestions
🔴 Blocking
- Document test does not verify the formatting it advertises —
e2e/tests/document-editing.spec.ts:10-11. TheCtrl+b/Ctrl+ipresses have no corresponding assertion; @j-base64 verified locally that commenting them out still passes. A "bold/italic" smoke test that stays green when formatting is entirely removed gives false confidence. Adopt @j-base64's concrete fix:(Or rename the test/PR to "type text" and drop the formatting claim — but verifying is clearly the intent.)const isBold = await editorApi(editorPage, (api) => api.get_TextProps().get_TextPr().get_Bold()); const isItalic = await editorApi(editorPage, (api) => api.get_TextProps().get_TextPr().get_Italic()); expect(isBold).toBe(true); expect(isItalic).toBe(true);
⚠️ Major
- Spreadsheet cell reads can flake —
e2e/tests/spreadsheet-formula.spec.ts(theselect→expect(await cellText(...))block, ~lines 28-37). The name-boxEnterresolves beforeasc_getCellInfo()reflects the new cell. Wrap each read inexpect.poll(() => cellText(editorPage)).toBe(...), mirroring the presentation test. Noteretries: 1inplaywright.config.tswill mask rather than fix this — @moodyjmz's point is correct. - Parallel-worker cross-talk — @j-base64 reports tests intermittently open the wrong editor with multiple workers (clean with
--workers 1).playwright.config.tssetsfullyParallel: falsebut does not pinworkers: 1, so spec files still run concurrently and appear to collide on the shared example app / new-tab targeting. Worth pinningworkers: 1(the workflow comment already notes the e2e job assumes a single runner) or root-causing the tab mismatch before this lands, otherwise CI will be flaky. - No CI validation yet — the
e2ejob was SKIPPED becausebuildfailed on an unrelated 3rd-party patch error. There is currently no green run proving these specs pass in CI (only the author's local "3 passed"). Recommend rebasing onto currentmainto get a clean build + an actuale2erun before merging.
ℹ️ Minor / 💡 Suggestions
- Presentation test drives the API, not the UI —
presentation-slides.spec.tsadds slides viaapi.AddSlide()becauseCtrl+Mneeds thumbnail focus (documented). Reasonable for a smoke test, but it exercises the automation API more than user interaction; a follow-up focusing the thumbnail panel and using the shortcut would be closer to a real user path. 💡 - Selector robustness — tests rely on internal ONLYOFFICE ids (
#slot-btn-undo,#ce-cell-name,#editor_sdk,#loading-mask). These can shift on upstream bumps, but this matches the existingexample-page.spec.tsconvention, so it's consistent, not new debt. ℹ️ editorApiserialization —helpers.tsrebuilds the passed fn vianew Functioninsideframe.evaluate. The "must be self-contained (no closures)" caveat is documented; fine for tests. ℹ️- SPDX headers — the new
.tsfiles carry no SPDX header, but the pre-existingexample-page.spec.tsalso has none, so the e2e dir convention is header-less. Non-blocking; flag only if the repo later standardizes. ℹ️
Verdict
Request changes — Solid, well-scoped test scaffolding, but the headline document test provably does not verify bold/italic (already flagged inline by @j-base64), the spreadsheet reads have a genuine race (@moodyjmz), and no CI run has actually executed these specs (build failed on an unrelated 3rd-party patch, skipping the e2e job). Fix the bold/italic assertion, poll the spreadsheet reads, and get a green e2e run — after that this is an easy approve.
Assisted-by: ClaudeCode:claude-opus-4-8
Summary
Adds Playwright smoke tests exercising basic editing in each editor, run against the example app:
=A1+A2formula, verify the formula is accepted in the cellA shared
helpers.tsopens a new document from the example page, waits for the editor iframe to finish loading, and exposeseditorApi()to callwindow.Asc.editorinside the iframe.Notes
asc_*automation API rather than clicking toolbar buttons.15:asc_getCellInfo().asc_getText()returns the formula string andGetSelectedText()is empty for cell selections. A TODO documents the clipboard-based computed-value check (needs clipboard permissions in the project config).AddSlide()becauseCtrl+Monly fires when the thumbnail panel is focused.Testing
Verified locally against a running example server:
AI assistance
Tests authored with AI assistance (Claude Code, claude-opus-4-8); reviewed and verified by the contributor.