ci(e2e): add Playwright-based E2E workflow for Mage-OS#313
ci(e2e): add Playwright-based E2E workflow for Mage-OS#313digitalrisedorset wants to merge 4 commits intomage-os:mainfrom
Conversation
Introduce a reusable E2E GitHub Actions workflow designed for browser-based testing (Playwright) against Mage-OS. The workflow reuses existing Magento CI setup and delegates execution to a configurable test command, supporting a two-phase E2E model (setup + execution). Includes an internal workflow to validate the E2E pipeline and documentation clarifying the E2E execution model.
|
I started looking at this but I'm not prepared to approve the workflow or PR yet. Would appreciate review by someone more knowledgeable here. Can you explain what you based the workflow files on? It looks like there's a lot in common with https://github.com/mage-os/github-actions/blob/main/.github/workflows/integration.yaml for instance. |
|
@damienwebdev might be able to tell a bit more about how to run the e2e tests exactly. |
|
I've approved the workflow for testing. Since the workflow very closely mirrors the integration test suite, we should extract and update the common parts to reduce duplication and maintenance load. That's a separate matter though and I'll open an issue. |
rhoerr
left a comment
rhoerr
left a comment
There was a problem hiding this comment.
@digitalrisedorset Can we have the e2e suite skip if the directory doesn't exist?
| id: supported-version | ||
| - run: echo ${{ steps.supported-version.outputs.matrix }} | ||
| e2e-workflow: | ||
| if: ${{ hashFiles('_test/demo-package/**') != '' }} |
There was a problem hiding this comment.
this added line should test if the folder exist. I have a possible alternative if needed but this is simple and hopefully doing what we want
There was a problem hiding this comment.
That throws an error on execution:
[Invalid workflow file: .github/workflows/_internal-e2e.yaml#L1](https://github.com/mage-os/github-actions/actions/runs/22717864824/workflow)
(Line: 37, Col: 9): Unrecognized function: 'hashFiles'. Located at position 1 within expression: hashFiles('_test/demo-package/**') != ''
There was a problem hiding this comment.
I found this method mentioned in https://stackoverflow.com/questions/71336204/github-action-check-if-a-file-already-exists
I'd like to suggest the below but I could be more reliable if I understand to test this workflow end to end locally.
- name: Ensure demo package exists
run: |
if [ -z "${{ hashFiles('_test/demo-package/**') }}" ]; then
echo "Missing demo package"
exit 1
fi
Can I suggest to discuss this after the meeting this afternoon?
There was a problem hiding this comment.
@digitalrisedorset what's the purpose of the existence check? Won't this always exist in the internal version of this?
There was a problem hiding this comment.
The initial commit checks showed CI failures on all Magento versions because no Magento version has the Playwright tests bundled. We need that to not happen. https://github.com/mage-os/github-actions/actions/runs/21867114747
There was a problem hiding this comment.
@damienwebdev like @rhoerr explains, this is very much backward compliance. The hashfile is a github method that does exactly what we are after. But the method needs to be positioned at a place where github can handle it. The comment I am making above is a possible solution whereby I have just moved the hasfile statement higher up in the workflow.
Rather than guessing where to add this statement though, I'd like to know how I can trigger the workflow. I think sansec process prevents me to trigger the workflow.
There was a problem hiding this comment.
I made a new commit today. This commit is inspired of a working version of the same check made on a dummy repo: https://github.com/digitalrisedorset/dummy-repo
this dummy repo reproduces the same folder structure as our github-actions repo and only performs the step test if the folder demo-package exists. Can you let me know if this new commit helps preventing the e2e workflow to be triggered if the folder is missing? thanks
digitalrisedorset
left a comment
digitalrisedorset
left a comment
There was a problem hiding this comment.
Hi @rhoerr I made a small change to move this PR through.
… the existence of the folder demo-package
4 participants
Introduce a reusable E2E GitHub Actions workflow designed for browser-based testing (Playwright) against Mage-OS.
The workflow reuses existing Magento CI setup and delegates execution to a configurable test command, supporting a two-phase E2E model (setup + execution).
It adds:
e2e.yamlworkflow that provisions Magento and executes E2E tests via a configurable command_internal-e2e.yamlworkflow that validates the E2E pipeline using PlaywrightPR Checklist
Please check if your PR fulfills the following requirements:
PR Type
What kind of change does this PR introduce?
What is the current behavior?
Fixes: N/A
What is the new behavior?
Mage-OS now provides a reusable GitHub Actions workflow to run browser-based End-to-End (E2E) tests using Playwright against a real Magento storefront.
The new behavior allows:
test_command)This introduces a new E2E capability focused on real user journeys, without altering or replacing existing PHPUnit-based (legacy) E2E workflows.
Does this PR introduce a breaking change?
Other information