Siteimprove · MortenFriisSiteImprove · Sep 15, 2026 · Sep 15, 2026
@@ -9,6 +9,7 @@ Every pull request will be checked against WPCS through GitHub Actions.
 ## Development and releases
 
 See [GitHub Actions: tests and publishing](docs/workflows.md) for workflow selection, reports, and release instructions.
+See [Testing the plugin](tests/README.md) for local setup and test coverage.
 
 ## Version History
 ### 2.1.4

@@ -38,7 +38,7 @@ No real Siteimprove credentials are used.
 Open the failed step first, then download **pr-browser-test-report** or
 **pr-wordpress-test-reports** from the run's Artifacts. Reports are retained for
 14 days and contain individual assertions and synthetic failure screenshots.
-A startup failure may have no browser report. See [testing details](../TESTING.md)
+A startup failure may have no browser report. See [testing details](../tests/README.md)
 and the [regression catalog](../tests/integration-regressions/COVERAGE.md).
 
 ## Check the real Siteimprove connection
@@ -59,7 +59,8 @@ highlighting. See [live test coverage and setup](../tests/live/README.md).
 ## Create and deploy a release
 
 1. Merge the reviewed change after all PR checks pass. Update the plugin version
-   and release notes before tagging.
+   in `siteimprove/siteimprove.php` and the changelog in `siteimprove/readme.txt`
+   before tagging.
 2. Use a version tag such as `v2.1.5`. Pushing that tag automatically starts
    production validation and publishing. Do not push a production tag merely
    to try a dry run.
@@ -85,6 +86,15 @@ Production uses `WP_SVN_USERNAME`, `WP_SVN_PASSWORD`, and optional `WP_SVN_URL`
 `TEST_SVN_USERNAME`, `TEST_SVN_PASSWORD`, and `TEST_SVN_URL`, which must differ
 from production. Shared optional variables are `WP_PLUGIN_SLUG` and `WP_ASSETS_DIR`.
 
+### Troubleshooting
+
+- If no run starts, check that the version tag was pushed and contains the release workflow.
+- For SVN authentication failures, check the credentials for the selected mode and
+  their write access to the destination repository.
+- For missing-file errors, confirm the tagged commit contains `siteimprove/siteimprove.php`.
+- For an existing-version error, inspect the GitHub release and SVN tag before retrying;
+  do not delete published version tags to make a retry succeed.
+
 ## Cleanup map
 
 The separate `prepublish-test.yml`, `prepublish-wordpress.yml`, and

@@ -2,19 +2,19 @@
   "name": "siteimprove-wordpress-tests",
   "private": true,
   "scripts": {
-    "test": "playwright test",
+    "test": "playwright test --config=tests/config/playwright.config.js",
     "test:report": "playwright show-report",
     "env:start": "node scripts/wordpress-env.js start --runtime=playground",
     "env:stop": "node scripts/wordpress-env.js stop",
     "env:status": "node scripts/wordpress-env.js status",
-    "test:wordpress": "playwright test --config=playwright.wordpress.config.js",
+    "test:wordpress": "playwright test --config=tests/config/playwright.wordpress.config.js",
     "test:wordpress:report": "playwright show-report playwright-wordpress-report",
     "test:live": "node tests/live/run.js",
     "test:live:contracts": "node --test tests/live/*.test.js",
-    "test:live:fixture": "playwright test --config=playwright.live-fixture.config.js",
+    "test:live:fixture": "playwright test --config=tests/config/playwright.live-fixture.config.js",
     "env:regressions:start": "node scripts/integration-regressions-env.js start",
     "env:regressions:stop": "node scripts/integration-regressions-env.js stop",
-    "test:regressions": "playwright test --config=playwright.integration-regressions.config.js",
+    "test:regressions": "playwright test --config=tests/config/playwright.integration-regressions.config.js",
     "test:regressions:catalog": "node scripts/regression-coverage.js",
     "test:regressions:report": "playwright show-report playwright-integration-regressions-report"
   },

@@ -11,7 +11,6 @@
 	<!-- Test and tooling code. Only /siteimprove is published to WordPress.org,
 	     so WPCS is scoped to the code that actually ships. -->
 	<exclude-pattern>/tests/*</exclude-pattern>
-	<exclude-pattern>/playwright*.config.js</exclude-pattern>
 	<exclude-pattern>/scripts/*</exclude-pattern>
 	<exclude-pattern>/node_modules/*</exclude-pattern>
 

@@ -1,10 +1,13 @@
 # Testing the plugin
 
+Run the commands below from the repository root. Playwright configurations live in
+`tests/config/`; reports and test results remain in their existing root directories.
+
 ## WordPress integration regressions
 
-The [integration regression suite](tests/integration-regressions/README.md) adds real
+The [integration regression suite](integration-regressions/README.md) adds real
 WordPress multisite, capability, URL mapping and settings checks, plus browser
-recheck contracts. Its [coverage catalog](tests/integration-regressions/COVERAGE.md)
+recheck contracts. Its [coverage catalog](integration-regressions/COVERAGE.md)
 describes behavior and distinguishes automated coverage from planned scenarios.
 All fixtures must use synthetic identities, content, tokens and reserved domains.
 
@@ -13,7 +16,7 @@ All fixtures must use synthetic identities, content, tokens and reserved domains
 These tests cover the fixture browser layer for prepublish flow and related plugin UI/command handoff contracts. The deployment checks
 further down this document only test packaging/deployment, not this bug.
 
-For a plain-English checklist, read [Prepublish test scenarios](tests/SCENARIOS.md).
+For a plain-English checklist, read [Prepublish test scenarios](SCENARIOS.md).
 The automated tests use named Given / When / Then steps, shown in the console,
 GitHub Actions logs and the browser suite's HTML report. The checklist clearly
 separates existing automated coverage from planned authenticated checks.
@@ -108,7 +111,7 @@ by its command queue and callback contract; its actual UI is not exercised.
 ### GitHub Actions
 
 Use **Plugin tests** for automatic PR checks or manual plugin/runtime selection.
-See [the workflow guide](docs/workflows.md) for inputs, tested commits, reports,
+See [the workflow guide](../docs/workflows.md) for inputs, tested commits, reports,
 release procedures and credentials. The manual workflows have been consolidated;
 all browser, single-site and multisite suites remain.
 
@@ -223,7 +226,7 @@ the missing-title result. This concerns page-content fidelity, not SDK UI stylin
 ### What remains for an actual Siteimprove end-to-end test
 
 The live workflow and runner are documented in
-[tests/live/README.md](tests/live/README.md). GitHub runs against PR64 verified
+[tests/live/README.md](live/README.md). GitHub runs against PR64 verified
 login, Live page data, draft handoff, and loading-state exit. The missing-title
 result assertion is explicitly skipped. The following account and network
 requirements still apply; local fixture suites use no secrets.
@@ -319,7 +322,7 @@ References: [Playwright CI setup](https://playwright.dev/docs/ci-intro) and
 
 ## Deployment validation
 
-See [Create and deploy a release](docs/workflows.md#create-and-deploy-a-release)
+See [Create and deploy a release](../docs/workflows.md#create-and-deploy-a-release)
 for the supported tag formats, manual modes, destination settings, release checks,
 and dry-run limitations. The separate Marketplace deployment workflow is retired.
 
@@ -343,7 +346,7 @@ unchanged; the check names remain **Browser tests**, **WordPress tests**, and **
 
 **Create and deploy release** calls the shared `release-checks.yml` before any
 publishing. Every check must succeed, including the protected live test and ZIP
-lifecycle check. See [the workflow guide](docs/workflows.md) for the exact sequence.
+lifecycle check. See [the workflow guide](../docs/workflows.md) for the exact sequence.
 
 ### Live result assertion temporarily skipped
 

@@ -1,16 +1,18 @@
+const path = require('node:path');
+const root = path.resolve(__dirname, '../..');
 const { defineConfig } = require('@playwright/test');
 
 module.exports = defineConfig({
   // CI reports must not collect contributor identities or source diffs.
   captureGitInfo: { commit: false, diff: false },
-  testDir: './tests/browser',
+  testDir: path.join(root, './tests/browser'),
   timeout: 45000,
-  outputDir: './test-results/browser',
+  outputDir: path.join(root, './test-results/browser'),
   expect: { timeout: 5000 },
   forbidOnly: Boolean(process.env.CI),
   retries: 0,
   workers: process.env.CI ? 2 : undefined,
-  reporter: [['list', { printSteps: true }], ['html', { open: 'never' }]],
+  reporter: [['list', { printSteps: true }], ['html', { open: 'never', outputFolder: path.join(root, 'playwright-report') }]],
   use: { trace: 'retain-on-failure', screenshot: 'only-on-failure' },
   projects: [
     { name: 'chromium', use: { browserName: 'chromium' } },

@@ -1,19 +1,21 @@
+const path = require('node:path');
+const root = path.resolve(__dirname, '../..');
 const { defineConfig } = require('@playwright/test');
 
 module.exports = defineConfig({
   // CI reports must not collect contributor identities or source diffs.
   captureGitInfo: { commit: false, diff: false },
-  testDir: './tests/integration-regressions',
+  testDir: path.join(root, './tests/integration-regressions'),
   timeout: 60000,
   expect: { timeout: 5000 },
   workers: 1, // Tests reset options in one disposable multisite installation.
   retries: 0,
   forbidOnly: Boolean(process.env.CI),
-  outputDir: './test-results/integration-regressions',
+  outputDir: path.join(root, './test-results/integration-regressions'),
   reporter: [
     ['list'],
-    ['html', { open: 'never', outputFolder: 'playwright-integration-regressions-report' }],
-    ['json', { outputFile: 'test-results/integration-regressions-results.json' }],
+    ['html', { open: 'never', outputFolder: path.join(root, 'playwright-integration-regressions-report') }],
+    ['json', { outputFile: path.join(root, 'test-results/integration-regressions-results.json') }],
   ],
   projects: ['chromium', 'firefox'].map(browserName => ({ name: browserName, use: { browserName } })),
   use: { baseURL: 'http://127.0.0.1', trace: 'off', screenshot: 'only-on-failure', video: 'off' },

@@ -1,7 +1,9 @@
+const path = require('node:path');
+const root = path.resolve(__dirname, '../..');
 const { defineConfig } = require('@playwright/test');
 module.exports = defineConfig({
-  testDir:'./tests/live', testMatch:['fixture.spec.js','observer.spec.js'], timeout:60000, workers:1,
-  outputDir:'./test-results/live-fixture', retries:0, forbidOnly:Boolean(process.env.CI),
+  testDir:path.join(root, './tests/live'), testMatch:['fixture.spec.js','observer.spec.js'], timeout:60000, workers:1,
+  outputDir:path.join(root, './test-results/live-fixture'), retries:0, forbidOnly:Boolean(process.env.CI),
   reporter:[['list',{printSteps:true}]],
   projects:[{name:'chromium',use:{browserName:'chromium'}},{name:'firefox',testIgnore:'observer.spec.js',use:{browserName:'firefox'}}],
   use:{baseURL:'http://localhost:8888',trace:'off',screenshot:'off',video:'off'},

@@ -1,17 +1,19 @@
+const path = require('node:path');
+const root = path.resolve(__dirname, '../..');
 const { defineConfig } = require('@playwright/test');
 
 module.exports = defineConfig({
   // Keep contributor identities and source diffs out of reports.
   captureGitInfo: { commit: false, diff: false },
-  testDir: './tests/wordpress',
+  testDir: path.join(root, './tests/wordpress'),
   timeout: 60000,
-  outputDir: './test-results/wordpress',
+  outputDir: path.join(root, './test-results/wordpress'),
   workers: 1,
   forbidOnly: Boolean(process.env.CI),
   retries: 0,
   reporter: [
     ['list', { printSteps: true }],
-    ['html', { open: 'never', outputFolder: 'playwright-wordpress-report' }],
+    ['html', { open: 'never', outputFolder: path.join(root, 'playwright-wordpress-report') }],
   ],
   projects: [
     { name: 'chromium', use: { browserName: 'chromium' } },

@@ -42,7 +42,7 @@ test('The live runner and workflow do not capture or upload account artifacts',
   assert.doesNotMatch(workflow, /uses:\s*[^\n]*(?:upload-artifact|upload-pages-artifact)/);
   assert.match(workflow, /run: npm run test:live\s/);
   assert.doesNotMatch(workflow, /run:\s*(?:npx playwright test|npm run test:live:fixture)/);
-  const fixtureConfig = require('../../playwright.live-fixture.config');
+  const fixtureConfig = require('../config/playwright.live-fixture.config');
   assert.equal(fixtureConfig.use.screenshot, 'off');
   assert.equal(fixtureConfig.use.trace, 'off');
   assert.equal(fixtureConfig.use.video, 'off');