Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
89 changes: 89 additions & 0 deletions .github/workflows/quality.yml
Original file line number Diff line number Diff line change
Expand Up @@ -610,6 +610,71 @@ jobs:
path: .ci-artifacts/frontend-next-unit-junit.xml
if-no-files-found: warn
retention-days: 7
# #3318: this tier collected no coverage at all, so there was no
# baseline and nothing here could catch a change that deleted tested
# behaviour. The tests that remained would keep passing over code that
# had stopped being tested, and the job would stay green -- the tests
# that assert the deleted behaviour would be the thing a reviewer has
# to notice is missing.
#
# Two gates, both compared against the committed
# arcane/home/honeypot-dashboard/frontend-next/coverage-baseline.json
# and both computed from this run's own measurement. Neither is a
# threshold somebody chose:
#
# coverage:ratchet the non-regression floor. The baseline is what the
# suite measures today -- 806/7359 lines (10.95%)
# and 346/7250 branches (4.77%) across 127 files of
# src/ -- measured on this job's own #3331 node:22
# image. It is a floor, not a target, and the number
# in the file is a measurement rather than an
# aspiration on purpose: an importable 60% would
# have made this gate red on the day it landed and
# bought nothing. Raising it is a separate,
# deliberate commit.
# test:discovery a test-shaped file that no configured runner
# collects. Its own step so a coverage failure
# cannot hide a discovery failure.
#
# `npm test` above is untouched and stays uninstrumented: a second full
# run of the suite is the honest price of keeping the command deploy.yml,
# the README and a developer's own loop free of coverage, and it is
# cheap here -- 6.9s uninstrumented against 6.9s with the v8 provider on
# the image this job runs (the cost is transform/import, not
# instrumentation), against a 45-minute budget.
#
# `set -euo pipefail` so a failing coverage run cannot be followed by a
# ratchet that then reports on a missing report and takes the blame for
# it.
- name: "Coverage report and ratchet (#3318)"
working-directory: arcane/home/honeypot-dashboard/frontend-next
run: |
set -euo pipefail
npm run test:coverage
npm run coverage:ratchet
- name: "Test discovery guard (#3318)"
working-directory: arcane/home/honeypot-dashboard/frontend-next
run: npm run test:discovery
# Same per-(run, attempt) name as the JUnit upload above, for the same
# reason: run_id alone still collides on a re-run of the same run, and a
# name two uploads share is not "newest wins" under v4 -- it is the
# second uploader deleting the first one's evidence. No overwrite here
# either, and the retention matches.
#
# `coverage/` is named as a directory rather than file-by-file because
# it is not hidden -- the .ci-artifacts caveat that forces the JUnit
# upload to name its file does not apply, and a directory keeps the
# report extensible (adding the lcov html view later is a config change
# here, not a workflow change). The contents stay a closed set: only the
# files the coverage reporter wrote, from the run above, can land here.
- name: Upload unit test coverage
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: frontend-next-coverage-${{ github.run_id }}-${{ github.run_attempt }}
path: arcane/home/honeypot-dashboard/frontend-next/coverage/
if-no-files-found: warn
retention-days: 7
# No live-ES smoke suite here on purpose: port-tests/ needs the real
# cluster over an SSH tunnel to the homeserver (see its README) --
# not reachable from a GitHub-hosted runner, and not appropriate to
Expand Down Expand Up @@ -699,6 +764,30 @@ jobs:
path: .ci-artifacts/frontend-next-unit-junit.xml
if-no-files-found: warn
retention-days: 7
# #3318: twin of the homeserver copy's coverage + ratchet + discovery
# steps. Present here for the reason the header comment gives: steps stay
# byte-identical across the twins so a red result means the same thing
# wherever it ran. A ratchet that only ran on the self-hosted executor
# would let a regression through on every degraded day, which is exactly
# the day the fallback twin exists for. (Full rationale, and the measured
# baseline, live on the homeserver copy above.)
- name: "Coverage report and ratchet (#3318)"
working-directory: arcane/home/honeypot-dashboard/frontend-next
run: |
set -euo pipefail
npm run test:coverage
npm run coverage:ratchet
- name: "Test discovery guard (#3318)"
working-directory: arcane/home/honeypot-dashboard/frontend-next
run: npm run test:discovery
- name: Upload unit test coverage
if: always()
uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4.6.2
with:
name: frontend-next-coverage-${{ github.run_id }}-${{ github.run_attempt }}
path: arcane/home/honeypot-dashboard/frontend-next/coverage/
if-no-files-found: warn
retention-days: 7
# No live-ES smoke suite here on purpose -- see the homeserver twin.
- run: npm run build
working-directory: arcane/home/honeypot-dashboard/frontend-next
Expand Down
7 changes: 7 additions & 0 deletions arcane/home/honeypot-dashboard/frontend-next/.gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,10 @@ playwright-report/
# there, never committed -- a committed mutation report is a stale report.
reports/mutation/
.stryker-tmp/

# #3318: the v8 coverage report (coverage-summary.json, lcov.info and the html
# report). Same reasoning as the two blocks above: regenerated on every
# `npm run test:coverage`, uploaded to CI from there, and never committed --
# coverage-baseline.json at the package root is the committed record, and a
# committed report would be a second, stale one.
coverage/
40 changes: 40 additions & 0 deletions arcane/home/honeypot-dashboard/frontend-next/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,46 @@ for how a commit actually reaches the live host.
`backend-api.sh`, `bff-load.sh`) — see `../port-tests/README.md`. Run
against a real build, not `npm run dev`'s HMR server.

### Unit tests, coverage and the ratchet (#3318)

`npm test` is the unit suite and stays uninstrumented — no coverage, no
report tree — so it stays the cheap command that `deploy.yml`, this README
and your own loop all reach for. The coverage number lives in its own
command:

```bash
npm run test:coverage # vitest + v8 coverage for src/, into coverage/ (gitignored)
npm run coverage:ratchet # compare that measurement to the committed baseline; non-zero on a drop
npm run test:discovery # fail if a *.test.ts / *.spec.ts exists that no runner collects
```

`coverage-baseline.json` at this directory's root is the committed
non-regression floor: **806/7359 lines (10.95%) and 346/7250 branches
(4.77%)** across 127 files of `src/`, measured on the `node:22` image the
tier ships. It is a measurement, not a target, and the low number is the
real one — the tier's tests concentrate on `src/lib` server logic, and most
of `src/routes` and `src/components` is covered by the Playwright matrix
instead, which the v8 provider does not see. Do not hand-edit it, and do
not regenerate it to turn a red run green.

To move it on purpose, having first established the drop is intended
rather than a lost test:

```bash
npm run test:coverage && npm run coverage:baseline # commit both, together
```

The ratchet has two independent gates. `tolerance.coveredCount` is 0: no
covered line or branch may stop being covered, which is what catches a
change that deletes tested behaviour, and which no shrinking-denominator
trick can satisfy. `tolerance.pctPoints` is 1.0: the overall percentage may
not fall more than that, which catches a change that adds a lot of untested
source. At a ~11% baseline the second is the coarse one — it needs several
hundred new uncovered lines to move — which is the honest shape of a
low-coverage ratchet and the reason the first one carries the weight.
`scripts/tests/test_3318_coverage_ratchet.py` in the repository root pins
both, so neither can be loosened without that test going red.

## New-page review checklist (capped-truth discipline, #2179)

When authoring or reviewing a page, check the ways a number can quietly lie.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"about": "Generated measurement, not a target. Do not hand-edit these numbers and do not regenerate them to make a red run green -- coverage-baseline.json exists so that coverage cannot fall without a commit that says it did. Update only via `npm run coverage:baseline`, in a commit whose diff shows the change.",
"recordedAt": "2026-09-27T16:54:00.850Z",
"measuredWith": {
"node": "v22.23.2",
"vitest": "4.1.11",
"coverageProvider": "@vitest/coverage-v8 4.1.11",
"note": "measured on the runtime the image ships (node:22-alpine), which is the runtime CI runs"
},
"scope": "src/**/*.{ts,tsx} minus *.test.*, *.spec.*, *.d.ts and src/routeTree.gen.ts (vitest.config.ts coverage.include/exclude)",
"tolerance": {
"pctPoints": 1,
"coveredCount": 0
},
"toleranceWhy": {
"pctPoints": "percentage points of lines/branches. This is the half that fires when a change ADDS untested source. 1.0pp of a 7359-line denominator is ~74 lines, about 1.3 of src/'s average 58-line file.",
"coveredCount": "raw covered lines/branches allowed to disappear; 0 is deliberate. This is the half that fires when a change REMOVES tested behaviour, and comparing raw counts cannot be satisfied by deleting an untested file to shrink the denominator."
},
"files": 127,
"totals": {
"lines": {
"covered": 806,
"total": 7359,
"pct": 10.9526
},
"branches": {
"covered": 346,
"total": 7250,
"pct": 4.7724
},
"statements": {
"covered": 859,
"total": 8452,
"pct": 10.1633
},
"functions": {
"covered": 107,
"total": 2666,
"pct": 4.0135
}
}
}
Loading
Loading