Skip to content

Write down what the green tests taught this cycle - #95

Merged
0kaba0 merged 1 commit into
developfrom
docs/tests-readme
Sep 6, 2026
Merged

0kaba0 merged 1 commit into
developfrom
docs/tests-readme

Conversation

@0kaba0

@0kaba0 0kaba0 commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

Three rules for hosting-panels/common/tests/, each of which cost a real defect that a passing test did not see. Suggested in review of #93; the next hook will be written by someone who was not here for these.

A word being present is not a criterion

  • test_pdns_restart.sh checked that pgrep -x pdns_server appeared in the installer. It does — in the backend detection, ten lines above the restart. Removing the liveness check left the test green.
  • test_named_conf_edit.sh checked that named-checkconf appeared. It does — inside the message "Check: named-checkconf". Replacing the real guard with if true left the test green.

Both now assert the property: if named-checkconf gating the branch, a rollback restoring the backup, a did not come back message on the path where the service stayed down.

GNU extensions break silently on BSD

sed -i without an argument, \?, i\ with a leading tab — GNU-only. On macOS they do not fail, they produce different output: the old named.conf edit wrote talso-notify and left a quoted "none" in place. A test that runs the code catches this; a test that greps it cannot.

Every enumeration needs a lower bound

TestEveryHandlerRendersHTML looked for sendEmail, found 15 handlers, and the bound of 18 caught that three more send through sendEmailNoArchive and were being skipped entirely. checked >= 3 in two shell tests exists for the same reason.

Fixtures

Also records why fixtures/cpanel-named.conf is kept exactly as it came off the server: three views and a logging block are the shape that broke, and a tidied copy would stop reproducing the failure.

Every claim in the file was checked against the code before committing — the if named-checkconf guard, both checked >= 3 bounds, minHandlers = 18, the three view blocks in the fixture, and the awk insert.

Three rules, each of which cost a defect that a passing test did not see:

- a word being present is not a criterion. pgrep matched the backend
  detection, named-checkconf matched an echo string; both mutants stayed green
  until the checks asserted the property instead of the mention
- GNU extensions break silently on BSD. sed -i, \? and i\ with a tab produce
  different output on macOS without erroring, which is how 'talso-notify' and a
  surviving quoted "none" got written
- every enumeration needs a lower bound. Looking for sendEmail found 15
  handlers and skipped 3 that send through sendEmailNoArchive

Also says why fixtures/cpanel-named.conf is kept verbatim: three views and a
logging block are the shape that breaks, and a tidied copy stops reproducing
it.
@0kaba0

0kaba0 commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator Author

Ready to merge. Three rules, each anchored to a case from this cycle where a green test missed broken code, and the fixture note says why cpanel-named.conf must stay as taken from the server; no upstream names, no attribution, CI green.

@0kaba0
0kaba0 merged commit 8fe9814 into develop Sep 6, 2026
1 check passed
@0kaba0
0kaba0 deleted the docs/tests-readme branch September 6, 2026 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant