Skip to content

fix(score): no score without a measurement - #72

Merged
maksimtech merged 2 commits into
mainfrom
fix/no-score-without-measure
Oct 28, 2026
Merged

maksimtech merged 2 commits into
mainfrom
fix/no-score-without-measure

Conversation

@maksimtech

Copy link
Copy Markdown
Owner

When a DNS lookup got no answer (timeout, SERVFAIL) a check was marked "not verified" but still counted 0 points, so check printed 0/100 CRITICAL and exited with 2 — what a script reads as a domain measured and found bad. Reported behind a company resolver for asufc.sanita.fvg.it, which scores 19 from outside.

  • checker.analyze_domain: if any of DMARC, SPF, DKIM, BIMI, MTA-STS or TLS-RPT is not verified, total_score is None and the grade N/A. DomainReport.unverified names those checks.
  • check prints Score: N/A — analysis incomplete, not verified: … and exits with 3 (new, in the README's exit-code table); batch lists the domain as N/A after the graded ones.
  • GPG stays the exception, as report and send already treat it: a keyserver that rate-limits withholds at most its 5 points and says so in its own issue.
  • report and send already refused an incomplete analysis; they are unchanged.

TDD: TestNoScoreWithoutMeasure in tests/test_checker.py and tests/test_cli.py (6 tests) on the DNS replay, where _dmarc.example.com TXT really times out; red before the change. Full suite 678 passed, ruff and mypy clean; 6/6 hand mutants killed. The _mock_checks helper now declares error="": a MagicMock's own error is truthy and read as an unanswered lookup.

Follow-up in apkradar after a release: its DPO letter reads this score, and with None and a verified publisher domain it could reach "no issues found in the publisher's technical measures".

🤖 Generated with Claude Code

Comment thread tests/test_cli.py Fixed
@codspeed

codspeed Bot commented Oct 28, 2026 •

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 4 untouched benchmarks


Comparing fix/no-score-without-measure (6955dac) with main (efeafd3)

Open in CodSpeed

Comment thread tests/test_cli.py Fixed
@codecov

codecov Bot commented Oct 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!

maksimtech and others added 2 commits October 28, 2026 18:52
When a DNS lookup got no answer (timeout, SERVFAIL) a check was marked
"not verified" but still counted 0 points, so `check` printed
"0/100 CRITICAL" and exited with 2 — what a script reads as a domain
measured and found bad. Reported behind a company resolver for
asufc.sanita.fvg.it, which scores 19 from outside.

If any of DMARC, SPF, DKIM, BIMI, MTA-STS or TLS-RPT is not verified,
the analysis now has no score: total_score is None and the grade N/A.
`check` prints "Score: N/A — analysis incomplete, not verified: ..."
and exits with 3, a new and documented code; `batch` lists the domain
as N/A after the graded ones. DomainReport.unverified names the checks.
A keyserver that does not answer still leaves a score, as `report` and
`send` already treat it: it withholds at most the 5 GPG points.

The _mock_checks helper now says its results are verified (error=""):
a MagicMock's own `error` is truthy and read as an unanswered lookup.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
CodeQL read startswith("example.com") as a URL check on a substring
(py/incomplete-url-substring-sanitization). The row is now the one whose
first cell equals the domain, which is also what the test means.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@maksimtech
maksimtech force-pushed the fix/no-score-without-measure branch from 628460c to 6955dac Compare October 28, 2026 17:54
@sonarqubecloud

Copy link
Copy Markdown

@maksimtech
maksimtech merged commit 8db9793 into main Oct 28, 2026
24 of 25 checks passed
@maksimtech
maksimtech deleted the fix/no-score-without-measure branch October 29, 2026 08:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants