Skip to content

Add documentation checks and skip Gradle jobs on docs-only pull requests - #99

Merged
Paladin173 merged 1 commit into
mainfrom
ci-docs-checks
Aug 27, 2026
Merged

Paladin173 merged 1 commit into
mainfrom
ci-docs-checks

Conversation

@Paladin173

Copy link
Copy Markdown
Collaborator

What changed

Documentation checks

This repository is documentation-heavy — README.md, ROADMAP.md,
ARCHITECTURE.md, AGENTS.md, RELEASE.md — and AGENTS.md makes doc updates
part of "done". Nothing verified them, so a broken link or a link split across a
newline could sit in README.md indefinitely.

Two new jobs, both scoped to runs where Markdown actually changed:

  • docs-render (blocking) runs tools/check_markdown.py. It is deliberately
    not a style linter — no line length, heading level, or list marker rules.
    It reports only rendering breaks (an unclosed inline link, or a destination
    split across a newline, both of which render as literal [text]( garbage) and
    relative links or images pointing at a file that does not exist.
  • docs-links (informational) checks external URLs with lychee and is
    continue-on-error, because the network is not a build dependency.

The repository's current Markdown passes cleanly, and the checker ships with 13
unit tests covering both the positive and negative cases.

Documentation-only fast path

A README-only pull request previously ran the full Gradle unit test, Android
Lint, and debug APK build. Those three jobs now skip when every changed path
ends in .md.

The detector is fail-safe by construction: it always exits 0, always emits
docs_only, and defaults to false. A missing diff base, an unreadable diff, or
a single non-doc path all run the full suite. git diff --no-renames is used so
git mv MainActivity.kt notes.md still reveals the source deletion rather than
collapsing to a docs destination. The three jobs also carry always(), so a
detector failure degrades to running them rather than skipping them.

release-tooling-tests is deliberately excluded from the fast path: it
asserts that the README release metadata matches Gradle, which is precisely what
a docs-only change can break.

There are no required status checks on this repository today, so skipping a job
is safe. AGENTS.md records that if that ever changes, these jobs must be
converted to short-circuited steps first, because a skipped required check
reports as pending forever.

Testing

  • python -m unittest discover -s tools/tests -p 'test_*.py' — 46 passed,
    including 13 new Markdown checker tests and 2 new workflow contract tests that
    assert the docs jobs exist, that the external link check cannot block a merge,
    that the three Gradle jobs carry the fail-safe condition, and that
    release-tooling-tests stays outside the fast path.
  • python tools/check_markdown.py — 8 files checked, 0 problems.
  • Confirmed the workflow parses and that every job still declares a timeout.
  • No Android source or Gradle inputs changed, so Gradle verification was not
    re-run.

@Paladin173 Paladin173 added the documentation Improvements or additions to documentation label Aug 27, 2026
@Paladin173
Paladin173 merged commit 2b3c9c1 into main Aug 27, 2026
11 checks passed
@Paladin173
Paladin173 deleted the ci-docs-checks branch August 27, 2026 03:52
@nesquena-hermes

Copy link
Copy Markdown
Contributor

Exact-head review at eb1eaaa72 — fast path is fail-safe, but two proof gaps remain

The workflow’s current classification logic is conservative:

  • it diffs the full PR base…head range (or push before…sha), with full checkout history;
  • docs_only defaults false;
  • missing/empty/unreadable ranges and any non-.md path run the full suite;
  • --no-renames exposes source deletion + Markdown addition separately;
  • unit/lint/debug use always() and treat a missing detector output as non-docs;
  • release-tooling remains outside the skip path;
  • live GraphQL shows this repository currently has no branch protection/rulesets, so skipped heavy jobs do not wedge a required context.

Local trusted-source results: python3 -m unittest discover -s tools/tests 46/46 passed; python3 tools/check_markdown.py checked the eight current Markdown files with zero findings. Exact CI has docs, release tooling, JS, unit, lint, and debug green; API 35/36 instrumentation is still running.

Two issues remain before this should gate documentation:

1. The “in-repo” link check accepts paths outside the repository

tools/check_markdown.py:80-83 resolves a relative destination and checks only target.exists(). It never requires the resolved target to remain under root. I reproduced:

  • repository doc.md links to ../outside.md;
  • that file exists beside the repository;
  • checker exits 0 with 0 problem(s).

That is a false green based on runner filesystem layout, not GitHub repository authority. It also admits symlinks whose resolved target leaves the repo. After resolve(), require target.relative_to(root) to succeed (or equivalent), otherwise emit an explicit “target escapes repository” finding before exists(). Tests: existing outside sibling must fail; symlink outside must fail; a nested document’s ../guide.md that resolves inside root must pass.

2. The skip classifier has source-string assertions, not behavioral tests

test_docs_only_change_sets_skip_the_gradle_gates_but_fail_safe only searches YAML for docs_only="false", trap emit EXIT, and job condition strings. It never executes the classifier. A regression in range selection, rename handling, case logic, or output settlement can retain all those strings and skip gates incorrectly.

Move the classification into a small checked-in script/helper invoked by the workflow and test the actual outputs for: Markdown-only, mixed source+docs, source deletion + .md addition, uppercase .MD, empty diff, missing/all-zero base, invalid range, and renamed source→docs. Every uncertain case must return false/full-suite.

Verdict: changes needed, narrow and above the 70–80% completion threshold. Keep the current workflow shape; fix the checker authority and add behavior-level classifier tests, then rerun all exact-head jobs. No branch mutation or merge from this review lane.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants