Skip to content

Add versioned HTML index sources and systemd documentation - #137

Open
TheRealBecks wants to merge 1 commit into
neuledge:mainfrom
TheRealBecks:TheRealBecks/issue116
Open

Add versioned HTML index sources and systemd documentation#137
TheRealBecks wants to merge 1 commit into
neuledge:mainfrom
TheRealBecks:TheRealBecks/issue116

Conversation

@TheRealBecks

Copy link
Copy Markdown
Contributor

Systemd's reference manuals live in DocBook XML, so indexing its Markdown guides omits systemctl, journalctl, unit configuration and most APIs. This implements the rendered-HTML approach proposed in #116: a versioned HTML-index source and two systemd registry packages.

  • html-index accepts explicit numeric releases and an HTTPS URL with a {version} directory. It fetches the index's HTML links within that origin and directory, with bounded concurrency, timeouts, retries, response/page limits, and redirects checked before following them. Failed downloads fail the build.
  • Raw downloads are cached by URL; identical manual aliases are indexed once. The existing nightly publisher skips releases that are already published. The importer uses LinkeDOM, already present in the workspace. The schema, CLI listing and version discovery support explicit HTML releases.
  • systemd/systemd@258 supplies the reference manuals; systemd/systemd-guides@latest supplies the guides from Git tag v258. Registry documentation explains the source schema, limits and cache refresh procedure.

Live validation also exposed that Turndown flattens DocBook's bare <pre> elements. A small HTML-parser fix preserves these as fenced code blocks, with regression tests and a patch changeset.

Validation:

  • pnpm install --frozen-lockfile, pnpm lint, pnpm build, pnpm test — passed; 317 tests.
  • Systemd 258: 1,229 manual URLs deduplicated to 432 documents, 3,234 sections and 1,108,416 tokens. Confirmed key manuals, service examples and SQLite integrity.
  • Guides at v258: 531 sections and 220,331 tokens; passed the CI registry-validation command.
  • The manuals also passed automatic version discovery and a cached build with network requests disabled.

Versions are advanced deliberately in the definitions. Cached release content is reused until its cache is removed; no native XML processing is added.

Closes #116.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support DocBook XML sources for complete systemd documentation

1 participant