Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion .agents/agents/catalogue-auditor.agent.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,8 @@ Load and apply these skills:
packages, relationships, `order` sorting).
2. **Deterministic guard** — `just check-projects` passes.
3. **Documentation** — each `docs` config points at a path that exists, a root page that resolves,
exclusions that cover `_Sidebar.md`/`index.md` stubs, and a non-zero aggregated page count.
exclusions that keep the navigation files (`_Sidebar.md`, `_Footer.md`) out of the page list, and a
non-zero aggregated page count.
4. **Stability** — `status` matches the repository `archived` flag and the NuGet release channel.
5. **Metadata and tags** — repository description present and useful, topics non-empty (they are the
site's tag chips), homepage/link targets correct, and `discussions` left unset (it is detected
Expand Down
12 changes: 7 additions & 5 deletions .agents/skills/add-catalogue-project/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,8 +97,8 @@ Add the entry as `src/src/data/projects/<id>.yml` (see `project-manifest-referen
docs:
source: github-path | wiki | readme
path: docs # github-path only
rootPage: Getting-Started.md # optional; see docs-aggregation-rules
exclude: [ _Sidebar.md, index.md ]
rootPage: Home.md # optional; see docs-aggregation-rules
exclude: [ _Sidebar.md ] # only files the repository actually ships
targetFrameworks: [ net8.0, net9.0 ]
install: nuget | msbuild-sdk | dotnet-tool # NuGet packages only; omit for npm
packages:
Expand Down Expand Up @@ -169,9 +169,11 @@ docs not yet in a `docs/` folder, no stable release yet).
## Pitfalls

- `docs.path` must be a **repository path** (`docs`, `docs/wiki`), never a URL.
- A repository whose wiki is mirrored into `docs/wiki` usually needs
`exclude: [_Sidebar.md, Home.md, index.md]` plus `rootPage: Getting-Started.md`, otherwise the
aggregation raises `DocsValidationError`. See `docs-aggregation-rules`.
- A repository whose wiki is mirrored into `docs/wiki` is plain Markdown: `Home.md` is the hub and
`_Sidebar.md` (the navigation) is the only file that has to be excluded. A wiki that keeps a
different first page keeps `rootPage`; only a page the repository does **not** ship needs excluding,
and a stale pattern (an `index.md` the upstream deleted) is a silent no-op rather than an error.
See `docs-aggregation-rules`.
- Aggregation reads branch `main`. If the repository's default branch is `master`/`develop`, docs
aggregation silently yields nothing — rename or report it.
- The `README.md`-as-index mode (`readme:`) publishes exactly one page and cannot be combined with
Expand Down
8 changes: 6 additions & 2 deletions .agents/skills/audit-catalogue-projects/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -48,8 +48,12 @@ Related: `project-manifest-reference` (fields and vocabularies), `docs-aggregati
- `docs.source` matches reality: `github-path` when the repository has a docs folder,
`wiki` when the content lives in the wiki, `readme` only for single-page projects.
- `docs.path` exists **on branch `main`** and contains `.md` files.
- A `rootPage` either names the conventional index or is paired with an `exclude` that removes it.
- `exclude` covers `_Sidebar.md` (and `Home.md`/`index.md` stubs) so they do not appear as pages.
- `rootPage` names a page that exists under `docs.path`; when it names a non-index page and a
conventional root page (`index.md`/`Home.md`/`readme.md`) also exists, that conventional page must be
listed in `exclude`.
- `exclude` keeps the navigation files out of the page list: `_Sidebar.md` (and `_Footer.md` for a
`wiki` source). It is a filter over the files the repository ships, so a pattern naming a file that
no longer exists is a no-op, not a warning or a failure.
- The aggregated page count is non-zero and the landing page is the page a reader should land on.
- Archived projects that still declare `docs` are intentional (they are excluded from the portal).
- Documentation is not duplicated here: this repository only holds the mirror.
Expand Down
41 changes: 36 additions & 5 deletions .agents/skills/docs-aggregation-rules/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,20 +92,44 @@ A project may publish two views of its documentation:
4. If `rootPage` names a file that does not exist, aggregation raises
*"docs.rootPage: ... does not exist"*.

The wikis of several product repositories are mirrored into `docs/wiki`, so the working pattern is:
The product wikis are mirrored into `docs/wiki` and are **plain Markdown**: `Home.md` is the hub,
`_Sidebar.md` is the whole navigation, and the repository carries no site configuration at all — the
engineering playbook deleted its `mkdocs.yml` and its TechDocs `index.md`, and no repository needs to
add them back. A repository that still ships a site file (`mkdocs.yml`) or a generated `index.md` is a
**deviation this portal ignores**: aggregation reads `docs.path` and this entry's `exclude`, never the
site configuration. So the pattern for a wiki is:

```yaml
docs:
source: github-path
path: docs/wiki
rootPage: Getting-Started.md
exclude: [ _Sidebar.md, Home.md, index.md ]
rootPage: Home.md
exclude: [ _Sidebar.md ]
```

`Home.md` is itself a conventional root page, so `rootPage` restates the default rather than
overriding it, and nothing has to be excluded for it to win. Exclude only the files the repository
actually ships.

While a repository still carries a site file, `just data-sync` reports it once and **never fails**:

docs containers: mkdocs.yml is a repository site configuration and the portal ignores it — the
catalogue entry alone decides what is published, so the repository can drop the file.

That is an advisory (`findLegacySiteConfig` in `src/lib/docs/aggregate.ts`), not a gate, and it is
printed on stdout so it survives into the build log; on GitHub Actions the same text is repeated as a
`::warning` annotation on the run. The aggregation keeps stripping the Material markup such a file
stands for (`./mkdocs`), so the file can be deleted upstream in its own time and on its own pull
request; when it is gone the advisory simply stops appearing.

## Exclusions and ordering

- `exclude` matches the **base file name** (`index.md`, `_Sidebar.md`) with `*` wildcards; it is
applied **before** the root page is selected (that is what makes rule 2 work).
- A pattern that matches nothing is a **no-op**: excluding `index.md` from a wiki that has since
replaced it with `Home.md` neither warns nor fails, so a stale pattern cannot block the build while
the upstream repository is mid-reorganisation. Remove it when you notice, but it is not drift the
guard reports.
- Page order comes from, in priority: `docs.order` (explicit slug list) → a `_Sidebar.md` link order
(both for `wiki` and for a mirrored wiki under `github-path`) → `1000`.
- The landing page is always `order: -1`, so it heads the sidebar as "Overview".
Expand All @@ -126,8 +150,11 @@ the catalogue flags the project, ADR 0004), `lastReviewed`, `sourceProject`,
absent when there are none), and `sidebar.label`/`sidebar.order`.

Additional transformations: GitHub alert blockquotes become Starlight asides; relative links between
documents are rewritten to portal URLs; images are rewritten to GitHub raw URLs; duplicate top-level
H1s are demoted to keep one title per document.
documents are rewritten to portal URLs; **relative** images are rewritten to GitHub raw URLs while
**absolute** image URLs pass through untouched — a registry badge such as
`![Purview.BuildSdk](https://img.shields.io/nuget/v/Purview.BuildSdk)` keeps its own URL, so a page can
show a live version without stating a number that rots; duplicate top-level H1s are demoted to keep one
title per document.

## What the site does with it

Expand Down Expand Up @@ -159,3 +186,7 @@ just build && just check-generated # asserts the mirror matches the manifest
A reorganised tree is verified in the built output: the old flat slug must be a meta-refresh page
pointing at the new address, on every channel that publishes the tree (`/docs/<id>/<old>/` and
`/docs/<id>/prerelease/<old>/`), and the sidebar of a nested channel must contain a `group-label`.

A page that shows a registry badge instead of a version is verified the same way: the badge URL must
appear verbatim in the built page (`dist/docs/<id>/<page>/index.html`), not rewritten to a
`raw.githubusercontent.com` address. `docs.test.ts` covers an absolute badge URL beside a relative one.
9 changes: 7 additions & 2 deletions .agents/skills/project-manifest-reference/SKILL.md
Original file line number Diff line number Diff line change
Expand Up @@ -70,8 +70,13 @@ Adding a value to any of these lists is a schema change and requires an ADR. `ex

- `rootPage` names the markdown file that becomes the landing page (`index`). It may only select a
non-index page when the conventional root page is removed via `exclude` (the aggregation otherwise
raises `DocsValidationError`).
- `exclude` matches the **base file name** (not the repository path) and supports `*` wildcards.
raises `DocsValidationError`). A product wiki mirrored from `docs/wiki` uses `rootPage: Home.md`: the
wiki is plain Markdown with `Home.md` as its hub and `_Sidebar.md` as the whole navigation, and it
carries no site configuration — a repository that still ships one is a deviation the portal ignores,
not a requirement.
- `exclude` matches the **base file name** (not the repository path) and supports `*` wildcards. It is a
filter over the files the repository ships, so a pattern naming a file that has since been deleted is a
no-op, never a warning or a failure.
- `readmeAsIndex` uses the repository `README.md` as the landing page and cannot be combined with
`rootPage`; it aliases `readme` to `index`.
- `order` is an explicit list of page slugs, applied before the `_Sidebar.md` order.
Expand Down
21 changes: 0 additions & 21 deletions mkdocs.yml

This file was deleted.

12 changes: 8 additions & 4 deletions src/src/data/projects/engineering-playbook.yml
Original file line number Diff line number Diff line change
Expand Up @@ -36,9 +36,11 @@ docs:
source: github-path
path: docs/wiki
rootPage: Home.md
# `_Sidebar.md` is the wiki's navigation, not a page. Nothing else is
# excluded: the wiki is plain Markdown with no site configuration, so
# `Home.md` is the hub (`rootPage` above) and is published as the landing page.
exclude:
- _Sidebar.md
- index.md
related:
- build-sdk
- containers
Expand All @@ -54,16 +56,18 @@ useCases:
at a time.
code: |
{
"sdk": { "allowPrerelease": false },
"msbuild-sdks": { "Purview.BuildSdk": "1.0.4" },
"sdk": { "version": "LATEST", "allowPrerelease": false },
"msbuild-sdks": { "Purview.BuildSdk": "LATEST" },
"test": { "runner": "Microsoft.Testing.Platform" }
}
language: json
outcome: >-
A pinned SDK plus a `Directory.Build.props` that sets one namespace prefix is the whole of the
repository's build configuration. Every project beneath it inherits the target framework, language
version, nullable context, analyzers and assembly identity, so a project file is two lines and a test
project file is empty.
project file is empty. `LATEST` is the playbook's own placeholder for the newest version the runtime
supports — the file requires a number, so the snippet shows a lookup rather than a claim that expires
with the next release.
evidence: >-
Getting Started walks the steps in the order you would create the files, and every repository-level
file has a copy-paste version in the templates folder with a comment explaining the one thing about it
Expand Down
62 changes: 62 additions & 0 deletions src/src/lib/docs/aggregate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -305,6 +305,65 @@ function isIndexFile(baseName: string): boolean {
return ['home.md', 'index.md', 'readme.md'].includes(baseName.toLowerCase());
}

/**
* Site-configuration files a repository may still carry beside its Markdown.
*
* The portal never reads them: the catalogue entry's `docs` block — `path`,
* `rootPage`, `exclude` and the `_Sidebar.md` order — is the only thing that
* decides what is published, so the file is dead weight in the product
* repository. `mkdocs.yml` is the reason a page can arrive with Material
* markup at all (see `./mkdocs`), which is why the aggregation still strips it
* while the upstream repositories catch up.
*/
const LEGACY_SITE_CONFIG_FILES = new Set(['mkdocs.yml', 'mkdocs.yaml']);

/**
* The site-configuration file present in a repository tree, or null.
*
* Matched on the base name at any depth, so a repository that keeps its config
* inside `docs/` is reported too. Pure and exported so the advisory below is
* testable without a network.
*/
export function findLegacySiteConfig(paths: readonly string[]): string | null {
const match = paths.find((path) =>
LEGACY_SITE_CONFIG_FILES.has((path.split('/').pop() ?? '').toLowerCase()),
);
return match ?? null;
}

const warnedLegacySiteConfig = new Set<string>();

/** True when the sync is running inside GitHub Actions, where a workflow command draws an annotation. */
function inGithubActions(): boolean {
return process.env.GITHUB_ACTIONS === 'true';
}

/**
* Advisory, never a gate: a repository that still ships a site configuration is
* reported once per sync so it can be deleted upstream in its own time. Nothing
* here fails the build (ADR 0001 — fix the data, never weaken a check, and never
* turn an upstream tidy-up into a blocked release).
*
* The line goes to stdout rather than stderr because the pipeline captures
* stdout into the build log: an advisory nobody can see cannot be acted on.
* Inside GitHub Actions it is additionally emitted as a `::warning` workflow
* command, which annotates the run without failing it.
*/
function warnLegacySiteConfig(projectId: string, paths: readonly string[]): void {
const file = findLegacySiteConfig(paths);
if (!file || warnedLegacySiteConfig.has(projectId)) {
return;
}
warnedLegacySiteConfig.add(projectId);
const message =
`${file} is a repository site configuration and the portal ignores it — ` +
'the catalogue entry alone decides what is published, so the repository can drop the file.';
console.log(` docs ${projectId}: ${message}`);
if (inGithubActions()) {
console.log(`::warning title=Documentation site configuration::${projectId}: ${message}`);
}
}

function normalizeRootPage(path: string): string {
return path.replace(/\\/g, '/').replace(/^\.\//, '').replace(/^\/+/, '').replace(/\/+$/, '');
}
Expand Down Expand Up @@ -493,6 +552,9 @@ async function collectGithubPathChannel(
): Promise<AggregatedPage[]> {
const ref = channel.ref;
const files = await listRepositoryFiles(project.repository, ref);
// The tree is already in hand, so noticing a repository site configuration
// costs nothing — and it is reported, never enforced.
warnLegacySiteConfig(project.id, files);
const docRoot = layout.path.replace(/\/+$/, '');
const docFiles = files
.filter((path) => path.startsWith(`${docRoot}/`) && path.endsWith('.md'))
Expand Down
77 changes: 77 additions & 0 deletions src/tests/unit/docs.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { join } from 'node:path';
import {
DOCS_MANIFEST_SCHEMA,
DocsValidationError,
findLegacySiteConfig,
isExcluded,
isMirrorManifestCurrent,
selectDocsRootFile,
Expand Down Expand Up @@ -308,6 +309,30 @@ describe('link rewriting', () => {
);
});

test('keeps an absolute badge image beside a relative one', () => {
// A page shows the current version as a registry badge rather than stating
// a number. The badge URL is absolute, so the aggregator must neither
// prefix it with the image base nor mistake it for a repository path; the
// relative screenshot in the same table still resolves to the image base.
const output = rewriteDocMarkdown(
'| Purview.BuildSdk | ![Purview.BuildSdk](https://img.shields.io/nuget/v/Purview.BuildSdk) |\n' +
'| Traces | ![Traces](AspireTracesView.png) |',
context(),
);
expect(output).toBe(
'| Purview.BuildSdk | ![Purview.BuildSdk](https://img.shields.io/nuget/v/Purview.BuildSdk) |\n' +
'| Traces | ![Traces](https://github.com/purview-dev/demo/raw/main/docs/AspireTracesView.png) |',
);
});

test('keeps a scoped npm badge image unchanged', () => {
const output = rewriteDocMarkdown(
'![astroc4](https://img.shields.io/npm/v/@purview-dev/astroc4)',
context(),
);
expect(output).toBe('![astroc4](https://img.shields.io/npm/v/@purview-dev/astroc4)');
});

test('keeps external URLs unchanged', () => {
const output = rewriteDocMarkdown('[GitHub](https://github.com/purview-dev)', context());
expect(output).toBe('[GitHub](https://github.com/purview-dev)');
Expand Down Expand Up @@ -467,6 +492,58 @@ describe('docs exclude matching', () => {
expect(isExcluded('Getting-Started.md', ['index.md', '_Sidebar.md'])).toBe(false);
expect(isExcluded('index.md', undefined)).toBe(false);
});

test('a pattern for a file the repository no longer ships is a no-op', () => {
// `exclude` is a filter over the files the repository has, not a promise
// about them: a wiki that dropped its `index.md` in favour of a `Home.md`
// hub keeps working while the stale pattern is still declared. The root
// page resolves to real files only, so nothing fails and nothing warns.
const exclude = ['_Sidebar.md', 'index.md'];
const sourceFiles = [
rawDoc('docs/wiki/Home.md'),
rawDoc('docs/wiki/_Sidebar.md'),
rawDoc('docs/wiki/foundations/Dependency-Versions.md'),
].filter((file) => !isExcluded(file.path.split('/').pop() ?? '', exclude));

const selected = selectDocsRootFile(
'engineering-playbook',
sourceFiles,
'docs/wiki',
'Home.md',
);

expect(selected.index.path).toBe('docs/wiki/Home.md');
expect(selected.regular.map((file) => file.path)).toEqual([
'docs/wiki/foundations/Dependency-Versions.md',
]);
expect(selected.slugAliases.get('home')).toBe('index');
});
});

describe('legacy site configuration', () => {
test('finds a repository site config file at any depth', () => {
expect(findLegacySiteConfig(['README.md', 'mkdocs.yml', 'docs/index.md'])).toBe('mkdocs.yml');
expect(findLegacySiteConfig(['docs/mkdocs.yaml'])).toBe('docs/mkdocs.yaml');
expect(findLegacySiteConfig(['MKDOCS.YML'])).toBe('MKDOCS.YML');
});

test('a plain-Markdown wiki has none', () => {
// The engineering playbook's tree: `Home.md` is the hub, `_Sidebar.md` is
// the navigation, and the repository carries no site configuration at all.
expect(
findLegacySiteConfig([
'docs/wiki/Home.md',
'docs/wiki/_Sidebar.md',
'docs/wiki/foundations/Dependency-Versions.md',
'skills/purview-architecture/SKILL.md',
]),
).toBeNull();
});

test('does not match a file that merely mentions the name', () => {
expect(findLegacySiteConfig(['docs/mkdocs-notes.md', 'mkdocs.yml.bak'])).toBeNull();
expect(findLegacySiteConfig([])).toBeNull();
});
});

describe('resolveDocRelativeLink', () => {
Expand Down
Loading