diff --git a/content/docs/guide/ci-cd-pipeline.md b/content/docs/guide/ci-cd-pipeline.md index 75a0cd6f40..65e9043f45 100644 --- a/content/docs/guide/ci-cd-pipeline.md +++ b/content/docs/guide/ci-cd-pipeline.md @@ -251,10 +251,24 @@ all**, which is the point of the workflow. It appears in the checks list as **In Check**. Runs `scripts/check-doc-links.mjs`, which walks every `.md` / `.mdx` file under `content/docs/` and -resolves each internal markdown link against the files actually on disk (`/docs/foo` must have a -`foo.md`, `foo.mdx` or `foo/index.md*` under `content/docs/`). External `http(s)` and `mailto:` -links and bare `#anchors` are skipped — those belong to Lychee, below. No install, no build, no -network: a checkout and one `node` call. +asks of each internal markdown link the only question a reader cares about — **does the site serve +this URL?** Four checks, by href shape: + +| Href shape | Resolved against | Rejected when | +|---|---|---| +| relative (`../plugins/plugin-charts.mdx`) | the linking file's directory | the target file is missing… | +| relative escaping the collection (`../../../packages/x/README.md`) | — | …**or** it resolves outside `content/docs/` (fumadocs can only resolve inside its page index, so the href reaches the browser verbatim — a 404 even though the file exists) | +| absolute `/docs/...` | `content/docs/` as a **route** | no `foo.md`, `foo.mdx` or `foo/index.md*` backs it — a `.md`/`.mdx` suffix always fails, since that URL 404s whatever is on disk | +| any other absolute (`/spec/...`, `/img/...`) | the **site itself**: route segments enumerated from `apps/site/app`, plus static files under `apps/site/public` | no route pattern or static file matches | + +External `http(s)` and `mailto:` links and bare `#anchors` are skipped — those belong to Lychee, +below. No install, no build, no network: a checkout and one `node` call. + +The last two rows are objectui#3490. Reading `apps/site` widens the script's responsibility, and +that is the deliberate purchase: it is the only way to catch a link to a route that does not exist, +and 18 such 404s had accumulated while the check waved every non-`/docs` absolute href through. The +cost is that a docs PR can now go red because `apps/site` moved under it — correct, but real. The +header of the script argues the trade-off in full. **Why it blocks a merge.** A broken internal link is a 404 on the published site, and nothing else in CI sees it: the site build succeeds with a dead link in it. The script itself is older than its @@ -292,7 +306,7 @@ There are **two** link checkers, and they cover different things (objectui#3213) | | Covers | Network | Runs | |---|---|---|---| -| `scripts/check-doc-links.mjs` | **Internal** `/docs/...` routes, resolved against `content/docs/` | No | `docs-links.yml` — every push and PR, no path filter (previous section) | +| `scripts/check-doc-links.mjs` | **Internal** links in `content/docs/`: relative hrefs, `/docs/...` routes, and every other site-absolute href (against `apps/site`) | No | `docs-links.yml` — every push and PR, no path filter (previous section) | | Lychee (this workflow) | **External** URLs, plus **relative** in-repo file links, in `content/docs/`, `docs/` and `README.md` | Yes | Weekly cron and manual dispatch | Lychee sweeps **both** documentation trees: `content/docs/` (the 183 pages the site publishes) and diff --git a/content/docs/guide/component-registry.md b/content/docs/guide/component-registry.md index c3bac255be..df2fe6c7ff 100644 --- a/content/docs/guide/component-registry.md +++ b/content/docs/guide/component-registry.md @@ -459,10 +459,10 @@ export { RatingComponent } - [Expression System](./expressions.md) - Learn about dynamic expressions - [Schema Rendering](./schema-rendering.md) - Understand the rendering engine -- [Creating Custom Components](/spec/component-package.md) - Deep dive into component creation +- [Custom Plugin Development](/docs/guide/plugin-development) - Deep dive into component creation ## Related Documentation -- [Core API](/api/core) - Component registry API -- [React API](/api/react) - React integration -- [Component Specification](/spec/component.md) - Component metadata spec +- [`@object-ui/core` README](https://github.com/objectstack-ai/objectui/tree/main/packages/core) - Component registry API +- [`@object-ui/react` README](https://github.com/objectstack-ai/objectui/tree/main/packages/react) - React integration +- [Schema Type Reference](/docs/api/schema-reference) - Component metadata reference diff --git a/content/docs/guide/data-source.md b/content/docs/guide/data-source.md index 7b30708552..8b170c41ae 100644 --- a/content/docs/guide/data-source.md +++ b/content/docs/guide/data-source.md @@ -199,5 +199,5 @@ behaviour: probe `/batch` and fall back to the non-atomic emulation on `404`/`405`/`501`. Atomic cross-object saves are therefore guaranteed only against backends that advertise the capability; older ones still save, but best-effort. See the -[adapter README](../../../packages/data-objectstack/README.md#cross-object-atomic-batch-batchtransaction) +[adapter README](https://github.com/objectstack-ai/objectui/blob/main/packages/data-objectstack/README.md#cross-object-atomic-batch-batchtransaction) for the full capability table and minimum-backend note. diff --git a/content/docs/guide/expressions.md b/content/docs/guide/expressions.md index c49c824d04..303bbf37a9 100644 --- a/content/docs/guide/expressions.md +++ b/content/docs/guide/expressions.md @@ -636,10 +636,10 @@ const data: UserData = { /* ... */ } - [Schema Rendering](./schema-rendering.md) - Learn the rendering engine - [Component Registry](./component-registry.md) - Understand components -- [Protocol Overview](/protocol/overview) - Explore schema specifications +- [Schema Overview](/docs/guide/schema-overview) - Explore schema specifications ## Related Documentation -- [Core API](/api/core) - Expression evaluator API -- [Form Protocol](/protocol/form) - Form-specific expressions -- [View Protocol](/protocol/view) - Data view expressions +- [`@object-ui/core` README](https://github.com/objectstack-ai/objectui/tree/main/packages/core) - Expression evaluator API +- [Form Plugin](/docs/plugins/plugin-form) - Form-specific expressions +- [View Plugin](/docs/plugins/plugin-view) - Data view expressions diff --git a/content/docs/guide/fields.md b/content/docs/guide/fields.md index 8326539d61..b20fa7c097 100644 --- a/content/docs/guide/fields.md +++ b/content/docs/guide/fields.md @@ -66,7 +66,7 @@ Once registered, you can simply use the new type in your JSON schema. ## Standard Fields -Object UI comes with built-in support for the standard [ObjectStack Protocol](/protocol) types: +Object UI comes with built-in support for the standard [ObjectStack Protocol](https://github.com/objectstack-ai/objectstack/tree/main/packages/spec) types: | Type | Description | |---|---| diff --git a/content/docs/guide/objectos-integration.mdx b/content/docs/guide/objectos-integration.mdx index fa8d77e56a..7e2904e8eb 100644 --- a/content/docs/guide/objectos-integration.mdx +++ b/content/docs/guide/objectos-integration.mdx @@ -659,8 +659,8 @@ test('create contact workflow', async ({ page }) => { - [ObjectStack Documentation](https://docs.objectstack.ai) - [ObjectUI Components Reference](/docs/components) - [ObjectQL Schemas](/docs/api/schema-reference#objectql-schemas) -- [Example: CRM Application](/examples/crm) -- [Example: Kitchen Sink](/examples/kitchen-sink) +- [Example: ObjectStack Console Starter](https://github.com/objectstack-ai/objectui/tree/main/examples/console-starter) +- [Schema Catalog](/docs/guide/schema-catalog) - every schema rendered in these docs ## Support diff --git a/content/docs/guide/plugins.md b/content/docs/guide/plugins.md index 67b293fcec..74645a6c69 100644 --- a/content/docs/guide/plugins.md +++ b/content/docs/guide/plugins.md @@ -512,7 +512,7 @@ export function registerComponents() { - [Component Registry](./component-registry.md) - Understanding the registry - [Schema Rendering](./schema-rendering.md) - How schemas become UI -- [Creating Components](/spec/component-package.md) - Component development +- [Custom Plugin Development](/docs/guide/plugin-development) - Component development - **[Create Plugin Utility](/docs/utilities/create-plugin)** - Scaffold new plugins quickly - **[CLI Tool](/docs/utilities/cli)** - Test plugins with the CLI - **[All Utilities](/docs/utilities)** - Complete toolkit for development diff --git a/content/docs/guide/schema-rendering.md b/content/docs/guide/schema-rendering.md index 13bd854df3..d0eaca1b29 100644 --- a/content/docs/guide/schema-rendering.md +++ b/content/docs/guide/schema-rendering.md @@ -411,11 +411,11 @@ Always type your schemas for better IDE support and fewer runtime errors. - [Component Registry](./component-registry.md) - Learn about component registration - [Expression System](./expressions.md) - Master expressions -- [Protocol Overview](/protocol/overview) - Explore all available schemas +- [Schema Overview](/docs/guide/schema-overview) - Explore all available schemas ## Related Documentation -- [Schema Specification](/spec/schema-rendering) - Technical specification -- [Architecture](/spec/architecture) - System architecture -- [Core API](/api/core) - Core package API reference -- [React API](/api/react) - React package API reference +- [SchemaRenderer](/docs/core/schema-renderer) - Technical reference for the renderer +- [Architecture Overview](/docs/guide/architecture) - System architecture +- [`@object-ui/core` README](https://github.com/objectstack-ai/objectui/tree/main/packages/core) - Core package API reference +- [`@object-ui/react` README](https://github.com/objectstack-ai/objectui/tree/main/packages/react) - React package API reference diff --git a/scripts/__tests__/check-doc-links.test.ts b/scripts/__tests__/check-doc-links.test.ts index 4f77ade290..a5e5eaa1df 100644 --- a/scripts/__tests__/check-doc-links.test.ts +++ b/scripts/__tests__/check-doc-links.test.ts @@ -4,7 +4,7 @@ import os from 'node:os'; import path from 'node:path'; import { fileURLToPath } from 'node:url'; -import { collectBrokenLinks, routeExists, stripCode } from '../check-doc-links.mjs'; +import { collectBrokenLinks, collectSiteRoutes, routeExists, siteUrlExists, stripCode } from '../check-doc-links.mjs'; /** * objectui#3479 — the behaviour test for `scripts/check-doc-links.mjs`. @@ -27,13 +27,34 @@ import { collectBrokenLinks, routeExists, stripCode } from '../check-doc-links.m * * The A-class defect in #3479 is exactly the first shape violated: 13 links * spelled `../plugins/plugin-*.md` while every one of those files is `.mdx`. + * + * objectui#3490 removed the two waivers those checks were still built on — a + * non-`/docs` absolute href was waved through unchecked, and a relative href was + * judged purely as a path on disk. Both answered "does a file exist?" when the + * question is "does the site serve this URL?", and 18 live 404s had collected + * behind them. The gate now enumerates `apps/site/app` + `apps/site/public` as + * its truth source for absolute hrefs, and rejects relative hrefs that resolve + * out of the docs collection. The describes at the bottom pin both. */ const repoRoot = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '../..'); const tempRoots: string[] = []; -/** Materialises `{ 'guide/a.md': '...' }` into a throwaway docs root. */ -function docsRootWith(files: Record): string { +/** + * The default site fixture: the shapes the real `apps/site/app` uses — a route + * group serving `/`, a plain segment, an optional catch-all, a nested route + * handler — plus one static file under `public/`. + */ +const SITE_FIXTURE: Record = { + 'apps/site/app/(home)/page.tsx': 'export default () => null', + 'apps/site/app/playground/page.tsx': 'export default () => null', + 'apps/site/app/docs/[[...slug]]/page.tsx': 'export default () => null', + 'apps/site/app/api/search/route.ts': 'export const GET = () => null', + 'apps/site/public/img/guide/shot.png': 'PNG', +}; + +/** Materialises `{ 'content/docs/guide/a.md': '...' }` into a throwaway repo. */ +function repoWith(files: Record): string { const root = fs.mkdtempSync(path.join(os.tmpdir(), 'check-doc-links-')); tempRoots.push(root); for (const [rel, contents] of Object.entries(files)) { @@ -44,9 +65,21 @@ function docsRootWith(files: Record): string { return root; } +/** Docs-only sugar: keys are relative to `content/docs`, site fixture implied. */ +function docsRootWith(files: Record): string { + const prefixed = Object.fromEntries(Object.entries(files).map(([rel, body]) => [`content/docs/${rel}`, body])); + return path.join(repoWith({ ...SITE_FIXTURE, ...prefixed }), 'content/docs'); +} + +function scan(repo: string): { file: string; href: string; line: number; reason: string }[] { + return collectBrokenLinks(path.join(repo, 'content/docs'), path.join(repo, 'apps/site')); +} + /** The hrefs reported broken, in file order. */ function brokenHrefs(files: Record): string[] { - return collectBrokenLinks(docsRootWith(files)).map((item: { href: string }) => item.href); + const docsRoot = docsRootWith(files); + const repo = path.resolve(docsRoot, '../..'); + return scan(repo).map((item) => item.href); } afterAll(() => { @@ -179,14 +212,16 @@ describe('absolute /docs hrefs stay strict — they are routes, not files', () = expect(brokenHrefs({ 'index.md': '[home](/docs)' })).toEqual([]); }); - it('waves through site routes outside the docs collection', () => { - // `/api/core`, `/img/...`, `/spec/...` are not in this collection and cannot - // be resolved from `content/docs` alone. + it('is checked before the site-route table, so the /docs rule stays the strict one', () => { + // `/docs/[[...slug]]` is a real route in the site fixture, so the generic + // route matcher would accept `/docs/guide/b.md`. It must not get the chance: + // the extension makes it a 404, and the `/docs` branch above owns that call. expect( brokenHrefs({ - 'guide/a.md': '[api](/api/core) ![shot](/img/guide/x.png)', + 'guide/a.md': '[b](/docs/guide/b.md)', + 'guide/b.md': '# B', }), - ).toEqual([]); + ).toEqual(['/docs/guide/b.md']); }); }); @@ -263,11 +298,7 @@ describe('the repo it guards', () => { it('has no broken internal docs links', () => { // The other half of objectui#3479: the extended check must land GREEN, not // arrive with a backlog it merely describes. - const broken = collectBrokenLinks(path.join(repoRoot, 'content/docs')) as { - file: string; - href: string; - line: number; - }[]; + const broken = scan(repoRoot); const report = broken.map((b) => `${path.relative(repoRoot, b.file)}:${b.line} -> ${b.href}`); expect(report).toEqual([]); }); @@ -279,4 +310,154 @@ describe('the repo it guards', () => { expect(routeExists('../plugins/plugin-charts.mdx', { fromFile, docsRoot: root })).toBe(true); expect(routeExists('../plugins/plugin-charts.md', { fromFile, docsRoot: root })).toBe(false); }); + + it('reads the real apps/site tree as its route truth source', () => { + // The coupling objectui#3490 deliberately bought, pinned against the actual + // site: the enumerator must understand this router's shapes (a `(home)` + // group serving `/`, dot-named literal segments, an optional catch-all) and + // must NOT invent the prefixes the 17 dead links assumed. + const site = collectSiteRoutes(path.join(repoRoot, 'apps/site')); + + for (const served of ['/', '/playground', '/llms.txt', '/docs/guide/expressions', '/logo.svg']) { + expect([served, siteUrlExists(served, site)]).toEqual([served, true]); + } + for (const dead of ['/spec/component.md', '/protocol/overview', '/api/core', '/api/react', '/examples/crm']) { + expect([dead, siteUrlExists(dead, site)]).toEqual([dead, false]); + } + }); +}); + +describe('absolute non-/docs hrefs are resolved against the site — objectui#3490 B-class', () => { + it('reports the shape all 17 dead links had: a prefix the router has no segment for', () => { + // The assertion that dies if `if (cleanHref.startsWith('/')) return true;` + // ever comes back. `/spec`, `/protocol`, `/examples` are not route segments + // in apps/site/app, and never were. + expect( + brokenHrefs({ + 'guide/a.md': [ + '[spec](/spec/component-package.md)', + '[protocol](/protocol/overview)', + '[example](/examples/crm)', + ].join('\n\n'), + }), + ).toEqual(['/spec/component-package.md', '/protocol/overview', '/examples/crm']); + }); + + it('reports a sibling of a real segment — /api exists, /api/core does not', () => { + // The near-miss that makes prefix-allowlisting useless: the fixture has + // `app/api/search/route.ts`, so `/api` is a real path prefix. It is not a + // route, and neither is `/api/core`. + expect( + brokenHrefs({ 'guide/a.md': '[core](/api/core) and [search](/api/search)' }), + ).toEqual(['/api/core']); + }); + + it('accepts a static file under public/ — the 3 /img links in #3490 were fine', () => { + expect(brokenHrefs({ 'guide/a.md': '![shot](/img/guide/shot.png)' })).toEqual([]); + }); + + it('reports a public/ path that does not exist', () => { + expect(brokenHrefs({ 'guide/a.md': '![gone](/img/guide/missing.png)' })).toEqual(['/img/guide/missing.png']); + }); + + it('accepts a plain route and the site root, trailing slash or not', () => { + expect(brokenHrefs({ 'guide/a.md': '[p](/playground) [q](/playground/) [home](/)' })).toEqual([]); + }); + + it('refuses to judge an absolute href with no route table rather than waving it through', () => { + // A silent `true` here is exactly how the 18 accumulated. Callers that skip + // the truth source get an error, not a green. + const docsRoot = docsRootWith({ 'guide/a.md': '# A' }); + expect(() => routeExists('/spec/component.md', { fromFile: path.join(docsRoot, 'guide/a.md'), docsRoot })).toThrow( + /site route table/, + ); + }); +}); + +describe('the route table follows Next.js segment semantics', () => { + function tableFor(files: Record) { + return collectSiteRoutes(path.join(repoWith(files), 'apps/site')); + } + + it('drops route groups, skips private and slot folders, and needs a page or route file', () => { + const site = tableFor({ + 'apps/site/app/(marketing)/pricing/page.tsx': 'x', + 'apps/site/app/_internal/secret/page.tsx': 'x', + 'apps/site/app/components/Thing.tsx': 'x', + 'apps/site/app/feed/route.ts': 'x', + }); + + expect(siteUrlExists('/pricing', site)).toBe(true); + expect(siteUrlExists('/(marketing)/pricing', site)).toBe(false); + expect(siteUrlExists('/_internal/secret', site)).toBe(false); + expect(siteUrlExists('/components', site)).toBe(false); + expect(siteUrlExists('/feed', site)).toBe(true); + }); + + it('matches dynamic, catch-all and optional catch-all segments by arity', () => { + const site = tableFor({ + 'apps/site/app/blog/[slug]/page.tsx': 'x', + 'apps/site/app/og/[...slug]/page.tsx': 'x', + 'apps/site/app/wiki/[[...slug]]/page.tsx': 'x', + }); + + expect(siteUrlExists('/blog/hello', site)).toBe(true); + expect(siteUrlExists('/blog', site)).toBe(false); + expect(siteUrlExists('/blog/a/b', site)).toBe(false); + + expect(siteUrlExists('/og/a', site)).toBe(true); + expect(siteUrlExists('/og/a/b', site)).toBe(true); + expect(siteUrlExists('/og', site)).toBe(false); + + expect(siteUrlExists('/wiki', site)).toBe(true); + expect(siteUrlExists('/wiki/a/b', site)).toBe(true); + }); +}); + +describe('relative hrefs may not leave the collection — objectui#3490 A-class', () => { + it('reports a link out of content/docs even though the file is really there', () => { + // `guide/data-source.md:202` pointed at `../../../packages/data-objectstack/ + // README.md#...`. The file exists, so `lychee --offline` and the pre-#3490 + // check both passed it; the site still 404s, because fumadocs can only + // resolve inside the docs page index. The fixture materialises the target, + // so this test fails if existence is ever allowed to decide the verdict. + const repo = repoWith({ + ...SITE_FIXTURE, + 'content/docs/guide/data-source.md': '[adapter README](../../../packages/data-objectstack/README.md#batch)', + 'packages/data-objectstack/README.md': '# Adapter', + }); + + expect(fs.existsSync(path.join(repo, 'packages/data-objectstack/README.md'))).toBe(true); + expect(scan(repo).map((item) => [item.href, item.reason])).toEqual([ + ['../../../packages/data-objectstack/README.md#batch', 'escapes-collection'], + ]); + }); + + it('still accepts relative links that stay inside the collection', () => { + expect( + brokenHrefs({ + 'guide/plugins.md': '[Charts](../plugins/plugin-charts.mdx) and [self](./plugins.md)', + 'plugins/plugin-charts.mdx': '# Charts', + }), + ).toEqual([]); + }); + + it('labels each failure with the check that rejected it', () => { + const repo = repoWith({ + ...SITE_FIXTURE, + 'content/docs/guide/a.md': [ + '[out](../../../elsewhere/README.md)', + '[site](/spec/component.md)', + '[docs](/docs/guide/a.md)', + '[rel](./nowhere.md)', + ].join('\n\n'), + }); + + expect(scan(repo).map((item) => item.reason)).toEqual([ + 'escapes-collection', + 'site-route', + 'docs-route', + 'relative', + ]); + }); }); diff --git a/scripts/check-doc-links.mjs b/scripts/check-doc-links.mjs index ec82bab589..ff7e8ec684 100644 --- a/scripts/check-doc-links.mjs +++ b/scripts/check-doc-links.mjs @@ -54,9 +54,53 @@ * therefore kept strict: `/docs/guide/plugins.md` is NOT accepted just because * `guide/plugins.md` exists on disk, because that URL 404s on the site. * - * Non-`/docs` absolute hrefs (`/spec/...`, `/api/...`, `/img/...`) are still - * waved through: they are routes outside this collection and cannot be resolved - * from `content/docs` alone. + * ## Why this file changed again (objectui#3490) + * + * The paragraph that used to sit here said non-`/docs` absolute hrefs + * (`/spec/...`, `/api/...`, `/img/...`) were "waved through: they are routes + * outside this collection and cannot be resolved from `content/docs` alone", + * and that a relative href was checked purely as a path on disk. Both waivers + * were load-bearing, and both were wrong in the same way: they made the gate's + * question "does a file exist?" when the only question that matters to a reader + * is "does the site serve this URL?". 18 live 404s had collected behind them — + * 17 absolute (`/spec/*`, `/protocol/*`, `/api/core`, `/api/react`, + * `/examples/*`) plus one relative link that resolved out of the collection + * entirely. `lychee --offline` scores all 18 green for the same reason. + * + * **The trade-off this makes explicit.** Answering "does the site serve it?" + * means the script can no longer read only `content/docs`: it now also reads + * `apps/site` — the App-Router segment tree under `apps/site/app` and the static + * files under `apps/site/public` — as its truth source for what URLs exist. That + * widens the script's responsibility (and couples a `content/` gate to an + * `apps/` layout), and it is a deliberate purchase, not an accident: + * + * - Without it there is no gate at all on this class. #3479 is the evidence + * that "no gate" means "it accumulates": 33 dead references before anyone + * looked. The 18 here accumulated behind exactly the waiver being removed. + * - The coupling is to *structure*, not to content. `apps/site/app` is + * enumerated, never imported or executed, so a routing change is picked up + * automatically — the truth source cannot drift from the router the way a + * hand-maintained allowlist of prefixes would. + * - The cost is real and worth naming: adding a route to the site is now the + * thing that makes a docs link legal, so a docs PR can go red because + * `apps/site` moved under it. That failure is *correct* (the link really + * would 404), but it does mean the two trees are no longer independent. + * + * What is deliberately NOT bought here: `next.config.mjs` rewrites/redirects are + * not modelled (the one rewrite, `/docs/:path*.mdx`, sits under the stricter + * `/docs` branch above and never reaches this one), and the scan surface is + * still only `content/docs` — `examples/**` and the root `README.md` remain + * unscanned, tracked separately. + * + * **Relative hrefs may not leave the collection.** `../../../packages/foo/ + * README.md` resolves on disk, so the pre-#3490 check passed it, but + * `source.resolveHref` can only look inside the docs page index; `packages/**` + * is not in it, so the href reaches the browser verbatim and the browser + * resolves it against the *page URL* — `/docs/guide/data-source` + + * `../../../packages/...` lands on `/packages/...`, which is not a route. Such a + * link is rejected with a pointer to the fix the repo already uses everywhere + * else: an absolute `https://github.com/objectstack-ai/objectui/blob/main/...` + * URL (the "Package README" form in `content/docs/plugins/*.mdx`). * * ## Code spans are stripped before scanning * @@ -86,6 +130,8 @@ const FENCE_RE = /^\s*(`{3,}|~{3,})(.*)$/; const INLINE_CODE_RE = /(`+)[^`\n]*\1/g; /** Any URI scheme (`https:`, `mailto:`, `tel:`) — not ours to resolve. */ const EXTERNAL_HREF_RE = /^(?:#|[a-zA-Z][a-zA-Z0-9+.-]*:)/; +/** Files that turn an App-Router directory into a servable URL. */ +const ROUTE_ENTRY_RE = /^(?:page|route)\.(?:js|jsx|ts|tsx|md|mdx)$/; const blank = (text) => text.replace(/[^\n]/g, ' '); @@ -148,13 +194,112 @@ function routeCandidates(base) { return [`${base}.md`, `${base}.mdx`, path.join(base, 'index.md'), path.join(base, 'index.mdx')]; } +/** + * Classifies one `apps/site/app` directory name as a URL segment. + * + * Next.js App Router, the four forms that matter here: + * `(group)` — organisational only, contributes no URL segment + * `_private` — never routable, subtree skipped + * `[[...slug]]` — optional catch-all, matches 0+ segments + * `[...slug]` — catch-all, matches 1+ segments + * `[slug]` — dynamic, matches exactly 1 segment + * anything else is a literal segment (`llms.txt` is a literal, dot included). + */ +function classifySegment(name) { + if (name.startsWith('_') || name.startsWith('@')) return { kind: 'skip' }; + if (name.startsWith('(') && name.endsWith(')')) return { kind: 'group' }; + if (name.startsWith('[[...') && name.endsWith(']]')) return { kind: 'optionalCatchAll' }; + if (name.startsWith('[...') && name.endsWith(']')) return { kind: 'catchAll' }; + if (name.startsWith('[') && name.endsWith(']')) return { kind: 'dynamic' }; + return { kind: 'literal', value: name }; +} + +/** + * Enumerates every URL the site can serve, as a truth source for absolute + * hrefs: the App-Router segment tree under `/app`, plus the static + * files under `/public` (Next serves those from `/`). + * + * Structure only — nothing here is imported or executed, so the table tracks + * whatever the router currently is. See the header for why this script reads + * outside `content/docs` at all. + * + * @returns {{ routes: object[][], assets: Set }} + */ +export function collectSiteRoutes(siteRoot) { + const routes = []; + const assets = new Set(); + + const walkApp = (dir, pattern) => { + let entries; + try { + entries = readdirSync(dir, { withFileTypes: true }); + } catch { + return; + } + if (entries.some((entry) => entry.isFile() && ROUTE_ENTRY_RE.test(entry.name))) { + routes.push(pattern); + } + for (const entry of entries) { + if (!entry.isDirectory()) continue; + const segment = classifySegment(entry.name); + if (segment.kind === 'skip') continue; + walkApp(path.join(dir, entry.name), segment.kind === 'group' ? pattern : [...pattern, segment]); + } + }; + + const walkPublic = (dir, prefix) => { + let entries; + try { + entries = readdirSync(dir, { withFileTypes: true }); + } catch { + return; + } + for (const entry of entries) { + const child = `${prefix}/${entry.name}`; + if (entry.isDirectory()) walkPublic(path.join(dir, entry.name), child); + else assets.add(child); + } + }; + + walkApp(path.join(siteRoot, 'app'), []); + walkPublic(path.join(siteRoot, 'public'), ''); + + return { routes, assets }; +} + +/** Does `segments` match one route pattern? Catch-alls are terminal in Next. */ +function matchesPattern(segments, pattern) { + let index = 0; + for (let cursor = 0; cursor < pattern.length; cursor += 1) { + const part = pattern[cursor]; + if (part.kind === 'literal' || part.kind === 'dynamic') { + if (index >= segments.length) return false; + if (part.kind === 'literal' && segments[index] !== part.value) return false; + index += 1; + continue; + } + // catch-all / optional catch-all: swallows the rest, so it must be last. + if (cursor !== pattern.length - 1) return false; + return segments.length - index >= (part.kind === 'catchAll' ? 1 : 0); + } + return index === segments.length; +} + +/** Is `pathname` (an absolute site path) served by a route or a static file? */ +export function siteUrlExists(pathname, site) { + const normalized = pathname.length > 1 ? pathname.replace(/\/+$/, '') : pathname; + if (site.assets.has(normalized)) return true; + const segments = normalized.split('/').filter(Boolean); + return site.routes.some((pattern) => matchesPattern(segments, pattern)); +} + /** * Resolves one href. * * @param {string} href the raw href as authored - * @param {{ fromFile: string, docsRoot: string }} context + * @param {{ fromFile: string, docsRoot: string, site?: { routes: object[][], assets: Set } }} context */ -export function routeExists(href, { fromFile, docsRoot }) { +export function routeExists(href, { fromFile, docsRoot, site }) { let cleanHref = href.split('#')[0].split('?')[0].trim(); if (!cleanHref) return true; // pure in-page anchor or query try { @@ -170,18 +315,50 @@ export function routeExists(href, { fromFile, docsRoot }) { return routeCandidates(routePath ? path.join(docsRoot, routePath) : docsRoot).some(isFile); } - // Site routes outside this docs collection — not resolvable from here. - if (cleanHref.startsWith('/')) return true; + // Site routes outside this docs collection — checked against the enumerated + // router + `public/` tree (objectui#3490). No table means the caller has not + // supplied a truth source; failing loudly beats silently waving 404s through, + // which is precisely how the 18 links in #3490 accumulated. + if (cleanHref.startsWith('/')) { + if (!site) { + throw new Error( + `routeExists() needs a site route table to judge the absolute href "${href}". ` + + 'Pass `site: collectSiteRoutes(/apps/site)`.', + ); + } + return siteUrlExists(cleanHref, site); + } const base = path.resolve(path.dirname(fromFile), cleanHref); + // A relative href must stay inside the collection: fumadocs' page index is + // the only thing that can resolve one, and it holds nothing else. Resolving + // on disk is not enough — `../../../packages/x/README.md` is a real file and + // still a 404 on the site. See the header. + if (base !== docsRoot && !base.startsWith(docsRoot + path.sep)) return false; + // File form first (what fumadocs' `resolveHref` keys on), then the // extensionless-route spellings. return [base, ...routeCandidates(base)].some(isFile); } -/** @returns {{ file: string, href: string, line: number }[]} */ -export function collectBrokenLinks(docsRoot) { +/** Which of the four checks rejected this href — drives the hint printed below. */ +function classifyBroken(href, { fromFile, docsRoot }) { + const cleanHref = href.split('#')[0].split('?')[0].trim(); + if (cleanHref === DOCS_ROUTE_PREFIX || cleanHref.startsWith(`${DOCS_ROUTE_PREFIX}/`)) return 'docs-route'; + if (cleanHref.startsWith('/')) return 'site-route'; + const base = path.resolve(path.dirname(fromFile), cleanHref); + if (base !== docsRoot && !base.startsWith(docsRoot + path.sep)) return 'escapes-collection'; + return 'relative'; +} + +/** + * @param {string} docsRoot the `content/docs` tree to scan + * @param {string} siteRoot `apps/site` — truth source for absolute hrefs + * @returns {{ file: string, href: string, line: number, reason: string }[]} + */ +export function collectBrokenLinks(docsRoot, siteRoot) { const broken = []; + const site = collectSiteRoutes(siteRoot); for (const file of walk(docsRoot)) { const source = stripCode(readFileSync(file, 'utf8')); @@ -191,12 +368,13 @@ export function collectBrokenLinks(docsRoot) { while ((match = MARKDOWN_LINK_RE.exec(source)) !== null) { const href = match[1].trim(); if (EXTERNAL_HREF_RE.test(href)) continue; - if (routeExists(href, { fromFile: file, docsRoot })) continue; + if (routeExists(href, { fromFile: file, docsRoot, site })) continue; broken.push({ file, href, line: source.slice(0, match.index).split('\n').length, + reason: classifyBroken(href, { fromFile: file, docsRoot }), }); } } @@ -209,9 +387,30 @@ export function collectBrokenLinks(docsRoot) { // as scripts/check-control-bytes.mjs. const invokedDirectly = process.argv[1] && path.resolve(process.argv[1]) === fileURLToPath(import.meta.url); +const HINTS = { + relative: + 'Relative links must name the target FILE including its real extension' + + ' (`../plugins/plugin-charts.mdx`) — that is the key fumadocs indexes pages by.', + 'docs-route': + 'Absolute `/docs/...` links must be extensionless ROUTES' + + ' (`/docs/plugins/plugin-charts`); a `.md`/`.mdx` suffix 404s whatever is on disk.', + 'site-route': + 'Absolute non-`/docs` links must resolve to a real site URL — a route under' + + ' `apps/site/app` or a file under `apps/site/public`. There is no `/spec`,' + + ' `/protocol`, `/api/` or `/examples` route: link the equivalent' + + ' `/docs/...` page, or an absolute `https://github.com/...` URL.', + 'escapes-collection': + 'A relative link may not leave `content/docs` — fumadocs can only resolve' + + ' hrefs inside the docs page index, so it reaches the browser verbatim and' + + ' 404s even though the file exists. Use an absolute' + + ' `https://github.com/objectstack-ai/objectui/blob/main/...` URL instead' + + ' (the "Package README" form used throughout content/docs/plugins/).', +}; + if (invokedDirectly) { const docsRoot = path.resolve('content/docs'); - const broken = collectBrokenLinks(docsRoot); + const siteRoot = path.resolve('apps/site'); + const broken = collectBrokenLinks(docsRoot, siteRoot); if (broken.length > 0) { const targets = new Set(broken.map((item) => `${path.dirname(item.file)}|${item.href.split('#')[0]}`)); @@ -219,14 +418,12 @@ if (invokedDirectly) { `Found ${broken.length} broken docs link${broken.length === 1 ? '' : 's'} (${targets.size} distinct target${targets.size === 1 ? '' : 's'}):`, ); for (const item of broken) { - console.error(`- ${path.relative(process.cwd(), item.file)}:${item.line} -> ${item.href}`); + console.error(`- [${item.reason}] ${path.relative(process.cwd(), item.file)}:${item.line} -> ${item.href}`); } - console.error( - '\nRelative links must name the target FILE including its real extension' + - ' (`../plugins/plugin-charts.mdx`); absolute `/docs/...` links must be' + - ' extensionless routes (`/docs/plugins/plugin-charts`). See the header of' + - ' scripts/check-doc-links.mjs.', - ); + for (const reason of Object.keys(HINTS)) { + if (broken.some((item) => item.reason === reason)) console.error(`\n${reason}: ${HINTS[reason]}`); + } + console.error('\nSee the header of scripts/check-doc-links.mjs.'); process.exit(1); }