Skip to content

feat: integrate vitepress-plugin-llms for llms.txt generation#266

Open
sriramveeraghanta wants to merge 1 commit intomasterfrom
feat/llms-txt-plugin
Open

feat: integrate vitepress-plugin-llms for llms.txt generation#266
sriramveeraghanta wants to merge 1 commit intomasterfrom
feat/llms-txt-plugin

Conversation

@sriramveeraghanta
Copy link
Copy Markdown
Member

@sriramveeraghanta sriramveeraghanta commented Apr 22, 2026

Summary

  • Adds vitepress-plugin-llms which generates llms.txt (index, ~67 KB) and llms-full.txt (~1.4 MB) in dist/ on build, plus cleaned per-page markdown.
  • Replaces the custom buildEnd walk introduced in feat: agent discovery (Link headers, markdown negotiation, Content-Signal) #263 that copied raw source .md into dist/. Agents hitting /path.md via Accept: text/markdown now get LLM-friendly output (no layout: frontmatter, no Vue component markup).
  • One known tradeoff: the home page (docs/index.md, layout: home) is skipped by the plugin, so / → markdown no longer has a direct equivalent. llms.txt serves as the LLM-oriented index instead.

Test plan

  • pnpm build succeeds
  • dist/llms.txt and dist/llms-full.txt generated
  • Per-page .md files in dist/ are plugin-cleaned (verified self-hosting/overview.md)
  • pnpm check:format passes
  • Spot-check a deployed preview: fetch a page with Accept: text/markdown and confirm cleaned markdown is served
  • Confirm /llms.txt and /llms-full.txt are reachable on the deployed preview

Summary by CodeRabbit

  • Chores
    • Integrated a new VitePress plugin to enhance documentation processing
    • Simplified build configuration by removing custom file copying logic previously used during builds

Replaces the custom buildEnd walk that copied raw source .md into dist
with vitepress-plugin-llms, which emits llms.txt, llms-full.txt, and
per-page cleaned markdown. Agents hitting /path.md via Accept:
text/markdown now get LLM-friendly output without frontmatter layout
directives or Vue component markup.
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 22, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
developer-docs Ready Ready Preview, Comment Apr 22, 2026 3:46pm

Request Review

@coderabbitai
Copy link
Copy Markdown

coderabbitai Bot commented Apr 22, 2026

📝 Walkthrough

Walkthrough

The changes replace a custom build-end hook that manually copied markdown files into the output directory with an integrated VitePress plugin (vitepress-plugin-llms). This simplifies the build pipeline by delegating file handling to the plugin while removing filesystem utilities no longer needed.

Changes

Cohort / File(s) Summary
VitePress Plugin Integration
docs/.vitepress/config.mts, package.json
Added vitepress-plugin-llms dependency and integrated it as a Vite plugin. Removed custom buildEnd() hook that recursively copied markdown files to output directory; removed associated filesystem utility imports (readdirSync, statSync, mkdirSync, copyFileSync, join, relative, dirname).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

Poem

🐰 A plugin hops in, so neat and so clean,
No more manual copying—a build machine!
VitePress now lighter, filesystem freed,
The build pipeline blooms like a well-planted seed.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'feat: integrate vitepress-plugin-llms for llms.txt generation' directly and clearly summarizes the main change: integrating a new VitePress plugin for LLM-friendly Markdown generation, which is the primary objective of the PR.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/llms-txt-plugin

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
docs/.vitepress/config.mts (1)

63-63: Consider passing plugin options for richer llms.txt output.

Invoking llmstxt() with no options works, but the generated llms.txt/llms-full.txt will use defaults (relative links, site title/description inferred from VitePress config). For LLM consumers fetching by URL, configuring at minimum a domain so emitted links are absolute — plus a tailored title/description and optional ignoreFiles to exclude pages you don't want indexed (e.g., stub/landing content) — typically produces a more useful index. This also partially mitigates the known tradeoff that docs/index.md (layout: home) is skipped, since you can provide a custom description/intro via the plugin rather than relying on the home page.

♻️ Example
-      plugins: [llmstxt()],
+      plugins: [
+        llmstxt({
+          domain: "https://developers.plane.so",
+          title: "Plane developer documentation",
+          description:
+            "Self-host Plane, integrate with our API, configure webhooks, and extend your project management platform.",
+        }),
+      ],
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@docs/.vitepress/config.mts` at line 63, The plugin is instantiated with
default options causing emitted llms.txt/llms-full.txt to contain relative links
and inferred metadata; update the llmstxt() call in the VitePress config to pass
explicit options (e.g., domain to force absolute URLs, title and description for
a custom intro, and ignoreFiles to exclude pages) so the generated index is
richer and usable by LLM consumers; locate the llmstxt(...) entry in the plugins
array and replace it with a configured call like llmstxt({ domain:
'https://your.site', title: 'Your Docs Title', description: 'Custom
intro/summary', ignoreFiles: ['path/to/stub.md', ...] }) adjusting values for
your site.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@docs/.vitepress/config.mts`:
- Line 63: The plugin is instantiated with default options causing emitted
llms.txt/llms-full.txt to contain relative links and inferred metadata; update
the llmstxt() call in the VitePress config to pass explicit options (e.g.,
domain to force absolute URLs, title and description for a custom intro, and
ignoreFiles to exclude pages) so the generated index is richer and usable by LLM
consumers; locate the llmstxt(...) entry in the plugins array and replace it
with a configured call like llmstxt({ domain: 'https://your.site', title: 'Your
Docs Title', description: 'Custom intro/summary', ignoreFiles:
['path/to/stub.md', ...] }) adjusting values for your site.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b7a07ee0-3eb2-40df-9da3-e092ab98dc7a

📥 Commits

Reviewing files that changed from the base of the PR and between 6757ff2 and 079bad2.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (2)
  • docs/.vitepress/config.mts
  • package.json

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant