LLMS.txt for WordPress

A WordPress plugin that manages and serves /llms.txt for AI-friendly site discovery.

Description

LLMS.txt for WordPress helps you create and maintain a /llms.txt file that makes your WordPress site more discoverable and understandable for AI assistants and language models. The plugin automatically generates a structured text file containing your site's important pages, posts, and content.

Features

• Manual file generation - full control over when /llms.txt is created
• Proper llms.txt format - follows the official spec with Markdown links
• Markdown endpoints - serve any page/post as .md for AI consumption (like Cursor/Mintlify)
• AI Crawler Tracking - tracks visits from GPTBot, ClaudeBot, PerplexityBot, and more
• Custom title and description for your llms.txt header
• Configurable section labels and manual links
• Live preview of generated content
• Cache support for better performance
• Excludes SEO-hidden content automatically (Yoast, Rank Math, SEOPress, AIOSEO)
• Per-post/page control over inclusion
• REST API endpoints for programmatic access

Tracked AI Crawlers

The plugin tracks visits from AI crawlers only (not standard search bots):

Confirmed AI:

• OpenAI (GPTBot, ChatGPT-User, OAI-SearchBot)
• Anthropic (ClaudeBot, Claude-Web)
• Perplexity
• Mistral AI
• Google AI (Google-Extended)
• Cohere AI
• Meta AI
• Common Crawl (CCBot - used for AI training)

Possible AI:

• ByteDance/TikTok (Bytespider)
• Amazon (Amazonbot)
• Google Other

Installation

1. Upload the plugin files to /wp-content/plugins/llms-txt/
2. Activate the plugin through the 'Plugins' menu in WordPress
3. Navigate to Settings > LLMS.txt to configure
4. Click "Generate File" to create your llms.txt

Frequently Asked Questions

What is llms.txt?

llms.txt is a text file that helps AI assistants understand your website structure and content. It's similar to robots.txt but designed for AI discovery.

Where can I configure the plugin?

Go to Settings > LLMS.txt in your WordPress admin dashboard.

Does it work with caching plugins?

Yes, the plugin works with caching plugins. However, AI crawler tracking requires PHP to run. If your caching plugin serves llms.txt as a static file (bypassing PHP), crawler visits won't be logged. To enable tracking with full-page caching, exclude /llms.txt from your cache rules.

How do I see which AI crawlers visited my site?

Go to the "AI Crawlers" tab in the plugin settings to see a summary and recent activity log.

Is my data private?

Yes. All data is stored locally on your WordPress site and anonymized. Only the crawler name, timestamp, and a partial IP address are logged. No content or personal information is collected or stored.

What are Markdown endpoints?

When enabled, your pages and posts become accessible as Markdown files. For example, example.com/about can also be accessed at example.com/about.md. The llms.txt file will link to these .md URLs, making it easier for AI models to consume your content. This is how Cursor and Mintlify structure their documentation.

Changelog

1.0.0

• Initial release
• File management (generate/delete)
• Markdown endpoints for AI-friendly content
• Custom title and description
• Cache management
• AI crawler tracking
• SEO plugin integration

Credits

Built by Fahim Reza