---

PHP Robots.txt

A modern, fluent PHP package for building valid robots.txt files with type-safe crawlers and fail-fast validation. Invalid paths, sitemap URLs, or crawl delays throw a clear exception as you build — never a broken file at render time.

It powers the robots.txt on Dead by Stats.

Requirements

• PHP 8.4 or higher
• A code coverage driver (development only)

Installation

composer require fkrzski/robots-txt

Quick start

Chain rules and render the result with toString():

use Fkrzski\RobotsTxt\RobotsTxt;

$robots = new RobotsTxt();

echo $robots
    ->disallow('/admin')
    ->allow('/public')
    ->crawlDelay(5)
    ->toString();

User-agent: *
Disallow: /admin
Allow: /public
Crawl-delay: 5

Target a specific crawler with userAgent() (or the closure-based forUserAgent()), and write the file to disk with toFile():

use Fkrzski\RobotsTxt\Enums\CrawlerEnum;

(new RobotsTxt())
    ->disallow('/admin')
    ->sitemap('https://example.com/sitemap.xml')
    ->forUserAgent(CrawlerEnum::GOOGLE, function (RobotsTxt $robots): void {
        $robots->disallow('/private')->crawlDelay(10);
    })
    ->toFile(); // writes ./robots.txt

Documentation

Full documentation is hosted at docs.fkrzski.dev/robots-txt:

• Guide — how global and crawler-specific rules compose, output order, and writing to disk.
• API reference — every method with its signature, validation rules, and verified output.
• Crawlers — the full CrawlerEnum list mapped to official user-agent strings.

Contributing

Contributions are welcome — see the Contributing Guide. Run the full test and quality suite with:

composer test

License

Open-sourced software licensed under the MIT License.

Author

PHP Robots.txt was created by Filip Krzyżanowski.