agent-output-contract

A tiny local-first CLI for checking that AI coding agents delivered the files and artifacts they promised.

Use it as a lightweight acceptance gate after an agent run: assert required files exist, expected text appears, forbidden text is absent, JSON files parse, and Markdown docs contain required headings. It makes agent handoffs easier to verify without calling an LLM or sending code anywhere.

Why

AI coding agents often finish with a confident summary even when a README, test, migration, changelog, or generated artifact is missing. agent-output-contract lets you keep a small YAML or JSON contract next to the task and fail fast in CI or a local hook when the output does not match.

Install

python -m pip install -e .

No runtime dependencies beyond Python 3.9+.

Usage

Create agent-output-contract.yml:

required_files:
  - README.md
  - pyproject.toml

forbidden_paths:
  - .env
  - secrets/**

files:
  README.md:
    contains:
      - "## Install"
      - "## Usage"
    markdown_headings:
      - "Install"
      - "Usage"
  pyproject.toml:
    contains:
      - "[project]"

json_files:
  - package.json

Run:

agent-output-contract check --contract agent-output-contract.yml --root .

Exit codes:

• 0: all checks passed
• 1: one or more contract checks failed
• 2: contract or invocation error

Contract fields

Top-level fields:

• required_files: list of files or glob patterns that must match at least one file.
• forbidden_paths: list of files or glob patterns that must not match any file.
• files: mapping of path/glob to per-file checks:
  • contains: strings that must appear in each matched file.
  • not_contains: strings that must not appear in each matched file.
  • markdown_headings: Markdown heading titles that must appear (case-sensitive, heading level ignored).
  • max_bytes: maximum allowed file size in bytes.
• json_files: list of JSON files/globs that must parse.

YAML is supported when Python has PyYAML installed. JSON contracts work everywhere.

CI example

- name: Check agent output contract
  run: |
    python -m pip install git+https://github.com/thecatnamedkuro/agent-output-contract.git
    agent-output-contract check --contract agent-output-contract.yml --root .

This repository also ships a GitHub Actions workflow that runs tests, the documented JSON-contract smoke check, and a package build on pushes and pull requests.

Development

python -m pip install -e '.[test,yaml]' build
python -m pytest
agent-output-contract check --contract examples/agent-output-contract.json --root .
python -m build --sdist --wheel

Commercial angle

The open-source CLI is intentionally local and boring. A hosted version could later collect contract pass/fail history across teams, expose reusable policy packs for common agent workflows, and provide PR annotations.

Safety

This tool does not call network APIs, does not invoke an LLM, and only reads files under the chosen --root.

License

MIT