Skip to content

feat: AI-powered code review for pull requests #1067

@myakove

Description

@myakove

Summary

Add an AI-powered code review feature that analyzes PR diffs and posts inline review comments on GitHub. Each AI provider uses 3 specialized review agents (quality, guidelines, security) downloaded from myk-org/pi-config. When multiple providers are configured, they enter a peer review consensus loop.

Problem / Motivation

Currently, code reviews are entirely manual. An AI-powered review system can catch common issues (security vulnerabilities, code quality problems, guideline violations) automatically on every PR, reducing reviewer burden and improving code quality. The peer review consensus loop (when multiple AI providers are configured) reduces false positives and increases confidence in findings.

Requirements

  1. New ai-review config section under existing ai-features in schema.yaml
  2. Config supports single provider (standard review) and multiple providers (peer review with consensus)
  3. Config available at global level, per-repo in config.yaml, and per-repo in .github-webhook-server.yaml
  4. Trigger on PR opened and synchronize (new commits), skip on clean rebase (existing detection)
  5. Download 3 review agent files from myk-org/pi-config to all provider dirs (.claude/agents/, .gemini/agents/, .cursor/agents/) before each review
  6. Each AI provider runs all 3 agents (quality, guidelines, security) and consolidates findings
  7. Peer review loop adapted from jenkins-job-insight: anti-sycophancy prompts, cross-peer visibility, consensus checking
  8. Post inline review comments via GitHub Pull Request Review API (create_review) - real code comments on diff lines
  9. Review summary with severity counts (CRITICAL/WARNING/SUGGESTION)
  10. Fire-and-forget background task (same pattern as test-oracle)
  11. Default timeout: 30 minutes per AI CLI call
  12. Never break webhook processing on failures

Deliverables

  • Add ai-review schema to webhook_server/config/schema.yaml (under $defs.ai-features and repo-level)
  • Create webhook_server/libs/ai_review.py (agent setup, prompts, JSON parsing, peer review loop, posting)
  • Extract ai_review_config in webhook_server/libs/github_api.py
  • Integrate trigger in webhook_server/libs/handlers/pull_request_handler.py (opened + synchronize non-clean)
  • Create webhook_server/tests/test_ai_review.py with comprehensive tests
  • Update AGENTS.md with AI review documentation
  • Ensure 90% test coverage for new code

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions