feat: Learning Mode with automated tuning suggestions

[bihius]

Problem

Tuning OWASP CRS for a new application is tedious and error-prone. Currently, Guard Proxy requires administrators to manually review every WAF event in detect_only mode, understand which rule fired, decide whether it is a false positive, and then hand-craft a rule exclusion through the API or UI. This process takes weeks to months per application and demands deep CRS knowledge.

There is no open-source Coraza-based tool that automates or semi-automates this workflow. Commercial WAFs such as F5 BIG-IP ASM solve it with "Traffic Learning" — a human-in-the-loop suggestion engine — but nothing comparable exists in the open-source ecosystem.

Goal

Add a Learning Mode to Guard Proxy that:

1. Lets a policy run in detect_only mode while collecting traffic.
2. Periodically analyzes collected WAF events and suggests rule exclusions for likely false positives.
3. Presents suggestions in the admin panel for human review and approval.
4. Turns an approved suggestion into a RuleExclusion record and regenerates the Coraza config.

Proposed Architecture

Phase 1 — Backend: Suggestion Engine

New database model:

class TuningSuggestion(Base):
    __tablename__ = "tuning_suggestions"
    id: Mapped[int] = mapped_column(primary_key=True)
    policy_id: Mapped[int] = mapped_column(ForeignKey("policies.id", ondelete="CASCADE"))
    rule_id: Mapped[int]
    target_type: Mapped[TargetType]  # args, request_headers, etc.
    target_value: Mapped[str]
    scope_path: Mapped[str | None]
    confidence_score: Mapped[int]   # 0-100 heuristic score
    event_count: Mapped[int]
    sample_event_ids: Mapped[list[int]]  # JSON array of log IDs
    status: Mapped[SuggestionStatus]  # pending, approved, rejected
    created_at / resolved_at

New service tuning_analyzer.py:

• Groups logs rows by (policy_id, rule_id, vhost_id, request_uri prefix) within a learning window.
• Parses raw_context (Coraza audit JSON) to extract the target variable that triggered the rule.
• Computes a confidence score using heuristics:
  • Frequency of occurrence
  • Source IP diversity (many IPs = more likely FP)
  • Time distribution (business hours = more likely FP)
  • Anomaly score (score 5 = single rule hit, more likely FP than score 25)
• Persists suggestions with status pending.

New router tuning_suggestions.py:

• GET /policies/{id}/suggestions
• POST /policies/{id}/suggestions/{id}/approve → creates RuleExclusion
• POST /policies/{id}/suggestions/{id}/reject

Phase 2 — Scheduler Integration

Extend the existing AsyncIOScheduler (used for certificate renewal) with a nightly job:

scheduler.add_job(
    analyze_detect_only_policies,
    'interval',
    hours=24,
    id='tuning_analyzer'
)

Alternatively, start with a manual "Analyze Now" button in the UI and add the scheduled job later.

Phase 3 — Frontend: Tuning Suggestions Page

New page in the admin panel:

• Table of pending suggestions: rule name, path, target, confidence, event count.
• Approve button → creates exclusion → prompts to run /config/apply.
• Reject button → marks dismissed.
• View Sample Events link → opens filtered log view for investigation.

Phase 4 — Documentation & Polish

• Update README.waf-rules.md with a "Learning Mode" section.
• Add an ADR in notes/decisions/.

Acceptance Criteria

• Admin can set a policy to detect_only mode.
• After traffic flows, the analyzer produces TuningSuggestion records.
• Admin sees suggestions in the UI with enough context to decide (rule, path, target, count).
• Approving a suggestion creates a RuleExclusion scoped to the correct path and target.
• Rejecting a suggestion dismisses it permanently.
• The feature never auto-approves anything — always human-in-the-loop.
• All new backend code has unit and integration tests.
• Frontend passes pnpm run type-check and pnpm run lint.

Out of Scope (for this issue)

• ML-based scoring — heuristic scoring only. ML can be a future research direction.
• Application presets / questionnaire — related but separate; see Policy profiles: seed a policy with app-specific CRS exclusions (WordPress, Nextcloud, …) #227.
• Crawler-based tuning — integrating ZAP baseline scans is a post-MVP extension.
• Auto-apply or auto-approve — never. Security-critical decisions require human review.

Related

• Policy profiles: seed a policy with app-specific CRS exclusions (WordPress, Nextcloud, …) #227 — Policy profiles / application presets (complementary approach: static exclusion packages vs. dynamic learning)
• README.waf-rules.md — existing tuning documentation
• notes/research/automated-waf-tuning.md — full research write-up

Risks

Risk	Mitigation
Over-exclusion creating security holes	Only suggest target-level exclusions (RuleExclusion), never full rule disables. Require admin approval.
Baseline poisoning during learning	Scoring penalises single-IP clusters and suspicious time patterns.
Admin never reviews suggestions	UI reminders for policies stuck in detect_only for >7 days.
Maintenance drift	Suggestions include the endpoint path; if the app changes, old exclusions can be re-evaluated.

---

Estimated effort: ~900–1200 lines across backend + frontend. Manageable as a single feature branch with incremental commits.

[bihius]

Keeping this open but parking it — this is a large, research-scoped feature (automated tuning suggestions from WAF events) rather than a near-term product requirement. Maybe someday, not currently planned.