Skip to content

feat: tech debt metrics — cumulative review history and trend tracking #206

Closed

Description

@ajianaz

Summary

Add a Tech Debt Metrics system that aggregates review findings across multiple reviews and tracks quality trends over time. Currently, every Cora review is stateless — it has no memory of what was found in previous reviews. This makes it impossible to answer: "Is our codebase getting better or worse?"

Motivation

  • SonarQube's OpenAI-compatible LLM engine (reqwest-based, streaming status) #2 feature after Quality Gates — tracking tech debt over time is how teams justify quality investment
  • Data-driven decisions — "we have 17 open issues, 3 worsening categories" is more actionable than "review found some stuff"
  • Visibility — managers and leads need a dashboard view, not individual PR review comments
  • Cora differentiation — no AI code review tool currently offers cumulative debt tracking

Proposed Design

Data Storage

Store review history in .cora/history/ as lightweight JSON files:

.cora/
├── history/
│   ├── 2026-06-01_abc123.json     # one file per review run
│   ├── 2026-06-03_def456.json
│   └── 2026-06-07_ghi789.json
└── baseline.sarif                  # optional baseline

Each history file:

{
  "timestamp": "2026-06-07T14:30:00+07:00",
  "commit": "abc1234",
  "branch": "feature/auth",
  "files_reviewed": 12,
  "lines_reviewed": 847,
  "findings": {
    "critical": 0,
    "high": 1,
    "medium": 3,
    "low": 2,
    "info": 1
  },
  "categories": {
    "security": { "count": 0, "trend": "stable" },
    "bug_risk": { "count": 1, "trend": "improving" },
    "performance": { "count": 2, "trend": "worsening" },
    "error_handling": { "count": 1, "trend": "improving" },
    "style": { "count": 1, "trend": "stable" },
    "complexity": { "count": 1, "trend": "stable" }
  },
  "quality_score": 7.8,
  "gate_status": "passed",
  "duration_ms": 12453
}

CLI Commands

cora debt — Show Tech Debt Report

$ cora debt
┌──────────────────────────────────────────────────┐
│  CodeCora Tech Debt Report                       │
│  Project: trapfall                               │
│  Period: last 30 days (7 reviews)                │
├──────────────┬───────────┬─────────┬─────────────┤
│ Category     │ Open      │ Change  │ Trend       │
├──────────────┼───────────┼─────────┼─────────────┤
│ Security     │ 0         │ -2      │ ▼ improving │
│ Bug Risk     │ 1         │ ±0      │ ► stable    │
│ Performance  │ 3         │ +1      │ ▲ worsening │
│ Error Handle │ 2         │ -1      │ ▼ improving │
│ Complexity   │ 1         │ ±0      │ ► stable    │
│ Style        │ 2         │ +2      │ ▲ worsening │
├──────────────┼───────────┼─────────┼─────────────┤
│ TOTAL        │ 9         │ -2      │ ▼ improving │
│ Quality Avg  │ 7.5/10    │ +0.3    │ ▲ improving │
│ Est. Debt    │ ~1.5 days │         │             │
└──────────────┴───────────┴─────────┴─────────────┘

cora debt --trend — Show Historical Graph

$ cora debt --trend
CodeCora Quality Trend (last 10 reviews)

8.0 ┤                              ╭──╮
7.5 ┤         ╭──╮    ╭╮    ╭─╮ ╭╯  │
7.0 ┤      ╭╮ │  │ ╭╮ ││ ╭╮ │ │╭╯   │
6.5 ┤   ╭╮ ││ │  │ ││ ││ ││ │ ││    │
6.0 ┤╭╮ ││ ││ │  │ ││ ││ ││ │ ││    │
    └──┴┴─┴┴─┴──┴─┴┴─┴┴─┴┴─┴─┴┴────┘
     R1 R2 R3 R4 R5 R6 R7 R8 R9 R10

cora debt --json — Machine-Readable Output

For integration with dashboards, CI badges, etc.

$ cora debt --json
{
  "total_findings": 9,
  "change_from_previous": -2,
  "trend": "improving",
  "quality_score_avg": 7.5,
  "reviews_analyzed": 7,
  "categories": { ... }
}

cora debt --since v0.3.0 — Filter by Tag/Version

$ cora debt --since v0.3.0
# Only aggregate reviews since git tag v0.3.0

CI Badge Integration

Generate a quality badge SVG that can be embedded in README:

[![CodeCora](https://img.shields.io/endpoint?url=...codecora-quality.json)]()

Badge shows: quality score + trend arrow, e.g. quality: 7.5 ▲

.cora.yaml Configuration

debt:
  enabled: true                    # default: true
  history_dir: .cora/history       # where to store review snapshots
  retention_days: 90               # auto-cleanup old history
  estimate_debt: true              # calculate estimated fix time
  categories:                      # map AI categories to tracking groups
    security: ["security", "vulnerability", "injection"]
    bug_risk: ["bug", "logic", "correctness"]
    performance: ["performance", "memory", "speed"]
    error_handling: ["error", "panic", "unwrap"]
    complexity: ["complexity", "cognitive"]
    style: ["style", "naming", "formatting"]

Architecture

src/
├── engine/
│   ├── debt_tracker.rs       # NEW: History storage + aggregation
│   │   ├── struct DebtSnapshot
│   │   ├── struct DebtReport
│   │   ├── fn save_snapshot(review_result) 
│   │   ├── fn aggregate(since) -> DebtReport
│   │   ├── fn calculate_trend(snapshots) -> Trend
│   │   └── fn estimate_debt_hours(findings) -> f64
│   └── ...
├── commands/
│   └── debt.rs               # NEW: `cora debt` subcommand
├── output/
│   └── badge.rs              # NEW: SVG badge generation
└── commands/
    └── review.rs             # MODIFY: Auto-save snapshot after review

Implementation Checklist

  • Create src/engine/debt_tracker.rs with snapshot save/load
  • Define DebtSnapshot and DebtReport structs
  • Implement category mapping from AI review findings
  • Implement trend calculation (compare snapshots over time)
  • Implement debt estimation (findings × avg fix time)
  • Create src/commands/debt.rs for cora debt subcommand
  • Add --trend flag for ASCII graph output
  • Add --json flag for machine-readable output
  • Add --since <tag> filter by git tag or date
  • Add history auto-save to review pipeline
  • Add .cora.yaml debt config section
  • Implement history retention + cleanup
  • Create SVG badge generator
  • Add --badge flag to output badge JSON for shields.io
  • Write unit tests for aggregation logic
  • Write integration test: multiple reviews → debt report
  • Update README with debt tracking docs

Out of Scope (Future)

  • Web dashboard (separate project / CodeCora Cloud)
  • Cross-repo aggregation
  • Team-level metrics
  • Automated debt ticket creation

Dependencies

  • Quality Gate (Issue #N) — gate status feeds into debt history
  • Secrets Scanner (Issue #N) — secrets findings feed into security category
  • Requires structured category output from AI review (enhancement to existing prompt)

Activity

  1. reopened this on Jun 9, 2026
  2. ajianaz commented on Jun 10, 2026

    @ajianaz
    CollaboratorAuthor

    Phase 1 Done ✅ (PR #252 — merged)

    Completed:

    • src/engine/debt_tracker.rs — DebtSnapshot, DebtReport, DebtConfig, save/load/aggregate
    • Category mapping from LLM issue_type → tracking groups
    • Quality score calculation (0-10)
    • Trend analysis (improving/stable/worsening)
    • Config section debt: in .cora.yaml
    • Auto-save hook in execute_review + execute_chunked_review
    • 32 unit tests, clippy clean, 10/10 CI green

    Next (Phase 2):

    • cora debt subcommand
    • --json machine-readable output
    • --trend ASCII graph
    • --since <tag/date> filter
    • History retention cleanup hook
  3. ajianaz commented on Jun 11, 2026

    @ajianaz
    CollaboratorAuthor

    ✅ All 4 phases implemented and merged. PR #252 (Phase 1), #253 (Phase 2), #254 (Phase 3+4). 495 tests, CI green.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions