Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@ internal/
- **Clone strategy**: Shallow clone (depth 1, single branch, no tags) to temp dir, deleted after analysis.
- **scc initialization**: `processor.ProcessConstants()` called via `sync.Once` since scc requires global initialization.
- **AI estimation**: When `--ai-estimate` is used, a second pass fetches commit history via provider REST APIs. `provider.CommitLister` interface provides `ListCommits` and `CommitStats`. `aidetect.Detect` classifies commits, `aiestimate.Estimate` orchestrates per-repo. Results attach to existing report model as optional fields.
- **Health classification**: When `--health` is used, repos are classified as Active (<180d), Maintained (180-365d), or Abandoned (>365d) based on last commit date. `--health-details` adds deep analysis: per-window author counts, code churn, bus factor, and velocity trend. Uses the same `CommitLister` interface.
- **Health classification**: When `--health` is used, repos are classified as Active (<180d), Maintained (180-365d), or Abandoned (>365d) based on last commit date. Repos where commit history cannot be fetched (API errors, permissions) are classified as Failed with the error message stored in `RepoHealth.Error`. `--health-details` adds deep analysis: per-window author counts, code churn, bus factor, and velocity trend. Uses the same `CommitLister` interface.
- **Error logging**: API errors from health, health-details, AI estimation, and partial commit stat failures are collected and written to `<report>.error.log` (derived from the report path, e.g. `report.error.log` for `report.json`) when any errors occur. Each line is prefixed with a category for easy filtering. `AnalyzeDetails` and `aiestimate.Estimate` return `(result, []string, error)` where `[]string` contains partial error messages.
- **Vendor/generated filtering**: Always-on filtering using `go-enry` to skip vendor, generated, and binary files during analysis. `FilteredFiles` count is tracked per repo and in report totals.
- **License detection**: After analysis, `license.Detect` scans the cloned repo directory for SPDX license identifiers (e.g., "MIT", "Apache-2.0"). Results appear in the per-repo License column.
- **Code churn / hotspots**: Opt-in via `--churn` flag. Uses provider REST APIs to fetch per-file change data (`--churn-limit N` sets max commits, default 500). `churn.Analyze` collects per-file change frequencies; `churn.ComputeHotspots` ranks files by churn x complexity. Top 20 hotspots shown per repo.
Expand Down
14 changes: 4 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -111,12 +111,6 @@ export CODEMIUM_GITHUB_TOKEN=your_personal_access_token
```
This prompts for your token, verifies it against the GitLab API, and stores it at `~/.config/codemium/credentials.json`.

For self-hosted GitLab instances, set `CODEMIUM_GITLAB_URL`:
```bash
export CODEMIUM_GITLAB_URL=https://gitlab.example.com
codemium auth login --provider gitlab
```

**Option 2: glab CLI**

If you have the [GitLab CLI](https://gitlab.com/gitlab-org/cli) installed and authenticated, codemium can use its token automatically:
Expand Down Expand Up @@ -183,9 +177,6 @@ codemium analyze --provider gitlab --group myorg/mysubgroup

# Specific repos
codemium analyze --provider gitlab --group mygroup --repos api,frontend

# Self-hosted GitLab
CODEMIUM_GITLAB_URL=https://gitlab.example.com codemium analyze --provider gitlab --group mygroup
```

### Analyze trends over time
Expand Down Expand Up @@ -267,7 +258,7 @@ This is especially useful when Bitbucket project codes or repo naming convention

### Repository health classification

Classify repositories as Active, Maintained, or Abandoned based on last commit date:
Classify repositories as Active, Maintained, Abandoned, or Failed based on commit history:

```bash
# Quick health check (1 API call per repo, no cloning)
Expand All @@ -284,6 +275,9 @@ Health categories:
- **Active**: last commit < 180 days ago
- **Maintained**: 180–365 days ago
- **Abandoned**: > 365 days ago
- **Failed**: commit history could not be fetched (API error, permissions, etc.)

When API errors occur during health classification, AI estimation, or detailed analysis, an error log is automatically written next to the JSON report (e.g., `output/report.error.log` for `output/report.json`). Each line is prefixed with a category (`[health]`, `[health-details]`, `[ai-estimate]`, `[ai-estimate-detail]`) for easy filtering with `grep`.

### Additional flags

Expand Down
79 changes: 72 additions & 7 deletions cmd/codemium/main.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ import (
"path/filepath"
"sort"
"strings"
"sync"
"time"

tea "github.com/charmbracelet/bubbletea"
Expand All @@ -20,11 +21,11 @@ import (

"github.com/dsablic/codemium/internal/aiestimate"
"github.com/dsablic/codemium/internal/analyzer"
"github.com/dsablic/codemium/internal/churn"
"github.com/dsablic/codemium/internal/license"
"github.com/dsablic/codemium/internal/auth"
"github.com/dsablic/codemium/internal/churn"
"github.com/dsablic/codemium/internal/health"
"github.com/dsablic/codemium/internal/history"
"github.com/dsablic/codemium/internal/license"
"github.com/dsablic/codemium/internal/model"
"github.com/dsablic/codemium/internal/narrative"
"github.com/dsablic/codemium/internal/output"
Expand All @@ -39,6 +40,13 @@ var (
date = "unknown"
)

// errorEntry represents a diagnostic error for the error log.
type errorEntry struct {
Category string
Repo string
Message string
}

func main() {
root := &cobra.Command{
Use: "codemium",
Expand Down Expand Up @@ -441,6 +449,10 @@ func runAnalyze(cmd *cobra.Command, args []string) error {
program = nil
}

// Diagnostic error collection (written to error.log if non-empty)
var diagErrors []errorEntry
var diagMu sync.Mutex

// AI estimation phase
aiEstimateFlag, _ := cmd.Flags().GetBool("ai-estimate")
aiCommitLimit, _ := cmd.Flags().GetInt("ai-commit-limit")
Expand Down Expand Up @@ -471,7 +483,14 @@ func runAnalyze(cmd *cobra.Command, args []string) error {
}

aiResults := worker.RunWithProgress(ctx, repoList, concurrency, func(ctx context.Context, repo model.Repo) (*model.RepoStats, error) {
est, err := aiestimate.Estimate(ctx, commitLister, repo, aiCommitLimit)
est, partialErrs, err := aiestimate.Estimate(ctx, commitLister, repo, aiCommitLimit)
if len(partialErrs) > 0 {
diagMu.Lock()
for _, pe := range partialErrs {
diagErrors = append(diagErrors, errorEntry{Category: "ai-estimate-detail", Repo: repo.Slug, Message: pe})
}
diagMu.Unlock()
}
if err != nil {
return nil, err
}
Expand All @@ -491,7 +510,11 @@ func runAnalyze(cmd *cobra.Command, args []string) error {
// Attach AI estimates to analysis results
aiByRepo := make(map[string]*model.AIEstimate)
for _, r := range aiResults {
if r.Err == nil && r.Stats != nil && r.Stats.AIEstimate != nil {
if r.Err != nil {
diagErrors = append(diagErrors, errorEntry{Category: "ai-estimate", Repo: r.Repo.Slug, Message: r.Err.Error()})
continue
}
if r.Stats != nil && r.Stats.AIEstimate != nil {
aiByRepo[r.Repo.Slug] = r.Stats.AIEstimate
}
}
Expand Down Expand Up @@ -548,16 +571,40 @@ func runAnalyze(cmd *cobra.Command, args []string) error {
healthResults := worker.RunWithProgress(ctx, repoList, concurrency, func(ctx context.Context, repo model.Repo) (*model.RepoStats, error) {
commits, err := commitLister.ListCommits(ctx, repo, commitLimit)
if err != nil {
return nil, err
diagMu.Lock()
diagErrors = append(diagErrors, errorEntry{Category: "health", Repo: repo.Slug, Message: err.Error()})
diagMu.Unlock()
return &model.RepoStats{
Repository: repo.Slug,
Health: &model.RepoHealth{
Category: model.HealthFailed,
DaysSinceCommit: -1,
Error: err.Error(),
},
}, nil
}

h := health.ClassifyFromCommits(commits, now)

var details *model.RepoHealthDetails
if healthDetailsFlag && len(commits) > 0 {
details, err = health.AnalyzeDetails(ctx, commitLister, repo, commits, now)
var partialErrs []string
details, partialErrs, err = health.AnalyzeDetails(ctx, commitLister, repo, commits, now)
if len(partialErrs) > 0 {
diagMu.Lock()
for _, pe := range partialErrs {
diagErrors = append(diagErrors, errorEntry{Category: "health-details", Repo: repo.Slug, Message: pe})
}
diagMu.Unlock()
}
if err != nil {
return nil, err
diagMu.Lock()
diagErrors = append(diagErrors, errorEntry{Category: "health-details", Repo: repo.Slug, Message: err.Error()})
diagMu.Unlock()
return &model.RepoStats{
Repository: repo.Slug,
Health: h,
}, nil
}
}

Expand Down Expand Up @@ -648,6 +695,24 @@ func runAnalyze(cmd *cobra.Command, args []string) error {
}
}

// Write error.log if there were any diagnostic errors
if len(diagErrors) > 0 {
ext := filepath.Ext(outputPath)
errorLogPath := strings.TrimSuffix(outputPath, ext) + ".error.log"
if err := os.MkdirAll(filepath.Dir(errorLogPath), 0o755); err != nil {
return fmt.Errorf("create error log directory: %w", err)
}
f, err := os.Create(errorLogPath)
if err != nil {
return fmt.Errorf("create error log: %w", err)
}
for _, e := range diagErrors {
fmt.Fprintf(f, "[%s] %s | %s\n", e.Category, e.Repo, e.Message)
}
f.Close()
fmt.Fprintf(os.Stderr, "Error log written to %s (%d entries)\n", errorLogPath, len(diagErrors))
}

// Build report — use user/group as organization in metadata when set
reportOrg := org
if user != "" {
Expand Down
12 changes: 8 additions & 4 deletions internal/aiestimate/estimate.go
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ package aiestimate

import (
"context"
"fmt"
"sync"

"github.com/dsablic/codemium/internal/aidetect"
Expand All @@ -12,10 +13,11 @@ import (
const statsConcurrency = 10

// Estimate computes AI attribution metrics for a single repo.
func Estimate(ctx context.Context, cl provider.CommitLister, repo model.Repo, commitLimit int) (*model.AIEstimate, error) {
// It returns the estimate, a list of partial error messages (per-commit stat failures), and a fatal error.
func Estimate(ctx context.Context, cl provider.CommitLister, repo model.Repo, commitLimit int) (*model.AIEstimate, []string, error) {
commits, err := cl.ListCommits(ctx, repo, commitLimit)
if err != nil {
return nil, err
return nil, nil, err
}

est := &model.AIEstimate{
Expand Down Expand Up @@ -68,10 +70,12 @@ func Estimate(ctx context.Context, cl provider.CommitLister, repo model.Repo, co
}
wg.Wait()

var partialErrors []string
for i, fc := range flagged {
d := details[i]
if d.err != nil {
continue // partial failure — skip this commit's stats
partialErrors = append(partialErrors, fmt.Sprintf("CommitStats %s: %v", fc.info.Hash, d.err))
continue
}

est.AIAdditions += d.additions
Expand All @@ -95,5 +99,5 @@ func Estimate(ctx context.Context, cl provider.CommitLister, repo model.Repo, co
})
}

return est, nil
return est, partialErrors, nil
}
8 changes: 4 additions & 4 deletions internal/aiestimate/estimate_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ func TestEstimateBasic(t *testing.T) {
}

repo := model.Repo{Slug: "test-repo", URL: "https://github.com/org/test-repo"}
estimate, err := aiestimate.Estimate(context.Background(), mock, repo, 500)
estimate, _, err := aiestimate.Estimate(context.Background(), mock, repo, 500)
if err != nil {
t.Fatalf("Estimate: %v", err)
}
Expand Down Expand Up @@ -68,7 +68,7 @@ func TestEstimateNoAICommits(t *testing.T) {
}

repo := model.Repo{Slug: "test-repo", URL: "https://github.com/org/test-repo"}
estimate, err := aiestimate.Estimate(context.Background(), mock, repo, 500)
estimate, _, err := aiestimate.Estimate(context.Background(), mock, repo, 500)
if err != nil {
t.Fatalf("Estimate: %v", err)
}
Expand All @@ -95,7 +95,7 @@ func TestEstimatePercentCalculation(t *testing.T) {
}

repo := model.Repo{Slug: "r", URL: "https://github.com/o/r"}
est, err := aiestimate.Estimate(context.Background(), mock, repo, 500)
est, _, err := aiestimate.Estimate(context.Background(), mock, repo, 500)
if err != nil {
t.Fatalf("Estimate: %v", err)
}
Expand All @@ -116,7 +116,7 @@ func TestEstimateFirstLineMessage(t *testing.T) {
}

repo := model.Repo{Slug: "r", URL: "https://github.com/o/r"}
est, err := aiestimate.Estimate(context.Background(), mock, repo, 500)
est, _, err := aiestimate.Estimate(context.Background(), mock, repo, 500)
if err != nil {
t.Fatalf("Estimate: %v", err)
}
Expand Down
2 changes: 1 addition & 1 deletion internal/auth/gitlab.go
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ import (
// GlabCLIToken attempts to get a GitLab token from the glab CLI tool.
// Returns the token and true if successful, or empty string and false otherwise.
func GlabCLIToken() (string, bool) {
out, err := exec.Command("glab", "auth", "token").Output()
out, err := exec.Command("glab", "config", "get", "token", "--host", "gitlab.com").Output()
if err != nil {
return "", false
}
Expand Down
14 changes: 7 additions & 7 deletions internal/health/details.go
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ package health

import (
"context"
"fmt"
"strings"
"sync"
"time"
Expand All @@ -21,12 +22,13 @@ const (
)

// AnalyzeDetails performs deep health analysis on a repo's commits.
func AnalyzeDetails(ctx context.Context, lister provider.CommitLister, repo model.Repo, commits []provider.CommitInfo, now time.Time) (*model.RepoHealthDetails, error) {
// It returns the details, a list of partial error messages (e.g. per-commit stat failures), and a fatal error.
func AnalyzeDetails(ctx context.Context, lister provider.CommitLister, repo model.Repo, commits []provider.CommitInfo, now time.Time) (*model.RepoHealthDetails, []string, error) {
if len(commits) == 0 {
return &model.RepoHealthDetails{
AuthorsByWindow: map[string]int{},
ChurnByWindow: map[string]model.WindowChurnStats{},
}, nil
}, nil, nil
}

sixMoAgo := now.AddDate(0, -6, 0)
Expand Down Expand Up @@ -63,7 +65,7 @@ func AnalyzeDetails(ctx context.Context, lister provider.CommitLister, repo mode
sem := make(chan struct{}, statsConcurrency)
var wg sync.WaitGroup
var mu sync.Mutex
var firstErr error
var partialErrors []string

for i, c := range commits {
if ctx.Err() != nil {
Expand All @@ -78,9 +80,7 @@ func AnalyzeDetails(ctx context.Context, lister provider.CommitLister, repo mode
adds, dels, err := lister.CommitStats(ctx, repo, hash)
if err != nil {
mu.Lock()
if firstErr == nil {
firstErr = err
}
partialErrors = append(partialErrors, fmt.Sprintf("CommitStats %s: %v", hash, err))
mu.Unlock()
return
}
Expand Down Expand Up @@ -145,7 +145,7 @@ func AnalyzeDetails(ctx context.Context, lister provider.CommitLister, repo mode
ChurnByWindow: churnByWindow,
BusFactor: busFactor,
VelocityTrend: velocityTrend,
}, nil
}, partialErrors, nil
}

func commitWindow(date time.Time, sixMoAgo, twelveMoAgo time.Time) string {
Expand Down
14 changes: 10 additions & 4 deletions internal/health/health.go
Original file line number Diff line number Diff line change
Expand Up @@ -37,11 +37,13 @@ func Classify(lastCommitDate, now time.Time) *model.RepoHealth {
}
}

// ClassifyFromCommits classifies a repo based on a list of commits.
// It finds the most recent commit date. Returns nil if commits is empty.
func ClassifyFromCommits(commits []provider.CommitInfo, now time.Time) *model.RepoHealth {
if len(commits) == 0 {
return nil
return &model.RepoHealth{
Category: model.HealthAbandoned,
LastCommitDate: "",
DaysSinceCommit: -1,
}
}

latest := commits[0].Date
Expand All @@ -52,7 +54,11 @@ func ClassifyFromCommits(commits []provider.CommitInfo, now time.Time) *model.Re
}

if latest.IsZero() {
return nil
return &model.RepoHealth{
Category: model.HealthAbandoned,
LastCommitDate: "",
DaysSinceCommit: -1,
}
}

return Classify(latest, now)
Expand Down
Loading