feat(security): add `repowise security scan --history` for full git-history secret/risk scanning

[Ayush7614]

Summary

Add a repowise security scan --history command that runs the existing SecurityScanner (packages/core/src/repowise/core/analysis/security_scan.py) over the entire git history of a repo, not just the working tree, and persists hits into the existing security_findings table.

Today SecurityScanner only runs during indexing (packages/core/src/repowise/core/pipeline/persist.py) against the current checked-out files. Secrets, tokens, and dangerous patterns committed in the past — even if later removed — are never surfaced. A full-history mode closes that gap from inside the product (distinct from the CI secret-scanning PRs, which operate on diffs/CI gates).

Motivation

• Leaked credentials often live in old commits and are "fixed" by deleting them in a later commit. They remain in history and remain a risk.
• The pattern registry, severity model, and DB persistence already exist; only the scan target (working tree vs. full history) is missing.
• Surfacing these in the same security_findings table means the existing API (GET /api/repos/{id}/security) and UI (findings table, severity×directory matrix) light up automatically with no new storage.

Proposed design

1. New CLI subcommand security scan --history [--since <rev>] [--to <rev>] in packages/cli/src/repowise/cli/commands/ that:
   • Iterates commits in the range (default git rev-list --all or configurable).
   • For each commit, checks out the tree's blob content (or uses git show <rev>:<path>) for tracked source files.
   • Applies the existing _PATTERNS / _SYMBOL_KEYWORDS checks via SecurityScanner.scan_file.
   • Persists findings with the commit SHA + author date attached (extend the security_findings schema with commit_sha / commit_at columns, nullable for working-tree scans).
2. Reuse, don't fork: keep SecurityScanner's regex registry and persist() logic; add an optional commit_sha/commit_at field rather than a parallel implementation.
3. Dedup + severity rollup: same (repo_id, file_path, kind, line, commit_sha) should not be double-inserted on re-runs; provide a clear summary (counts by severity, by kind) on stdout and JSON --output.
4. UI/API: show a "found in history" badge/filter so users can tell working-tree findings from historical ones, and optionally emit remediation guidance (rotate, purge via filter-repo/BFG).

Out of scope (to avoid duplicating active work)

• CI gating / pre-commit / pre-push secret scanning (covered by open PRs Auto-configure MCP and write .claude/CLAUDE.md during repowise init #7, Feat UI beautification #9, feat: output language support for wiki generation #99, feat: JVM type-ref resolution and annotation-processor symbol synthesis #274, feat(vscode): sidebar home dashboard with freshness and theme switcher #650, [Bug] Stuck at parsing files #678, fix(mcp,claude-md): serve curated entry points, cap tour barrels #418, etc.).
• This is a local, developer-facing, full-history exploration tool that complements those gates.

Acceptance criteria

• repowise security scan --history scans all historical revisions of tracked source files.
• Findings persist to security_findings with commit provenance and appear in the existing security API/UI.
• Re-runs are idempotent (no duplicate rows).
• --since/--to range limiting and --output json are supported.
• Unit tests cover history iteration + dedup, mirroring existing security_scan test patterns.

References

• Existing scanner: packages/core/src/repowise/core/analysis/security_scan.py
• Wiring: packages/core/src/repowise/core/pipeline/persist.py:633
• API: packages/server/src/repowise/server/routers/security.py
• Related (different scope — CI gates): PRs Auto-configure MCP and write .claude/CLAUDE.md during repowise init #7, Feat UI beautification #9, feat: output language support for wiki generation #99, feat: JVM type-ref resolution and annotation-processor symbol synthesis #274, feat(vscode): sidebar home dashboard with freshness and theme switcher #650, [Bug] Stuck at parsing files #678, fix(mcp,claude-md): serve curated entry points, cap tour barrels #418

[RaghavChamadiya]

Strong +1 on the framing. I traced the flow and the reuse instinct is right: the scanner, the security_findings table (migration 0011), the API, and the UI all exist, so this is a new scan target rather than a new subsystem. A few grounded caveats before someone picks it up.

1. Reusing persist() won't give you idempotency for free. It does a blind INSERT with no ON CONFLICT, and the table has no unique constraint (only the non-unique ix_security_findings_repo_file index). Re-runs will duplicate. It also swallows exceptions with break, so it stops at the first failing row silently. Meeting the "no duplicate rows" criterion means adding a unique key plus ON CONFLICT DO NOTHING, which touches persist() and needs a migration anyway. So this is closer to "extend persist()" than "reuse untouched."

2. Schema change is mandatory. No commit_sha/commit_at columns today, and the INSERT in persist() hardcodes its column list. This needs a 0012 migration (nullable provenance columns plus the dedup constraint) and an edit to that statement.

3. The symbol scan doesn't port to history. scan_file takes symbols from parsed_files, which come out of the ingestion parse phase. A raw historical blob has no resolved symbols, so _SYMBOL_KEYWORDS is effectively dead in history mode unless you re-parse every blob, which is expensive. The _PATTERNS regex path is the part that ports cleanly.

4. Set expectations on what gets flagged. Most of the 11 _PATTERNS are code smells (eval_call, os_system, weak_hash, fstring_sql) rather than leaked credentials. Running those across all of history tends to produce a lot of "os.system in a two-year-old commit" noise with little to act on. The history-relevant subset is really hardcoded_password / hardcoded_secret, and even those are naive regexes with no entropy check versus what gitleaks or trufflehog do. Might be worth gating history mode to the secret-oriented patterns, and positioning this as complementary to a real secret scanner rather than a replacement.

One design suggestion. git rev-list --all times tracked files is O(commits x files) and re-scans identical unchanged content thousands of times. Scanning unique blobs instead is dramatically cheaper: enumerate objects once (git rev-list --objects --all, dedup by blob SHA), scan each blob's content a single time, then attribute hits to the first-introducing commit for provenance. Git already dedups content by blob, so you get natural dedup and a big speedup for free.

Happy to review a PR that goes this route.

[Ayush7614]

Thanks for the detailed review and thoughtful suggestions @RaghavChamadiya I opened PR #821 implementing the feature and incorporated the key points around schema changes, commit provenance, idempotent persistence, and reuse of the existing scanner. I also kept the implementation aligned with the current architecture. I'd appreciate a review when you have a chance, and I'm happy to iterate further based on your feedback.