diff --git a/docs/adding-a-metric.md b/docs/adding-a-metric.md index 0222876e..ec6fc8bb 100644 --- a/docs/adding-a-metric.md +++ b/docs/adding-a-metric.md @@ -51,3 +51,13 @@ Check a new metric against these decisions before you add it. - Below the cut line: clones, coverage, CRAP, temporal coupling and trends. - Keep or cut a metric on decision cases: read each top-ranked case in source and record a verdict. Correlation alone does not decide. `evaluate_metric_portfolio.py` reports support, N/A and zero rates, Spearman correlation and top-10 overlap per language and scope as input to that review. - Tune defaults with evidence. The October 2026 pilot adjudicated 729 top-10 cases on four repositories, then set the defaults (for example, `comment_noise` flags 1.0 comment line per code line or more). Its method, results and open follow-ups are in `docs/research/metric-pilot-2026-10/README.md`. Rerun a pilot with `server/scripts/evaluate_metric_portfolio.py`. + +## Activity metrics + +Git activity (`server/app/services/activity.py`) is review context beside source structure. It is not a source metric. + +- It gives two metrics: `commit_count` and `churn_lines`. Churn stays a separate metric, because one bulk rewrite gives high churn in a single commit. +- The window is 30 days, 90 days (the default), 1 year or all history. History follows renames to the file's current path. +- No `.git`, a shallow clone, no commits, a git error or a timeout gives N/A with a reason. Source metrics stay as they are. Uncommitted edits add a note only. +- Activity is laid over a finished tree when the tree is served. It is never part of the analysis cache, so a window change does not rerun the analyzers. +- Out of scope: co-change, author scores and remote fetches. diff --git a/docs/initiative-history.json b/docs/initiative-history.json index 7b4a42c1..6d42b667 100644 --- a/docs/initiative-history.json +++ b/docs/initiative-history.json @@ -138,5 +138,19 @@ "follow_ups": [ "The pilot README lists its own follow-ups, including a second reviewer and a non-author repository." ] + }, + { + "title": "Local activity as review context", + "slug": "git-activity", + "outcome": "completed", + "retired_at": "2026-11-12", + "summary": "Git activity shipped inside the metric system overhaul as commit_count and churn_lines over a chosen window, with N/A reasons when history can't be trusted, and the pilot showed it changes review priority. Reconciled against origin/main 0bfb05ba.", + "reason": "Replaced by the metric system overhaul: #76 shipped Git activity and #79 evaluated it.", + "documentation": [ + "docs/adding-a-metric.md", + "docs/research/metrics-2026-10-15/README.md" + ], + "revision": "0bfb05ba", + "follow_ups": [] } ] diff --git a/docs/intent/git-activity/implementation-plan.md b/docs/intent/git-activity/implementation-plan.md deleted file mode 100644 index 7877a57c..00000000 --- a/docs/intent/git-activity/implementation-plan.md +++ /dev/null @@ -1,47 +0,0 @@ -# Local activity — implementation plan - -## Plan at a glance - -Verify bounded history extraction before any UI or combined score. Then compare useful decisions on real repositories. Source scanning stays independent throughout. - -## Implementation strategy - -Depend on metric trust's source and availability contract. Reuse repository-root discovery in `analysis.py`, but keep history in a small separate service, proposed as `server/app/services/git_activity.py`. Its input is a root, revision, window and explicit traversal policy. Its output is file frequency, commit evidence, completeness and dirty-source state. - -The first fixture uses saved Git output. Proposed focused loop: `cd server && uv run pytest tests/test_git_activity.py`. Add temporary real Git repositories to the same tests for protocol fidelity. No network or credentials are needed. Inject missing Git, shallow objects, malformed output and a timeout. - -The real local Git binary is the only new material dependency. Use argument arrays, NUL-delimited paths and the official `git log` contract. `--follow` applies to one file, so do not assume it solves repository-wide renames. Use a bounded batch with an explicit initial rename policy. [Git manual](https://git-scm.com/docs/git-log). - -Bypass activity with one option. Revert its integration commit to remove it. No mixed-version API or database is required. - -## Milestone 1: File histories produce correct, inspectable counts - -Parse saved `git log` output into a human-readable table with the exact window, HEAD, traversal policy and supporting commits. Then run the same cases against temporary real repositories with whitespace/non-ASCII paths, rename, merge and formatting commits. Choose and document whether merge changes are excluded; do not count the same change twice. - -Return explicit incomplete status for shallow history and unavailable status without Git. Pin history to HEAD, flag dirty source, and set a bounded runtime limit. Report truncation instead of silently treating a partial pass as complete. Do not add churn to frequency as a second vote. Folder frequency uses the union of touching commits rather than summed file counts. - -### Desired end state - -- A local table reconciles manually with Git for all fixture histories. -- Saved-output and real-Git tests establish both counting and protocol behavior. -- Source scanning succeeds when history is absent or disabled. - -## Milestone 2: Activity changes justified review priorities - -Using the portfolio protocol, verify ten file histories in each of three authorized repositories. Compare source-only and activity-assisted queues. Start with a 90-day window and inspect 180-day sensitivity; record these as provisional choices. - -Add an optional activity column and supporting-commit detail to `Explorer.tsx` and report output only after the extraction gate passes. A two-axis view may follow using existing visualization primitives; avoid a new default aggregate. Keep history configuration and source measurement manifests distinct. - -Default visibility requires useful new decisions in at least two repositories and acceptable local cost. Record runtime and memory against source-only scanning; propose a budget from observed pilot cost before enabling it by default. If no useful decisions appear, retain an optional report or remove the prototype. - -### Desired end state - -- Every displayed count opens the commits and policy that support it. -- No-history and dirty-source cases remain explicit in UI and reports. -- Focused tests, local browser use, client build and reporting tests verify the optional path. - -## Below the cut line - -- Function archaeology, remote fetch, author scores and ticket integration. -- Co-change until frequency earns its cost. A later proof must show a hidden peer, an expected test pair, and a bulk-change false positive. -- Universal churn thresholds or a combined health score. diff --git a/docs/intent/git-activity/initiative-map.json b/docs/intent/git-activity/initiative-map.json deleted file mode 100644 index 84aa2491..00000000 --- a/docs/intent/git-activity/initiative-map.json +++ /dev/null @@ -1,261 +0,0 @@ -{ - "schema_version": "0.5.0", - "initiative": { - "title": "Local activity as review context", - "slug": "git-activity", - "scale": "capability", - "current_phase": "plan", - "last_updated": "2026-10-15", - "summary": "Test bounded file commit frequency as a distinct prioritization input beside source structure.", - "status": "planned", - "next_action": "After metric-trust scope/support is available, prove saved history parsing and real Git fixture fidelity." - }, - "artifacts": { - "intent_brief": { - "path": "intent-brief.md", - "status": "current", - "updated_at": "2026-10-15" - }, - "brainstorm": { - "path": "brainstorm.md", - "status": "not-requested", - "updated_at": null - }, - "shape_brief": { - "path": "shape-brief.md", - "status": "current", - "updated_at": "2026-10-15" - }, - "implementation_plan": { - "path": "implementation-plan.md", - "status": "current", - "updated_at": "2026-10-15" - } - }, - "sources": [ - { - "id": "user", - "type": "conversation", - "location": "Current Codex chat", - "anchor": "2026-10-15 original unique-information request and full-process approval", - "authority": "user" - }, - { - "id": "research", - "type": "research", - "location": "docs/research/metrics-2026-10-15/originals/recommendations.md", - "anchor": "R10", - "authority": "external proposals checked against baseline; not product intent" - }, - { - "id": "repo", - "type": "repository", - "location": "server/app/services/analysis.py; server/app/models.py; server/app/services/reporting.py; client/src/components/Explorer.tsx", - "anchor": "83dcbf40879ca06ab07cdfcf365b5c0ccb7608a4", - "authority": "current source" - } - ], - "intent": { - "claims": [ - { - "id": "C1", - "statement": "Provide unique actionable information per metric.", - "kind": "outcome", - "commitment": "explicit", - "importance": "core", - "status": "active", - "time_horizon": "first release", - "sources": [ - "user" - ], - "confidence": "high" - }, - { - "id": "C2", - "statement": "Avoid several highly correlated metrics that add little information.", - "kind": "anti-goal", - "commitment": "explicit", - "importance": "core", - "status": "active", - "time_horizon": "first release", - "sources": [ - "user" - ], - "confidence": "high" - }, - { - "id": "C3", - "statement": "Review current metrics and the field including CRAP before proposing initiatives.", - "kind": "sequence", - "commitment": "explicit", - "importance": "core", - "status": "active", - "time_horizon": "proof", - "sources": [ - "user" - ], - "confidence": "high" - }, - { - "id": "C4", - "statement": "Research and planning are authorized; application implementation remains planned.", - "kind": "constraint", - "commitment": "explicit", - "importance": "core", - "status": "active", - "time_horizon": "proof", - "sources": [ - "user" - ], - "confidence": "high" - }, - { - "id": "C5", - "statement": "Test bounded file commit frequency as a distinct prioritization input beside source structure.", - "kind": "implementation hypothesis", - "commitment": "derived", - "importance": "supporting", - "status": "active", - "time_horizon": "proof", - "sources": [ - "research", - "repo" - ], - "confidence": "medium" - } - ], - "themes": [], - "conflicts": [], - "human_questions": [] - }, - "brainstorm": { - "status": "not-requested", - "requested_count": 7, - "options": [], - "selected_option": null - }, - "shape": { - "status": "current", - "appetite": "Small isolated local proofs; no enterprise infrastructure or compatibility layers.", - "problem": "Test bounded file commit frequency as a distinct prioritization input beside source structure.", - "recommended_shape": "Local activity as review context", - "success_moves": [ - "Parse saved Git output before adding UI.", - "Run the same cases against temporary real repositories.", - "Keep frequency and churn separate rather than adding correlated votes.", - "Keep activity optional and do not fetch missing history." - ], - "first_proof": { - "question": "Can the tool explain file frequency and history completeness under a bounded policy?", - "artifact": "History table and supporting commit list from temporary Git cases.", - "decision_rule": "Block UI integration if counts, rename/merge policy or shallow-history completeness cannot be explained." - }, - "rabbit_holes": [], - "open_decisions": [] - }, - "implementation": { - "status": "current", - "repository_snapshot": { - "root": "/Users/byronwall/.codex/worktrees/46c0/srcly", - "revision": "83dcbf40879ca06ab07cdfcf365b5c0ccb7608a4", - "worktree_status": "Research/planning documents only; no application implementation.", - "subsystems": [ - "server/app/services/analysis.py", - "server/app/models.py", - "server/app/services/reporting.py", - "client/src/components/Explorer.tsx" - ], - "test_commands": [ - "cd server && uv run pytest tests/test_git_activity.py", - "cd client && pnpm test", - "cd client && pnpm build" - ], - "deployment_surfaces": [] - }, - "dependencies": [ - { - "id": "git", - "name": "Local Git history", - "material": true, - "application_seam": "Proposed git_activity.py bounded history input/output", - "first_proof_role": { - "mode": "local-substitute", - "rationale": "Saved NUL-delimited output isolates counting from local Git protocol." - }, - "failure_modes": [ - "Git unavailable", - "shallow history", - "malformed paths/output", - "timeout" - ], - "proof_rungs": [ - "local-fixture", - "contract-test", - "real-local-output" - ], - "local_proof": "Saved output fixtures followed by temporary real Git repositories.", - "live_proof": "Read-only smoke on three authorized local histories; no credentials, fetch or service.", - "rollout": "Optional input or coherent commit; omit/revert without parallel API support." - } - ], - "fast_local_loop": { - "command": "cd server && uv run pytest tests/test_git_activity.py", - "network_required": false, - "shared_infrastructure_required": false, - "failure_injection": [ - "Git unavailable", - "shallow history", - "malformed paths/output", - "timeout" - ], - "status": "Proposed commands for implementation; new scripts/tests do not exist yet." - }, - "milestones": [ - { - "id": "m1", - "title": "File histories produce correct, inspectable counts", - "outcome": "History counts reconcile with real Git under explicit traversal and completeness rules.", - "dependencies_introduced": [], - "proof": "Saved-output plus real temporary Git cases: merges, renames, bulk changes, unusual paths, shallow and missing history.", - "valid_intermediate_state": "History counts reconcile with real Git under explicit traversal and completeness rules. Existing source-only scanning remains usable.", - "rollback": "Revert the scoped milestone commit or omit optional evidence; no stored-data migration.", - "status": "planned", - "evidence": [] - }, - { - "id": "m2", - "title": "Activity changes justified review priorities", - "outcome": "An optional view adds useful prioritization in two pilot repositories or is rejected.", - "dependencies_introduced": [], - "proof": "Ten manually verified histories per repository; source-only versus activity-assisted decision review and runtime accounting.", - "valid_intermediate_state": "An optional view adds useful prioritization in two pilot repositories or is rejected. Existing source-only scanning remains usable.", - "rollback": "Revert the scoped milestone commit or omit optional evidence; no stored-data migration.", - "status": "planned", - "evidence": [] - } - ], - "cut_line": [ - "Function archaeology, remote fetch, author scores and ticket integration.", - "Co-change until frequency earns its cost. A later proof must show a hidden peer, an expected test pair, and a bulk-change false positive.", - "Universal churn thresholds or a combined health score." - ], - "open_decisions": [], - "initiative_dependencies": [ - { - "initiative": "metric-trust", - "gate": "milestone 1: source identity and availability" - }, - { - "initiative": "metric-portfolio", - "gate": "Reuse evaluation protocol; candidate prototypes can precede its final catalog choice." - } - ] - }, - "history": [ - { - "date": "2026-10-15", - "change": "Created planned initiative from collected Pro research and independent source checks. Application proofs remain unrun." - } - ], - "closure": null -} diff --git a/docs/intent/git-activity/intent-brief.md b/docs/intent/git-activity/intent-brief.md deleted file mode 100644 index b3e757c7..00000000 --- a/docs/intent/git-activity/intent-brief.md +++ /dev/null @@ -1,32 +0,0 @@ -# Local activity as review context - -## My read - -Byron wants information that static structure cannot supply. Two equally complex files may deserve different attention when one changes often and the other remains stable. Git activity can help decide what to inspect now, while complexity helps explain what may be difficult to change. - -This initiative tests that distinction with bounded file histories. It should connect an activity count to the commits that produced it. A reviewer can then distinguish recurring changes from a one-time format update, merge convention, or generated file. Activity describes attention and change, not poor quality by itself. - -The audience is a developer using Srcly against a local repository. The tool should remain useful without Git history. History analysis must be optional and separately observable, so a normal source scan does not acquire an unexplained cost or a hidden external dependency. - -## What matters most - -- Add a different prioritization question rather than another size proxy. -- Show the history window, source revision and supporting commits. -- Explain merge, rename, shallow-history and bulk-change limitations. -- Keep change frequency separate from changed-line volume. - -## Boundaries - -Start with file-level commit frequency in a declared window. A proposed 90-day default and 180-day comparison are reversible study settings, not user requirements. Do not reconstruct function history, infer developer ability, connect ticket systems, or introduce a hosted service. - -Use only explicitly selected local repositories. Do not fetch missing history automatically. A shallow or unavailable history must remain unknown rather than displaying zero changes. Dirty source must be identified because its metrics may differ from the history revision. - -## What seems settled - -Filesystem modification time already exists but cannot substitute for Git activity. The research identifies activity as the first additional source to test after trustworthy structure. Its unique value has not been measured in Srcly. - -The portfolio evaluation supplies a review method. A small activity proof can use that method without waiting for every portfolio candidate. Broader co-change analysis depends on this proof earning its cost first. - -## Next step after confirmation - -Produce a local history table for hand-checkable cases, then compare activity-assisted and source-only decisions. The [shape](shape-brief.md) and [implementation plan](implementation-plan.md) retain a rejection path if the history adds no useful prioritization. All implementation remains planned. diff --git a/docs/intent/git-activity/shape-brief.md b/docs/intent/git-activity/shape-brief.md deleted file mode 100644 index c0d5f0c1..00000000 --- a/docs/intent/git-activity/shape-brief.md +++ /dev/null @@ -1,39 +0,0 @@ -# Local activity — shape - -**Outcome:** A reviewer can see why a structurally difficult file deserves attention now. - -**Primary flow:** Enable local history → choose a window → inspect activity beside complexity → open supporting commits. - -## Feature scope - -```text -Local activity context -├── PLANNED ADDITIONS -│ ├── File change frequency -│ │ ├── Window, revision, merge and rename policy visible -│ │ └── Supporting commit list; churn shown separately -│ ├── Comparison with source structure -│ │ └── Activity and complexity shown as distinct axes -│ └── Honest support -│ └── No Git, shallow history and dirty source clearly identified -└── EVALUATE BEFORE COMMITTING - └── Default visibility after distinct decision value is demonstrated -``` - -## Decisions and boundaries - -**Appetite:** One bounded local Git pass and a file-level review flow. - -**Key decision:** Read a declared history window and verify counts before adding a hotspot formula. A frequently changed file can be important and healthy. - -**Boundary:** No fetch, people scoring, background service or automatic refactoring advice. Keep the source-only path available by disabling activity. Cache against HEAD and policy; do not invalidate all AST measurements when only the window changes. - -**First proof:** A history table whose counts can be checked against Git. - -**Try:** Use a small local repository containing renames, a merge, a bulk change, and a shallow-history case. - -**Observe:** Compare every count and supporting commit with the declared traversal policy. - -**Decide:** Stop integration if history completeness or counts cannot be explained. Default visibility needs useful new decisions in two pilot repositories. - -See the [implementation plan](implementation-plan.md). diff --git a/docs/research/metrics-2026-10-15/README.md b/docs/research/metrics-2026-10-15/README.md index db12cd01..8b08c65a 100644 --- a/docs/research/metrics-2026-10-15/README.md +++ b/docs/research/metrics-2026-10-15/README.md @@ -10,7 +10,7 @@ The research covers 32 API fields, 27 report definitions, 22 selectable metrics, | --- | --- | --- | | [Metric trust](https://github.com/byronwall/srcly/blob/0bfb05ba/docs/intent/metric-trust/shape-brief.md) | Correct scope, units, support, rollups, and ranking | A measured zero differs from unavailable data in one real scan and report | | [Portfolio evaluation](https://github.com/byronwall/srcly/blob/0bfb05ba/docs/intent/metric-portfolio/shape-brief.md) | Evidence for a compact default and useful diagnostics | Three pinned repositories reveal retained and rejected decision cases | -| [Git activity](../../intent/git-activity/shape-brief.md) | Explain why difficult code deserves attention now | Verified file histories change a justified priority decision | +| [Git activity](https://github.com/byronwall/srcly/blob/0bfb05ba/docs/intent/git-activity/shape-brief.md) | Explain why difficult code deserves attention now | Verified file histories change a justified priority decision | | [Imported quality evidence](../../intent/imported-quality-evidence/shape-brief.md) | Inspect cycles, clones, diagnostics, and test gaps | An existing graph edge leads to its original source declaration | These are planned initiatives, with implementation plans and validated maps. This PR implements none of them. The first investment is metric trust and a small portfolio pilot. The other initiatives define independently gated next steps.