Skip to content

feat(spec): opt-in AI usage tracking for spec new#20

Merged
skapoor8 merged 1 commit into
mainfrom
spec-new-usage-tracking
Jul 9, 2026
Merged

feat(spec): opt-in AI usage tracking for spec new#20
skapoor8 merged 1 commit into
mainfrom
spec-new-usage-tracking

Conversation

@skapoor8

@skapoor8 skapoor8 commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Changes

  • Add --usage flag to /spec new. AI usage tracking runs if and only if the flag is passed; without it, nothing is tracked and no usage file is written.
  • Write usage telemetry to a separate ai-usage.md in the plan dir. It never mixes into plan.md, user-guide.md, or implementation files.
  • Prefer a background subagent to monitor the session file, so tracking never delays planning. The current agent runs the same tally as a fallback when a subagent cannot be launched.
  • Session-file discovery is agent-agnostic. Claude Code JSONL is parsed first-class (token totals + tool-call counts); other agents are probed best-effort.
  • Degrade gracefully: when no session file can be located or parsed, write a "telemetry unavailable" variant instead of failing the plan or emitting fake zeros.

New references: skills/spec/references/usage-monitor.md (Steps A-C: locate, aggregate, write) and skills/spec/references/ai-usage-template.md (the ai-usage.md layout, measured and unavailable variants). new.md wires the monitor at Step 5.2u (launch) and Step 6 (finalize/block).

Validation

  • /spec new <objective> without --usage produces no ai-usage.md
  • /spec new --usage <objective> writes {PLAN_DIR}/ai-usage.md with token and tool tables
  • Unavailable path writes the "telemetry unavailable" variant rather than erroring
  • Subagent monitor runs in background and does not delay plan output; current-agent fallback works when no subagent is available

Future Work

  • Broaden non-Claude-Code session-file probes (Cursor, OpenCode, Aider paths are placeholders) as those transcript formats are confirmed.

@skapoor8 skapoor8 self-assigned this Jul 9, 2026
Comment thread skills/spec/references/usage-monitor.md Outdated
Comment thread skills/spec/references/usage-monitor.md Outdated
Comment thread skills/spec/references/usage-monitor.md
Comment thread skills/spec/references/ai-usage-template.md Outdated
Gate AI usage tracking behind a new --usage flag on /spec new. When
set, a background subagent monitors the executing agent's session
files and writes token/tool telemetry to ai-usage.md inside the plan
directory. Behaviour is byte-for-byte unchanged when the flag is
absent.

**Wiring**
- Add USAGE_MODE parsing plus monitor and finalize steps to the new
  workflow.
- Add ai-usage-template.md and usage-monitor.md references.
- Document --usage in SKILL.md.

**Session detection and aggregation**
- Select the cwd-matching session file (skip the subagent's own
  transcript) so a background monitor measures the parent planning
  session, not itself.
- Bound token and tool aggregation to USAGE_START_EPOCH via each
  line's timestamp, matching the advertised planning window.
- Guard the best-effort other-agent probe with nullglob/globstar so
  placeholder globs expand instead of staying literal.
- Support Claude Code as a first-class source and other agents on a
  best-effort basis with graceful degradation.

**Reporting**
- Relabel the usage Total row "Total (all classes)" and clarify it is
  a tokens-processed record, not a cost figure.
@skapoor8 skapoor8 force-pushed the spec-new-usage-tracking branch from 0b6d6bb to 1cd972e Compare July 9, 2026 06:25
@skapoor8 skapoor8 marked this pull request as ready for review July 9, 2026 06:26
@skapoor8 skapoor8 merged commit 7641de5 into main Jul 9, 2026
2 checks passed
@skapoor8 skapoor8 deleted the spec-new-usage-tracking branch July 9, 2026 06:27
github-actions Bot pushed a commit that referenced this pull request Jul 9, 2026
## [1.57.0](v1.56.0...v1.57.0) (2026-07-09)

### Features

* **spec:** add opt-in AI usage tracking to spec new ([#20](#20))

Gate AI usage tracking behind a new --usage flag on /spec new. When
set, a background subagent monitors the executing agent's session
files and writes token/tool telemetry to ai-usage.md inside the plan
directory. Behaviour is byte-for-byte unchanged when the flag is
absent.

**Wiring**
- Add USAGE_MODE parsing plus monitor and finalize steps to the new
  workflow.
- Add ai-usage-template.md and usage-monitor.md references.
- Document --usage in SKILL.md.

**Session detection and aggregation**
- Select the cwd-matching session file (skip the subagent's own
  transcript) so a background monitor measures the parent planning
  session, not itself.
- Bound token and tool aggregation to USAGE_START_EPOCH via each
  line's timestamp, matching the advertised planning window.
- Guard the best-effort other-agent probe with nullglob/globstar so
  placeholder globs expand instead of staying literal.
- Support Claude Code as a first-class source and other agents on a
  best-effort basis with graceful degradation.

**Reporting**
- Relabel the usage Total row "Total (all classes)" and clarify it is
  a tokens-processed record, not a cost figure.
@github-actions

github-actions Bot commented Jul 9, 2026

Copy link
Copy Markdown

🎉 This PR is included in version 1.57.0 🎉

The release is available on GitHub release

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant