fix(parser): implement request dedup and deltaize for token accuracy

CHANGELOG.md

@@ -1,5 +1,26 @@
 # Changelog
 
+## [2.0.0-beta.3] - 2026-04-13
+
+### Fixed
+- **Token & cost accuracy** — Claude Code logs the same API response in multiple places (streaming chunks, sub-agent mirrors, `/compact` retries). CCMeter now dedupes by `requestId` (Anthropic's billing unit), eliminating the 2–3× over-counting previously observed on days with heavy sub-agent activity. Totals now match what Anthropic actually billed.
+- **Multi-minute timeline accuracy** — long streaming responses (extended thinking + large outputs) now correctly distribute their tokens across the minutes they actually spanned, instead of collapsing onto the final completion minute. `active_minutes` clustering, the minute-level heatmap, and rate-limit forecasts are all more accurate.
+- **Code metrics on partial-overlap streams** — when a non-canonical stream carries a unique `Edit`/`Write` block, its `lines_suggested` / `lines_added` / `lines_deleted` are preserved via zero-billing ghost markers, avoiding silent under-counting of code activity. Ghosts are deduped across multiple mirror files by timestamp so a 3-file (canonical + 2 mirrors) layout doesn't double-count line metrics.
+- **Ghost events no longer leak into model breakdowns** — zero-billing markers now carry an empty `model` field, so they fall through to `ModelId::Other` and are filtered out of model-share aggregations instead of producing phantom slices.
+- **User-side patch dedup** — patches replayed into sub-agent transcripts (Edit/Write acceptances) are now deduped by line `uuid`, fixing inflated `lines_added` and skewed efficiency scores on sub-agent–heavy days.
+- **Cost fallback includes `cache_creation`** — token-based cost estimation (used when raw `costUSD` is absent, i.e. Pro plans) now bills `cache_creation` at `input_price × 1.25` instead of ignoring it. Closes a 5–15 % under-estimate on cache-heavy sessions.
+
+### Added
+- **Versioned cache schema** with automatic invalidation. `~/.config/ccmeter/history.json` carries a `schema_version`; mismatches trigger a clean rebuild on next launch so accuracy fixes propagate without manual intervention.
+- **One-time cache-state banner** at the top of the dashboard:
+  - "Cache rebuilt" (warning color) when a schema migration occurs.
+  - "Cache was unreadable" (error color) when the on-disk file couldn't be read or parsed, with a hint to delete it if the issue persists.
+  - Both dismiss on any keypress.
+- **`CCMETER_FORCE_BANNER` env var** for testing the banners after migration has already happened. Set to `recovered` for the corruption banner, anything else for the migration banner.
+
+### Documentation
+- README: new "Accurate token counting" section explaining the dedup methodology and what users should expect when upgrading from a pre-dedup version.
+
 ## [2.0.0-beta.2] - 2026-04-13
 
 ### Added

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "ccmeter"
-version = "2.0.0-beta.2"
+version = "2.0.0-beta.3"
 edition = "2024"
 description = "A meter for Claude Code usage"
 repository = "https://github.com/hmenzagh/CCMeter"

README.md

@@ -45,6 +45,13 @@ CCMeter reads your local Claude Code session data and renders an interactive TUI
 - **Per-project detail** — dedicated charts, model distribution, cost sparklines, and estimated active time
 - **Time filters** — 1h, 12h, Today, Last week, Last month, All
 
+**Rate limit tracking** (press `` ` `` to toggle)
+- **Live usage monitor** — polls `/api/oauth/usage` for each Claude OAuth account and shows 5h, 7d, Opus, Sonnet, and Cowork window utilization in real time
+- **Credential cards** — one per source root with subscription tier, expiry, and current usage bars
+- **Session forecast** — extrapolates when you'll hit each rate limit window based on current token velocity
+- **Session chart & timeline** — historical rate-limit hits per source, plus minute-level token usage over the active window
+- **Overage tracking** — surfaces `extra_usage` credits and monthly limits when enabled
+
 **Project handling & performance**
 - **Auto-discovery & grouping** — finds Claude projects and groups them by git repository
 - **Multi-source roots** — switch between Claude config directories with `Shift+Tab`
@@ -95,8 +102,9 @@ ccmeter
 |-----|--------|
 | `Tab` | Cycle time filter |
 | `Shift+Tab` | Switch source root |
+| `` ` `` | Toggle rate limit tracking view |
 | `j` / `k` or `Up` / `Down` | Scroll projects |
-| `h` / `l` or `Left` / `Right` | Navigate between projects |
+| `h` / `l` or `Left` / `Right` | Navigate between projects (or credentials in rate tracking) |
 | `Esc` | Deselect project |
 | `.` | Open settings panel |
 | `r` | Reload data |
@@ -120,9 +128,21 @@ CCMeter discovers Claude Code sessions by scanning your home directory for any f
 Session JSONL → parallel parse → daily aggregates → cached history → TUI render
 ```
 
+### Accurate token counting
+
+Claude Code logs the same API response in several places — once per streaming chunk, again in every sub-agent's transcript, again in any `/compact` retry. Summing every line naïvely inflates tokens by 2–3× on busy days.
+
+CCMeter dedupes by `requestId` (Anthropic's billing unit: one request = one invoice line). For each request it picks the most complete log, then re-emits it as per-chunk deltas so multi-minute streams keep their real timestamps. Activity from non-canonical mirror logs survives as zero-billing "ghost" markers, so `active_minutes` and code metrics stay accurate even when the canonical log is just a terminal snapshot. User-side patches (Edit/Write acceptances) are deduped by line `uuid` for the same reason.
+
+Result: totals match what Anthropic actually billed, and the minute-level timeline reflects real activity.
+
 ### Cache
 
-Parsed metrics are persisted to `~/.config/ccmeter/history.json`. On subsequent launches, only new or modified session files are parsed, everything else is served from cache, making startup near-instant even with thousands of sessions.
+Parsed metrics are persisted to `~/.config/ccmeter/history.json` so subsequent launches only re-parse new or modified session files.
+
+The cache is versioned. When CCMeter ships a change that would make pre-existing aggregates wrong (e.g. the dedup work above), the schema is bumped and any older cache is rebuilt from scratch on next launch — a one-time banner explains why historical totals may have shifted. A second banner appears if the cache was unreadable (corruption, disk error). Both dismiss on any keypress.
+
+**Upgrading from before the dedup fix?** Expect your historical numbers to drop on days with heavy sub-agent or `/compact` activity. That's the over-counting going away, not data loss.
 
 ### Per-project view
 
@@ -139,6 +159,24 @@ Press `Esc` to go back to the global overview.
   <img src="assets/project.png" alt="CCMeter per-project view" />
 </p>
 
+### Rate limit tracking
+
+Press `` ` `` (backtick) to switch to the rate limit tracking view. CCMeter reads your Claude OAuth credentials and polls `/api/oauth/usage` at randomized intervals (5–10 min per account) to show real-time utilization of each rate limit window.
+
+- **Credential cards** — one per source root, with subscription tier, token expiry, and usage bars for the 5h, 7d, Opus, Sonnet, and Cowork windows
+- **Live summary & KPI bar** — currently selected account's status at a glance
+- **Session forecast** — projects when you'll hit each limit based on recent token velocity
+- **Usage timeline & session chart** — minute-level token usage for the active 5h window and historical rate-limit hits per source
+- **Overages** — surfaces `extra_usage` credits and monthly limits when enabled on your plan
+
+Navigate between accounts with `←` / `→` (or `h` / `l`), refresh with `r`, and press `` ` `` again to return to the main dashboard.
+
+CCMeter only sees tokens from Claude Code sessions (local JSONL logs). Tokens consumed through the Claude chat (claude.ai web/desktop) count against the same rate limits but are not visible to CCMeter, so forecasts and utilization bars may under-report actual usage when you also chat with Claude alongside coding. Rate limit history is persisted locally at `~/.config/ccmeter/rate-history.json` and `~/.config/ccmeter/usage-hit-history.json` so session charts and hit timelines survive restarts — delete these files to reset the tracking history.
+
+<p align="center">
+  <img src="assets/rate-tracking.png" alt="CCMeter rate limit tracking view" />
+</p>
+
 ## Configuration
 
 User overrides are stored at `~/.config/ccmeter/overrides.json` and can be edited through the settings panel or manually.
@@ -160,28 +198,31 @@ User overrides are stored at `~/.config/ccmeter/overrides.json` and can be edite
 ```
 src/
 ├── main.rs               # Entry point & event loop
-├── app.rs                # Core application state
+├── app.rs                # Core application state & view routing
+├── update_check.rs       # GitHub release version check
 ├── config/
-│   ├── mod.rs
 │   ├── discovery.rs      # Project auto-discovery
-│   └── overrides.rs      # User configuration & merges
+│   ├── overrides.rs      # User configuration & merges
+│   └── settings.rs       # Persistent user preferences
 ├── data/
-│   ├── mod.rs
 │   ├── parser.rs         # JSONL session parsing
 │   ├── cache.rs          # Persistent metric cache
+│   ├── index.rs          # Compact event index
 │   ├── tokens.rs         # Daily token aggregation
-│   └── models.rs         # Model pricing tables
+│   ├── models.rs         # Model pricing tables
+│   ├── oauth.rs          # OAuth credential loading & async usage polling
+│   ├── rate_limits.rs    # Rate-limit hit parsing
+│   ├── rate_history.rs   # Persisted rate-limit history
+│   └── hit_history.rs    # Historical hit aggregation
 └── ui/
-    ├── mod.rs
     ├── dashboard.rs      # Main layout
     ├── heatmap.rs        # Heatmap rendering
+    ├── loading.rs        # Startup loading screen
     ├── theme.rs          # Color theme
     ├── time_filter.rs    # Time range logic
     ├── settings_view.rs  # Settings panel
-    └── cards/
-        ├── mod.rs
-        ├── data.rs       # Card data aggregation
-        └── render.rs     # Card rendering
+    ├── cards/            # Per-project cards
+    └── rate_tracking/    # Rate limit tracking view (13 modules)
 ```
 
 ## License

src/app.rs

@@ -147,6 +147,12 @@ pub(crate) struct App {
 
     pub(crate) update_info: Option<UpdateInfo>,
     update_rx: mpsc::Receiver<UpdateInfo>,
+
+    /// Outcome of the initial on-disk cache load. `Migrated` and
+    /// `Recovered` both trigger a one-time banner but with distinct
+    /// messages so the user can tell a planned schema bump apart from a
+    /// corrupted cache. Dismissed on the first user keypress.
+    pub(crate) cache_load_state: cache::CacheLoad,
 }
 
 impl App {
@@ -166,7 +172,17 @@ impl App {
             discovery::discover_project_groups_with_root_map();
         let raw_groups = Arc::new(raw_groups);
         let session_map = Arc::new(session_map);
-        let (merged_cache, index) = load_data(&raw_groups, &session_map);
+        let (merged_cache, index, mut cache_load_state) = load_data(&raw_groups, &session_map);
+        // Debug escape hatch for screenshotting the banners after a
+        // migration already happened. Set CCMETER_FORCE_BANNER=recovered
+        // for the corruption banner, anything else for the migration one.
+        if let Ok(kind) = std::env::var("CCMETER_FORCE_BANNER") {
+            cache_load_state = if kind.eq_ignore_ascii_case("recovered") {
+                cache::CacheLoad::Recovered
+            } else {
+                cache::CacheLoad::Migrated
+            };
+        }
 
         let (daily_tokens, thresholds) = compute_daily_and_thresholds(&merged_cache, None, None);
         let minute_tokens = index.build_minute_tokens(None, None);
@@ -258,6 +274,7 @@ impl App {
             refresh_requested: false,
             update_info: None,
             update_rx,
+            cache_load_state,
         };
         app.record_rate_history();
         app
@@ -589,6 +606,12 @@ impl App {
             return false;
         }
 
+        // Dismiss the cache-state notice on the first user keypress.
+        if self.cache_load_state != cache::CacheLoad::Fresh {
+            self.cache_load_state = cache::CacheLoad::Fresh;
+            self.render_dirty = true;
+        }
+
         // Tab/BackTab cycle time filters globally, except in Settings/RateTracking.
         if !matches!(self.view, View::Settings(_) | View::RateTracking) {
             match key.code {
@@ -879,21 +902,21 @@ fn compute_hit_tokens(hits: &mut [RateLimitHit], index: &EventIndex) {
 fn load_data(
     raw_groups: &[discovery::ProjectGroup],
     session_map: &HashMap<String, (String, String)>,
-) -> (cache::Cache, EventIndex) {
+) -> (cache::Cache, EventIndex, cache::CacheLoad) {
     let all_session_files: Vec<std::path::PathBuf> = raw_groups
         .iter()
         .flat_map(|g| g.sources.iter())
         .flat_map(|s| s.session_files.iter().cloned())
         .collect();
     let events = parser::parse_session_files(&all_session_files);
 
-    let old_cache = cache::load();
+    let outcome = cache::load();
     let fresh_cache = cache::from_events(&events, session_map);
-    let merged = cache::merge(old_cache, &fresh_cache);
+    let merged = cache::merge(outcome.cache, &fresh_cache);
     cache::save(&merged);
 
     let index = EventIndex::build(&events, session_map);
-    (merged, index)
+    (merged, index, outcome.state)
 }
 
 fn spawn_reload(
@@ -905,7 +928,7 @@ fn spawn_reload(
     let session_map = Arc::clone(session_map);
     let tx = tx.clone();
     std::thread::spawn(move || {
-        let (cache, index) = load_data(&raw_groups, &session_map);
+        let (cache, index, _) = load_data(&raw_groups, &session_map);
         let _ = tx.send((cache, index));
     });
 }