diff --git a/SUMMARY.md b/SUMMARY.md new file mode 100644 index 0000000..ba5e160 --- /dev/null +++ b/SUMMARY.md @@ -0,0 +1,116 @@ +# Context Rot Compaction Targets — Executive Summary + +## What this is + +A single pure function: `compact_to(current_tokens, model, task, dial) → int` + +Given how many tokens you have, which model you're using, what kind of task you're doing, and where you want to sit on the recall/space tradeoff, it returns the number of tokens you should compact down to. It doesn't know or care how your compactor works. It just gives you the number. + +The output is always strictly less than the input. + +## What it's built on + +The function encodes empirical degradation curves from seven published sources: + +| Source | Date | Key finding | +|--------|------|-------------| +| Chroma "Context Rot" | Jul 2025 | All 18 tested models degrade at every input length increment | +| Mejba Ahmed testing | Mar 2026 | Opus 4.6: ~2%/100K degradation (100K→2%, 200K→4%, 500K→10%, 1M→14%) | +| Elvex Benchmarks | 2026 | Claude Sonnet 4 <5% degradation at 200K; most models drop sharply ~130K | +| LongCodeBench | Feb 2026 | Gemini 2.5 Pro >90% at 512K with MC options, ~50% without | +| Paulsen MECW | Jan 2026 | Effective context window is task-specific; complex reasoning fails at 1/100th of simple retrieval | +| Shi et al. | Feb 2025 | Optimal context length is bounded by training dataset size | +| Anthropic benchmarks | Mar 2026 | Opus 4.6 NIAH 91.9→78.3 across 1M; MRCR v2 76% at 1M | + +## How it works + +**Step 1: Sweet spot.** For each (model, task) pair, binary-search the degradation curve for the token count where estimated quality retention = 90%. This is the "sweet spot" — the point where the model is still performing well but is starting to bend. Because harder tasks amplify degradation (via a task multiplier), the sweet spot shifts lower for harder tasks. For example, Claude Opus 4.6's sweet spot for simple retrieval is ~500K tokens, but for coding it's ~200K, and for multi-hop reasoning it's ~167K. + +**Step 2: Dial.** The dial (0.0–1.0) slides the target between three anchors: + +| Dial | Target | Meaning | +|------|--------|---------| +| 0.0 | `floor` (8K default) | Maximum compression. Nuke it. | +| 0.5 | sweet spot | Balanced. Quality ~90% at this point. | +| 1.0 | `current × 0.90` | Minimal trim. ~10% space savings. | + +The mapping is piecewise-linear: `[0, 0.5]` interpolates between floor and sweet spot, `[0.5, 1.0]` interpolates between sweet spot and the 90% ceiling. + +**Step 3: Clamp.** Output is clamped to `[floor, current_tokens - 1]`. + +## Key numbers + +Balanced targets (dial=0.5) for coding at common context lengths: + +| Model | 128K → | 256K → | 500K → | 1M → | +|-------|--------|--------|--------|------| +| Claude Opus 4.6 | 96K | 200K | 200K | 200K | +| Claude Sonnet 4.6 | 96K | 150K | 150K | 150K | +| Gemini 2.5 Pro | 82K | 82K | 82K | 82K | +| Gemini 2.5 Flash | 64K | 64K | 64K | 64K | +| Gemini 3.0 Pro | 100K | 100K | 100K | 100K | +| GLM-4.7 | 64K | — | — | — | +| GLM-4.6 | 51K | — | — | — | + +Note how Opus 4.6's target scales with input size (because its sweet spot is high enough to move with the input), while Gemini's targets saturate early (sweet spot is low, so even at 1M you'd compact back to ~82K at balanced). + +## Confidence levels + +- **Claude Opus 4.6 / Sonnet 4.6**: High. Multiple independent benchmarks (Mejba, Elvex, Anthropic MRCR/NIAH, Chroma). +- **Gemini 2.5 Pro / 3.0 Pro**: Medium-high. LongCodeBench and Chroma data; Flash has less independent testing. +- **GLM-4.7 / 4.6**: Low. No published RULER/MRCR at scale. Estimates based on model class and 200K window boundary. +- **Task multipliers**: Medium. Derived from Chroma's semantic vs lexical gap (~2×), LongCodeBench's MC vs open gap, and Paulsen's task-specificity findings. The exact multipliers (1.0–3.0) are interpolated, not directly measured per-model. + +## API + +```python +from compaction_target import compact_to, quality_at + +# Known model +target = compact_to(250_000, model="claude-opus-4.6", task="coding", dial=0.5) + +# Unknown model — just pass the context window size +target = compact_to(250_000, max_context=512_000, task="coding", dial=0.5) + +# Unknown model name with fallback +target = compact_to(250_000, model="deepseek-r2", max_context=128_000, task="coding") + +# Aggressive +target = compact_to(250_000, model="claude-opus-4.6", task="coding", dial=0.0) # → 8,000 + +# Conservative +target = compact_to(250_000, model="claude-opus-4.6", task="coding", dial=1.0) # → 225,000 + +# Quality inspection +q = quality_at(250_000, task="coding", model="claude-opus-4.6") # → 0.858 +q = quality_at(250_000, task="coding", max_context=512_000) # → generic estimate + +# All dial points at once +from compaction_target import compact_to_range +targets = compact_to_range(250_000, model="gemini-2.5-pro", task="coding") +``` + +## Unknown models: the `max_context` parameter + +For models not in the profile database, pass `max_context` (the model's advertised +context window in tokens). This generates a conservative generic degradation curve: + +``` +quality = 1.0 - 0.30 × (tokens / max_context)^0.6 +``` + +This shape sits between the best (Claude Opus) and worst (Gemini Flash) profiled +models — a deliberate "assume median" stance. If `model` is given but not recognized +and `max_context` is also given, the generic profile is used as fallback. + +The generic profile is intentionally pessimistic for harder tasks. At 50% of max +context, the generic model estimates 80% quality for retrieval, 50% for coding, and +41% for reasoning. This means unknown models get aggressive compaction targets — which +is the right default when you don't have benchmark data to justify keeping more context. + +## What this doesn't do + +- It doesn't call your compactor. It just gives you a number. +- It doesn't model compaction quality. It doesn't know if your compactor is good or bad. +- It doesn't decide *when* to compact. It only answers "to how many tokens." +- It doesn't handle the trigger logic ("am I past the threshold?"). That's your call. diff --git a/compact.py b/compact.py index 5bf1500..f45b83f 100644 --- a/compact.py +++ b/compact.py @@ -68,6 +68,26 @@ def cmd_compact(args: argparse.Namespace) -> int: if total_tokens <= args.budget: console.print(f"[green]Already within budget ({total_tokens:,} <= {args.budget:,}), nothing to compact.[/green]") + if args.output: + from lib.selector import SelectionResult + # Wrap in SelectionResult and just use all turns as kept_turns + result = SelectionResult( + kept_turns=turns, + kept_scored=[], + dropped_turns=[], + budget=args.budget, + short_threshold=args.short_threshold, + user_tokens=sum(token_counts.get(t.index, 0) for t in user_turns), + short_system_tokens=sum(token_counts.get(t.index, 0) for t in short_system), + scored_kept_tokens=0, + scored_dropped_tokens=0, + total_input_tokens=total_tokens, + ) + fmt = getattr(args, "format", "jsonl") + if fmt == "summary": + write_summary_text(result, args.output) + else: + write_compacted_jsonl(result, args.output) return 0 # Identify long system turns that need scoring diff --git a/compaction_target.py b/compaction_target.py new file mode 100644 index 0000000..76805a3 --- /dev/null +++ b/compaction_target.py @@ -0,0 +1,356 @@ +""" +compaction_target.py +==================== + +Single pure function: given current token count, model (or just a max context +length), task type, and a recall/space dial, returns the target token count +to compact down to. + +Compactor-agnostic. Just outputs a number. Guaranteed less than current tokens. + +Empirical basis: + - Chroma "Context Rot" (Jul 2025): 18 models, degradation at every length. + - Mejba Ahmed (Mar 2026): Opus 4.6 ~2% degradation per 100K tokens. + - Elvex 2026: Claude Sonnet 4 <5% degradation across 200K. + - LongCodeBench (Feb 2026): Gemini 2.5 Pro >90% at 512K (MC), ~50% (open). + - Paulsen (Jan 2026): MECW is task-specific; complex reasoning fails early. + - Shi et al. (Feb 2025): optimal context length bounded by training data. + - Anthropic NIAH: Opus 4.6 scores 91.9→78.3 across 1M context. + - Anthropic MRCR v2: Opus 4.6 76% at 1M (vs Sonnet 4.5 18.5%). + +Usage: + from compaction_target import compact_to + + # Known model: + target = compact_to(250_000, model="claude-opus-4.6", task="coding", dial=0.5) + + # Unknown model — just pass max context length: + target = compact_to(250_000, max_context=512_000, task="coding", dial=0.5) + + # target is always < current_tokens + my_compactor(context, target) + +The dial: + 0.0 = maximize space savings (aggressive, lose more recall) + 0.5 = balanced + 1.0 = maximize recall preservation (conservative, keep more tokens) +""" + +from __future__ import annotations +from bisect import bisect_right +from typing import Optional + + +# ───────────────────────────────────────────────────────────────────────────── +# Degradation profiles: (token_count, quality_retention) anchors +# Piecewise-linear interpolation. All values for simple retrieval baseline; +# task multiplier scales degradation from here. +# ───────────────────────────────────────────────────────────────────────────── + +_PROFILES: dict[str, tuple[tuple[int, float], ...]] = { + "claude-opus-4.6": ( + (0, 1.0), (64_000, .99), (100_000, .98), (200_000, .96), + (300_000, .93), (500_000, .90), (750_000, .87), (1_000_000, .86), + ), + "claude-sonnet-4.6": ( + (0, 1.0), (64_000, .98), (100_000, .97), (200_000, .95), + (300_000, .91), (500_000, .87), (750_000, .83), (1_000_000, .80), + ), + "claude-sonnet-4": ( + (0, 1.0), (64_000, .98), (100_000, .97), (200_000, .95), + ), + "claude-opus-4": ( + (0, 1.0), (64_000, .99), (100_000, .98), (200_000, .96), + ), + "gemini-2.5-pro": ( + (0, 1.0), (64_000, .97), (100_000, .95), (128_000, .93), + (200_000, .90), (256_000, .88), (500_000, .82), (750_000, .75), + (1_000_000, .70), + ), + "gemini-2.5-flash": ( + (0, 1.0), (64_000, .96), (100_000, .93), (128_000, .90), + (200_000, .86), (256_000, .83), (500_000, .75), (750_000, .68), + (1_000_000, .63), + ), + "gemini-3.0-pro": ( + (0, 1.0), (64_000, .98), (100_000, .96), (128_000, .94), + (200_000, .91), (256_000, .89), (500_000, .83), (750_000, .77), + (1_000_000, .73), + ), + "glm-4.7": ( + (0, 1.0), (64_000, .96), (100_000, .93), (128_000, .90), + (200_000, .85), + ), + "glm-4.6": ( + (0, 1.0), (64_000, .95), (100_000, .91), (128_000, .88), + (200_000, .82), + ), +} + +_TASK_MULT: dict[str, float] = { + "retrieval": 1.0, "semantic": 1.5, "summary": 1.8, + "chat": 2.0, "coding": 2.5, "reasoning": 3.0, +} + +_ALIASES: dict[str, str] = { + "opus": "claude-opus-4.6", "opus-4.6": "claude-opus-4.6", + "opus-4": "claude-opus-4", + "sonnet": "claude-sonnet-4.6", "sonnet-4.6": "claude-sonnet-4.6", + "sonnet-4": "claude-sonnet-4", + "gemini-pro": "gemini-2.5-pro", "gemini-flash": "gemini-2.5-flash", + "gemini-3": "gemini-3.0-pro", "glm": "glm-4.7", +} +_TASK_ALIASES: dict[str, str] = { + "code": "coding", "agent": "coding", "coding_agent": "coding", + "code_generation": "coding", "agentic": "coding", + "multi_hop": "reasoning", "multi_hop_reasoning": "reasoning", + "simple_retrieval": "retrieval", "semantic_retrieval": "semantic", + "summarization": "summary", "conversation_qa": "chat", + "conversation": "chat", "qa": "chat", +} + + +# ───────────────────────────────────────────────────────────────────────────── +# Generic profile for unknown models +# ───────────────────────────────────────────────────────────────────────────── +# Shape based on median across all profiled models: +# quality = 1.0 - 0.30 * (tokens / max_context)^0.6 +# Conservative — sits between Claude (best) and Flash (worst). + +def _generic_profile(max_context: int) -> tuple[tuple[int, float], ...]: + """Generate a degradation profile from just a max context window size.""" + fracs = (0.0, 0.05, 0.10, 0.20, 0.30, 0.50, 0.70, 0.85, 1.0) + return tuple( + (int(max_context * f), round(1.0 - 0.30 * (f ** 0.6), 4)) + for f in fracs + ) + + +# ───────────────────────────────────────────────────────────────────────────── +# Internals +# ───────────────────────────────────────────────────────────────────────────── + +def _lerp(profile: tuple[tuple[int, float], ...], tokens: int) -> float: + if tokens <= 0: + return 1.0 + ts = [p[0] for p in profile] + if tokens >= ts[-1]: + t1, v1 = profile[-2] + t2, v2 = profile[-1] + slope = (v2 - v1) / (t2 - t1) if t2 != t1 else 0.0 + return max(0.3, v2 + slope * (tokens - t2)) + i = max(0, min(bisect_right(ts, tokens) - 1, len(profile) - 2)) + t1, v1 = profile[i] + t2, v2 = profile[i + 1] + frac = (tokens - t1) / (t2 - t1) if t2 != t1 else 0.0 + return v1 + frac * (v2 - v1) + + +def _resolve(model: str) -> Optional[str]: + k = model.lower().strip() + if k in _PROFILES: + return k + if k in _ALIASES: + return _ALIASES[k] + for key in _PROFILES: + if key in k or k in key: + return key + return None + + +def _task_mult(t: str) -> float: + k = t.lower().strip().replace(" ", "_").replace("-", "_") + if k in _TASK_MULT: + return _TASK_MULT[k] + if k in _TASK_ALIASES: + return _TASK_MULT[_TASK_ALIASES[k]] + return 2.5 + + +def _get_profile( + model: Optional[str], max_context: Optional[int], +) -> tuple[tuple[int, float], ...]: + if model is not None: + key = _resolve(model) + if key is not None: + return _PROFILES[key] + if max_context is not None and max_context > 0: + return _generic_profile(max_context) + if model is not None: + raise ValueError( + f"Unknown model {model!r} and no max_context provided. " + f"Known: {list(_PROFILES)}. Pass max_context for unknown models." + ) + raise ValueError("Provide either model or max_context.") + + +def _quality( + profile: tuple[tuple[int, float], ...], tokens: int, task: str, +) -> float: + base = _lerp(profile, tokens) + return max(0.3, 1.0 - (1.0 - base) * _task_mult(task)) + + +def _find_crossing( + profile: tuple[tuple[int, float], ...], + task: str, target_q: float, lo: int, hi: int, +) -> int: + if _quality(profile, lo, task) < target_q: + return lo + if _quality(profile, hi, task) >= target_q: + return hi + for _ in range(30): + mid = (lo + hi) // 2 + if lo >= hi - 1: + break + if _quality(profile, mid, task) >= target_q: + lo = mid + else: + hi = mid + return lo + + +# ───────────────────────────────────────────────────────────────────────────── +# Public API +# ───────────────────────────────────────────────────────────────────────────── + +def quality_at( + tokens: int, + task: str = "coding", + model: Optional[str] = None, + max_context: Optional[int] = None, +) -> float: + """Estimated quality retention at a given token count. Returns 0.0–1.0. + + Provide model (for known profile) or max_context (for generic), or both. + """ + return _quality(_get_profile(model, max_context), tokens, task) + + +def compact_to( + current_tokens: int, + model: Optional[str] = None, + task: str = "coding", + dial: float = 0.5, + floor: int = 8_000, + max_context: Optional[int] = None, +) -> int: + """Compute the target token count to compact to. + + Args: + current_tokens: Current context size in tokens. + model: Known model name (optional if max_context given). + task: "retrieval" | "semantic" | "summary" | "chat" | "coding" | "reasoning" + dial: 0.0 (aggressive) to 1.0 (conservative). Default 0.5 (balanced). + floor: Minimum output. Default 8_000. + max_context: Max context window in tokens. Required for unknown models. + Ignored when model resolves to a known profile. + + Returns: + int, always satisfying: floor <= result < current_tokens. + """ + if current_tokens <= floor: + return current_tokens + + dial = max(0.0, min(1.0, dial)) + profile = _get_profile(model, max_context) + prof_max = profile[-1][0] + + sweet = _find_crossing( + profile, task, 0.90, lo=floor, hi=min(current_tokens, prof_max), + ) + if sweet >= current_tokens: + sweet = max(floor, int(current_tokens * 0.75)) + + ceiling = max(floor + 1, int(current_tokens * 0.90)) + if dial <= 0.5: + target = floor + (dial / 0.5) * (sweet - floor) + else: + target = sweet + ((dial - 0.5) / 0.5) * (ceiling - sweet) + + return max(floor, min(int(target), current_tokens - 1)) + + +def compact_to_range( + current_tokens: int, + model: Optional[str] = None, + task: str = "coding", + floor: int = 8_000, + max_context: Optional[int] = None, +) -> dict[str, int]: + """Return targets at dial=0.0, 0.25, 0.5, 0.75, 1.0 in one call.""" + return { + f"{d:.2f}": compact_to(current_tokens, model, task, d, floor, max_context) + for d in (0.0, 0.25, 0.5, 0.75, 1.0) + } + + +# ───────────────────────────────────────────────────────────────────────────── +# CLI +# ───────────────────────────────────────────────────────────────────────────── + +if __name__ == "__main__": + DS = [0.0, 0.25, 0.5, 0.75, 1.0] + TS = [128_000, 256_000, 500_000, 1_000_000] + + def _row(name, tokens, task, **kw): + vals = [compact_to(tokens, task=task, dial=d, **kw) for d in DS] + cols = "".join(f" {v:>8,} |" for v in vals) + return f" {name:<22} |{cols}" + + def _header(): + h = f" {'':22} |" + for d in DS: + h += f" {'d='+str(d):>8} |" + return h + "\n " + "─" * 23 + "┼" + ("─" * 10 + "┼") * 4 + "─" * 10 + "┤" + + print("compact_to() — Target token counts") + print("=" * 90) + + for task in ("coding", "reasoning"): + # Known models + print(f"\n{'─'*90}\n TASK: {task} — known models\n{'─'*90}") + known = [ + ("claude-opus-4.6", {"model": "claude-opus-4.6"}), + ("claude-sonnet-4.6", {"model": "claude-sonnet-4.6"}), + ("gemini-2.5-pro", {"model": "gemini-2.5-pro"}), + ("gemini-2.5-flash", {"model": "gemini-2.5-flash"}), + ("glm-4.7", {"model": "glm-4.7"}), + ] + for tokens in TS: + print(f"\n Current: {tokens:,}") + print(_header()) + for name, kw in known: + p = _get_profile(kw.get("model"), None) + if tokens <= p[-1][0]: + print(_row(name, tokens, task, **kw)) + + # Generic models + print(f"\n{'─'*90}\n TASK: {task} — generic (max_context only)\n{'─'*90}") + generics = [ + ("generic 128K", 128_000), + ("generic 256K", 256_000), + ("generic 512K", 512_000), + ("generic 1M", 1_000_000), + ("generic 2M", 2_000_000), + ] + for tokens in TS: + print(f"\n Current: {tokens:,}") + print(_header()) + for name, mc in generics: + if tokens <= mc: + print(_row(name, tokens, task, max_context=mc)) + + # Generic profile shape + print(f"\n{'─'*90}") + print(" GENERIC PROFILE: quality = 1.0 - 0.30 * (tokens/max_context)^0.6") + print(f"{'─'*90}\n") + print(f" {'%':>5} | {'Retrieval':>10} | {'Coding':>10} | {'Reasoning':>10}") + print(f" {'─'*5}─┼─{'─'*10}─┼─{'─'*10}─┼─{'─'*10}") + for pct in (5, 10, 20, 30, 50, 70, 85, 100): + f = pct / 100 + base = 1.0 - 0.30 * (f ** 0.6) + r = max(0.3, 1.0 - (1.0 - base) * 1.0) + c = max(0.3, 1.0 - (1.0 - base) * 2.5) + rs = max(0.3, 1.0 - (1.0 - base) * 3.0) + print(f" {pct:>4}% | {r:>9.1%} | {c:>9.1%} | {rs:>9.1%}") diff --git a/lib/formatter.py b/lib/formatter.py index a824d71..4b42f11 100644 --- a/lib/formatter.py +++ b/lib/formatter.py @@ -161,6 +161,29 @@ def write_summary_text(result: SelectionResult, output_path: Path) -> None: def write_compacted_jsonl(result: SelectionResult, output_path: Path) -> None: """Write kept turns back to a JSONL file.""" + + # Fix the parentUuid chain so claude-code can load the transcript + # without breaking at missing messages. + last_uuid = None + for turn in result.kept_turns: + # Find the first message in this turn to link to the previous turn + first_msg_idx = -1 + for i, record in enumerate(turn.lines): + if isinstance(record, dict) and record.get("type") in ("user", "assistant", "system", "attachment"): + first_msg_idx = i + break + + if first_msg_idx >= 0 and last_uuid is not None: + # We ONLY rewrite parentUuid if it already exists, to avoid touching root nodes + if "parentUuid" in turn.lines[first_msg_idx]: + turn.lines[first_msg_idx]["parentUuid"] = last_uuid + + # Find the last message in this turn to be the parent for the next turn + for record in reversed(turn.lines): + if isinstance(record, dict) and "uuid" in record and record.get("type") in ("user", "assistant", "system", "attachment"): + last_uuid = record["uuid"] + break + with open(output_path, "w") as f: for turn in result.kept_turns: for record in turn.lines: diff --git a/plugins/claude-code/hooks-handlers/supercompact-precompact.sh b/plugins/claude-code/hooks-handlers/supercompact-precompact.sh index 99890f6..0e70ec3 100755 --- a/plugins/claude-code/hooks-handlers/supercompact-precompact.sh +++ b/plugins/claude-code/hooks-handlers/supercompact-precompact.sh @@ -107,7 +107,11 @@ if uv run python compact.py "${JSONL_FILE}" \ if command -v unleash-refresh &>/dev/null; then echo "$(date -Iseconds) Restarting via unleash-refresh" >> "${LOG_DIR}/hook.log" unleash-refresh "COMPACT COMPLETE. Previous context has been summarized. Continue with your current task." - # If unleash-refresh returns (shouldn't normally), exit cleanly + + # Block to ensure Claude Code processes the SIGINT and shuts down BEFORE this + # script exits. This prevents a race condition where the hook completes and + # Claude starts the API compaction call before the SIGINT is fully handled. + sleep 10 exit 0 else echo "$(date -Iseconds) WARNING: unleash-refresh not found — compacted JSONL is on disk but Claude's API compact will still run over it" >> "${LOG_DIR}/hook.log"